Takiri's Game

In this post I thought I'd talk a little bit about some of the other activity I've been engaged in with ATC.

Computer games have become very popular in recent years, as forms of entertainment but also for more serious applications. Geoff's investigating how video games or simulations can be used as training tools. Over on his blog I posted several comments that I'd like to reiterate here as they're relevant to my work on Second Life too. One of the principle areas I identified as being compelling for getting blind users online is the great interest educational establishments have in using virtual worlds as learning tools. This becomes especially important when students are physically unable to attend class as in distance learning or if the student has physical mobility issues.

The question Geoff starts with is "What is a game?" This actually turns out to be important for SL too. A lot of people might mistake it for a game due to the way it looks, but paying a bit more attention suggests the differences that make SL a Multi User Virtual Environment (MUVE) rather than a game.

I start by pointing out some of the discussions that have been important within the game studies community,

Interesting post!

I did my MA on computer games, and in the field of game studies / ludology there are a number of definitions that might be of interest to you. Juul [2003] would be a good place to start. He’s a widely acclaimed ‘ludologist’ (theorist of game studies), and he also includes definitions from other significant scholars. Pay particular attention to Roger Caillois (for an in-depth analysis of the relationships between play and games), Sutton Smith (life-long scholar of play), and Johan Huizinga (classic analysis of the relationships between play and culture more broadly). Finally two that I haven’t read myself but which are also highly recommended, Katie Salen & Eric Zimmerman (widely cited video game designers and scholars), and John Paul Gee (video games and learning).

All of these are top-notch reads, but they’re also all quite involved books so might be far too much information! On the off chance that you are interested in taking this further, I think you might be able to find some of them in the university library.

You might also find something relevant in The Journal of Simulation and Gaming (http://sag.sagepub.com/) to which I think the university provides an electronic subscription.

References:

Caillois, Roger. “Man, play, and games”. The Free Press, Glencoe, New York, 1961 (1958)

Gee, J. P. (2003). “What Video Games Have to Teach Us About Learning and Literacy”. Palgrave Macmillan.

Huizinga, Johan. “Homo Ludens - a study of the play element in culture”. The Beacon Press, Boston, 1950 (1944).

Juul, Jesper. “The Game, the Player, the World: Looking for a Heart of Gameness”
http://www.jesperjuul.net/text/gameplayerworld/

Salen, Katie & Zimmerman, Eric. “Rules of Play - Game Design Fundamentals”. MIT Press, Cambridge, 2003.

Sutton-Smith, Brian. “The Ambiguity of Play”. Cambridge, MA: Harvard University Press, 1997.

I then followed up with some further information,

I just came across a reference for Gee in the ACM which you can access from university. It looks like a 4 page summary of his book! I’m going to read it myself now, but it does sound like it might be useful for you too,

http://doi.acm.org/10.1145/950566.950595

@article{950595,
author = {James Paul Gee},
title = {What video games have to teach us about learning and literacy},
journal = {Comput. Entertain.},
volume = {1},
number = {1},
year = {2003},
issn = {1544-3574},
pages = {20–20},
doi = {http://doi.acm.org/10.1145/950566.950595},
publisher = {ACM},
address = {New York, NY, USA},
}

In addition to these references I made some further comments about additional reading, and a conference that he might be interested in too, where I expect there will be several speakers discussing SL,

2nd European Conference on Games Based Learning
The Hotel Silken Diagonal Barcelona, Barcelona, Spain
16-17 October 2008

One of the most exciting things for me as a student is being able to meet other people who have similar interests. As computer games (and simulations / MUVEs) become more mainstream the potential for the kind of cross-disciplinary collaboration increases exponentially.

At the HCT Postgrad Workshop last year I was amazed at the number of people studying computer games. It's a really exciting time to be involved in this area. I wonder how my experience differs from students in other fields though. Perhaps as this area is so young it's a lot easier to find a place to innovate in. Alternatively maybe we as researchers are more likely to follow dead-ends as there have been so few trailblazers.

My conclusion to this post is to emphasise community collaboration. I was very happy to be able to help Geoff with his research, and to meet and learn from other games researchers at the workshop last year. The spirit of open discussion on forums, blogs and mailing lists, manifest by the increase in conferences and journals, is probably the single most important feature of our burgeoning community. In other words: advanced technical communication.

Current community discussions

I've reviewed the literature and have identified major themes that are relevant to my work:

Audio for Navigation
Researchers at Helsinki University of Technology have produced some interesting work on using audio in virtual environments. Their experiments suggest that audio is useful for locating objects at a distance, but is a poor substitute for vision at close range.

@article{1101558,
author = {Matti Gr\"{o}hn and Tapio Lokki and Tapio Takala},
title = {Comparison of auditory, visual, and audiovisual navigation in a 3D space},
journal = {ACM Trans. Appl. Percept.},
volume = {2},
number = {4},
year = {2005},
issn = {1544-3558},
pages = {564--570},
doi = {http://doi.acm.org/10.1145/1101530.1101558},
publisher = {ACM},
address = {New York, NY, USA},
}

@article{1058321,
author = {Tapio Lokki and Matti Grohn},
title = {Navigation with Auditory Cues in a Virtual Environment},
journal = {IEEE MultiMedia},
volume = {12},
number = {2},
year = {2005},
issn = {1070-986X},
pages = {80--86},
doi = {http://dx.doi.org/10.1109/MMUL.2005.33},
publisher = {IEEE Computer Society Press},
address = {Los Alamitos, CA, USA},
}

@misc{ lokki00case,
author = "T. Lokki and M. Gr and o Savioja and T. Takala",
title = "A case study of auditory navigation in virtual acoustic environments",
text = "T. Lokki, M. Grohn, L. Savioja, and T. Takala, \A case study of auditory
navigation in virtual acoustic environments, Proc. ICAD 2000 ,(Atlanta GA,),
Apr 2000.",
year = "2000",
url = "citeseer.ist.psu.edu/lokki00case.html" }

@misc{ hn-utilizing,
author = "Matti Gr\"ohn",
title = "Utilizing Audio in Immersive Visualization",
url = "citeseer.ist.psu.edu/452324.html" }

@article{1101559,
author = {Matti Gr\"{o}hn and Tapio Lokki and Tapio Takala},
title = {Author's comments on Gr\öhn, Lokki, and Takala, ICAD 2003},
journal = {ACM Trans. Appl. Percept.},
volume = {2},
number = {4},
year = {2005},
issn = {1544-3558},
pages = {571--573},
doi = {http://doi.acm.org/10.1145/1101530.1101559},
publisher = {ACM},
address = {New York, NY, USA},
}

Audio for real world navigation
There are a couple of different applications that seem interesting, both concerned with using maps of the real world. One approach uses 3D audio presented to the user of a GPS device as a guide to their exploration, a technique which would be easy to implement in SL. The other interesting technique is the audible representation of a map that the user can familiarise themselves with in order to help build up a mental model of a real world location prior to visiting. Similarly this should be relatively easy to implement in SL.

@inproceedings{1182492,
author = {Wilko Heuten and Daniel Wichmann and Susanne Boll},
title = {Interactive 3D sonification for the exploration of city maps},
booktitle = {NordiCHI '06: Proceedings of the 4th Nordic conference on Human-computer interaction},
year = {2006},
isbn = {1-59593-325-5},
pages = {155--164},
location = {Oslo, Norway},
doi = {http://doi.acm.org/10.1145/1182475.1182492},
publisher = {ACM},
address = {New York, NY, USA},
}


Real World GPS / Audio Maps
Games
Physical properties in the virtual world

Second Life Accessibility
A number of people are already discussing the issues and the following trends seem to come up:
Visual impairment.
Most people who are registered as legally blind do have some degree of sight left, so the most common issues are not so much to do with navigating without sight, but rather adapting the display to their individusal needs. This means being able to modify colours and their contrast for the colour-blind, and adjust magnification or screen resolution for low-vision users. These are simple tasks that could be accomplished with relatively little technical work, but my research project takes on the larger and more theoretical task of enabling access to users without the use of vision at all.

Alternate clients
This theme is largely oriented around making SL accessible to screen readers, with the leader being SLeek. These projects only expose a small portion of SL, typically limiting the information conveyed to lists of local objects and avatars, a chat interface and teleportation facility. This approach is the fastest way to get blind users onto Second Life, but gives them little more than a new instant messaging client.

GuideBot
In the real world it is not uncommon for a sighted person to guide the blind, especially in new environments. This theme takes the principle of assistance and tries to automate it by using software-controlled avatars (or "bots"). Josh Markwordt's project is one example, though I believe the complexity of SL, and the ability of users to search for and teleport to specific locations, will limit the usefulness of this approach.

Virtual Representation
Disabled people have a presence in SL already, but these are mostly people who have restricted mobility. Simon Walsh is the owner of Wheelies, a disability-themed nightclub. One of the issues that comes up is the representation of physical disabilities in the virtual world. Avatars can be customised to represent oneself in any way possible, and there is a trend amongst wheelchair users to incorporate representations of chairs in their virtual identities.

For a comprehensive review of the literature, see my other posts:
SLED Accessibility Threads
Disability in SL
Mailing List Fora
Accessibility Analysis


Haptics
Jeff VanDrimmelen, an Academic Computing Expert in the Office of Arts and Sciences Information Services, University of North Carolina at Chapel Hill publishes research on a site called Haptic Education - Adding the Tactile Sensation to Virtual Learning.


VanDrimmelen's team have focused on another virtual environment called Croquet, but have also considered Second Life and make some interesting observations,

The creators of Second Life actually started their project out with a large haptic device, but soon abandoned it for more financially appealing options.

In Second Life the only way to navigate with a mouse is to bring up an on screen navigation menu that you have to click to move the avatar. It works okay when the avatar is flying, but otherwise you just end up using the buttons on the handle to move around. However, just in case anyone wants to work with the script, here it is.

In Linden's default client movement is controlled using the keyboard, but in my own research I have recently been able to control by walking and flying using a force feedback joystick (Logitech Wingman Strike Force 3D). This was made possible by using a free 3rd party tool called GlovePIE which VanDrimmelen's team also employed. The tool works by intercepting output from the joystick and injecting the corresponding keyboard signals, such that by moving the joystick left and right the Second Life avatar turns left and right, and moving the joystick forward and backwards moves the avatar forward and back. The same technique is used by VanDrimmelen's team to use the Novint Falcon as input device for Croquet. This approach appears to offer a very quick and easy way to prototype haptics in Second Life. VanDrimmelen continues, however,

It should be noted that about the same time we found the GlovePIE software Novint announced they are working on drivers that will work with not only Second Life, but World of Warcraft as well.

Currently both of these drivers are "in exploration phase" with no estimated completion date. Also in their (busy!) release schedule Novint also describe another interesting product, "Feelin' It: Blind Games™":

Novint will release a number of games that can be played entirely without sight. For example, in a bowling game, you will be able to feel the extents of the lane, feel the weight of the ball as it is thrown, and hear the pins crash down. After throwing the ball and hitting the pins, the game will bring up a touchable representation of how the ball traveled down the lane to guide the user's muscle memory for future shots, and the user will be able to feel with a 3D cursor which pins are still standing. All the information needed to play the game and become a true master, will be available without any graphics.

Further haptic research in Second Life is being conducted by Maurizio de Pascale, Sara Mulatto, Domenico Prattichizzo from the Haptics Group of the Siena Robotics and Systems Lab, in the Dipartimento di Ingegneria Informatica at the University of Siena. In particular they have a paper called "Bringing Haptics to Second Life: A Haptics-enabled Second Life Viewer for Blind Users", which is due for publication at the "Haptic in Ambient Systems" conference, which takes place in Quebec City, Canada on February 11-14, 2008.

Judging from the screenshot, I would imagine that the Siena team are not using the Novint, but rather a different haptic device that has a stylus, perhaps one of SensAble Technology's Phantom range which seem popular in academic research.

Another research project that is of interest as inspiration for our Second Life work is the Haptic Torch from the Interactive Systems Research Group at the University of Reading.

"The unique design of the torch allows users to range from sighted individuals in low-light conditions to people who are both deaf and blind. The torch provides a method of alerting users to presence of potentiol hazards using non-contact measurement techniques. An subtle tactile (touch) interface conveys relevent information to the user while not interfering with other senses." [sic]

Whereas the Haptic Torch is only capable of signifying the presence of objects, the Falcon could be used to reach out and feel their shape, and this immediate physical stimuli will assist the users construction of a mental map of the virtual space.

Why Second Life?

I thought it'd be worth enumerating the motivations for this project and to explain a little bit about what people do in SL.

There's a nice summary called 12 Things To Do In Second Life That Aren't Embarrassing If Your Priest Or Rabbi Finds Out

I don't really blame the people who think there's nothing to do in Second Life. One of the areas where Second Life is weakest is in introducing newcomers to the world. The user interface is confusing, and, worse, once you've got that mastered, it's hard to figure out what to do. The newbie is confronted with an array of cybersex areas, online casinos, and sleazy make-money-fast schemes. But once you get past that initial barrier, you'll find plenty of things to do in Second Life.

I've categorised the typical activities into the following:

Visual
These pleasures are largely inaccessible to blind users. I don't know of any feasible way to overcome this problem, that blind SL users will inevitably miss out on the visual aesthetic of the world. Fortunately there's some potential to convert content into text automatically, for example when using /gestures which are like /emote commands from MUDs. These are quite often used and constitute an important form of communication in themselves. Indeed the categories I'm using here are not intended as strict divisions but rather as conceptual guides for non-players to get an idea of what kinds of things are possible and common.

• See the sights (Visit Amsterdam)
• Dancing

Content Creation
Inaccessible, but with some work scripting might be possible (see also Blind Programming). I have also had the pleasure to interview a blind artist called Amaya Summers. She's legally blind but has sufficient vision to be able to work with the SL client. The fact that she has a job in SL gives here a sense of purpose, structure and meaning that would be difficult to find in the outside world. As with all of these categories there is a large degree of overlap. In this case content creation and trade are tightly bound, and as most content is visual in nature it has a strong relationship to visual aesthetic, and finally the visual aesthetic of the way one customises one's avatar, land, buildings and props is itself a form of communication similar to body language and fashion in the outside world.

• Building and creating things

Trade

Important as a means of self-expression, but mostly oriented around the Visual, making it mostly inaccessible.

• Doing business
• Shopping

Communication
Clearly a major draw for everyone. Particularly compelling for the visually impaired if the interface was made compatible with screen readers. Often blind users find it hard to make new friends, so it would be great if we could make use of SL's potential to act as a venue for new friendships to form. In this category I mean to include everything from learning environments like universities, to performances such as live music streaming, etc.

• Talk to other people
• Listen to live music and attend other live events

Games
A sub-category of Social, and clearly compelling for everyone, perhaps less accessible due to game-specific visual qualities. This allows me to point out that while SL can be used for games, they are not a necessary feature of it as a Multi User Virtual Environment.

• Role-playing games
• Other kinds of games

Citations

http://www.blogger.com/img/gl.link.gifI've been thinking recently about citations.
I had with Geraldine towards the beginning of this year where she showed me a document containing citations for lots of papers she'd read. We discussed how to save references like these, and I mentioned that in the past I'd relied on the facilities of my OSX computer to manage this material. During my MA I had to read a great deal of literature. I took a theoretical course so it was much more about reading than anything else. My strategy back then was to organise my work using a filesystem hierarchy like so:

University
- Module Name
- - Assignment
- - - References

In the references folder I'd keep downloaded versions of everything I'd read, however briefly. If I spent some time reading a piece then I'd create a text document to accompany the downloaded version which would include the normal citation data (author, date, title, publisher, etc). Following that I'd write up one or two paragraphs about the piece, trying to summarise the important points and give some kind of qualitative indication of what I thought about it. Having to sit down and summarise someone else work was a really useful practice that I think would be well worth including in future ATC courses. It forced me to really consider what the author was saying and to identify the core ideas they used. This practice reinforced my understanding of their work and I felt that it helped me to be able to recall who they were and what they did. In addition when I came to write up my dissertation and I had a lot of references to include it was really easy for me to copy and paste the citation itself, and also to rework my summary into a format that I could include in the body of my work. I still have access to this annotated bibliography and am sure that it'll come in handy for further work that I'll be conducting on video games.

Another thing to remember is to rename files that you download. Typically when I read PDF papers they have numerical filenames (eg, "2534595143623.pdf") which are probably only significant as they key to the database from which they originate. The format I use for renaming is similar to the format you'd use for writing a reference anyway:

Surname, Firstname. [other authors.] Title

Clearly this is much more useful than just a number. It also means that I can search my harddrive very quickly (using Spotlight) to find any documents by a particular author, say.

On reflection the trouble with this technique is that I get duplicates as I would file by module name. It's not unusual to use papers from one module in another - in fact if you're not doing that then it begs the question why you're keeping references anyway...

This didn't become a big deal though.

Geraldine's technique was just to keep a single plain text file with all of the citation info for each document. Simple but effective.

Since then I've been thinking about citations, and have gone off my original method of just typing out the typical data by hand. The problem with the manual approach is that different publications have different standards for references, so you might end up having to edit the format and/or data when you come to insert it into another document. Apparently this becomes a massive task when you have to write up your thesis.

The University of Sussex do offer a course on how to use EndNote, but I couldn't get on as it was full by the time I heard it was available. I took a look at a couple of different formats and decided that BibTex is the most human-readable version of a computer format, and so is probably the best format for me to store my personal annotated bibliography in. Being human-readable is especially important for me as I have an OSX laptop and a Windows PC, so I don't want to use an application that can only run on one of those platforms.

Compare the following for readability,

BibTex:

@article{1326555,
author = {Sergey Yekhanin},
title = {Towards 3-query locally decodable codes of subexponential length},
journal = {J. ACM},
volume = {55},
number = {1},
year = {2008},
issn = {0004-5411},
pages = {1--16},
doi = {http://doi.acm.org/10.1145/1326554.1326555},
publisher = {ACM},
address = {New York, NY, USA},
}

EndNote

%0 Journal Article
%1 1326555
%A Sergey Yekhanin
%T Towards 3-query locally decodable codes of subexponential length
%J J. ACM
%@ 0004-5411
%V 55
%N 1
%P 1-16
%D 2008
%R http://doi.acm.org/10.1145/1326554.1326555
%I ACM

I'm not entirely sure which is better. EndNote is more concise, but at the expense of having to memorise or infer what the % characters indicate. Either way, if I keep my own bibliography in some standard format then at any time I should be able to convert it into another format if I so desire.

One of the nice things that I've also been doing is maintaining my own personal blogs. Including references in them is pretty handy too because I can include a direct hyperlink to the original document. Having it online is also nice because it means I don't need to worry about syncing between the two computers, nor do I have to worry about backing up my data.

In fact I have another blog that I was using to keep some notes about my DPhil project (which I haven't actually started yet), and one of the draft posts is essentially a TODO list composed of (hyperlinked) BibTex references. This is a list of documents I should read that are relevant to my project. I've been keeping this blog and the list since September last year, so it's getting pretty big now!


You'll notice that in the sample reference above a hyperlink is included as a DOI. This is important (and something I've changed to recently) because it tries to provide a constant URL where the data will always be available from. Normal URLs have a tendency to "rot" or go "stale", that is, when the document is moved the URL no longer functions. This happens over time so it's important to keep your bibliography "fresh" by using DOIs wherever possible, especiially if it's going to be some time before you refer to a particular document again.

Another technique that I've been using recently is following hyperlinked references. Often when I read a paper I want to see the other work they reference, but manually searching for these can be time consuming. That's why I was really happy to see that the ACM list citations as well as references for documents in their electronic library, and many of these are hyperlinked. Following these can be useful to find similar work (that doesn't necessarily turn up in a search, or if you don't want to trawl through all the search results).