Signs, Symbols, Tools - Revisited

I'm rereading James Gee for my Cadre CXi course and was pondering his chapter on semiotic domains. He talks about expanding notions of sign systems beyond language, the obvious one and powerful one. He mentions visual literacy. N'kay, yeah, that's good. Old, but still true. I can recall having visual literacy classes in grad school.

But...

I have already been pondering the visual literacy of the digital life: games and online chat, blogs, etc. Huge amount of symbolic transactional work: colors, symbol sets, and...audio.

Yeah, I was thinking about how neglected audio is as a symbol system. Yet when I play WoW, if I don't have the sound on, I miss when my ass is getting chewed on by some nasty NPC I've picked up whilst running behind James (whom they never touch). Audio cues let me know when I've crossed boundaries of regions. Audio cues let me know when a blow has landed on the opponent, when a spell has been cast in my vicinity, when looting is happening behind me, when my buddy has stopped to do skinning, and so on. Sound provides me with useful, sometimes redundant but often critical, information. You see, while the pop up image of the bad guy chewing my ass does give me info that I'm under attack, I often miss it because my eyes are busy elsewhere, e.g., on the map or in my loot bags, or on enemies ahead. But my ears aren't busy. They can listen . There is nothing as creepy as the sound of a murloc approaching from the back. Those are some nasty sounds.

So...audio gets propers in the digital game age.

Symbols too. The amount of info on the lower horizon of my screen is impressive. Six rows of icons, representing everthing from the comfort of the hearthstone to the valuable potions and bandages, to the cruel curse of hellfire. (Yes, I can crack the earth with hellfire, unfortunately I burn as well. It's more of a suicide bomber move than an attack.)

Add to that the x,y coordinate system used in the world (if you don't have the addon, at least try typing /loc to find out where you are. Some WoW databases offer clues with x,y info.).

And in other games, such as Sonic the Hedgehog or Zelda, there are other, similar symbol systems and iconographies mediating the information in the game world.

The complexity is NOT JUST in the game narrative, but also in the interface. I've noticd that in some games, language as we think of it, is pretty rarely used. Yet clearly a lot of knowledge is available for consumption.

I wonder if Sarah would find it more compelling to take notes with iconic headers rather than word headers. Her soc'l sci teacher is deep into making them do Cornell notes. I could see icons easily for the left margin headers.

...thoughts...