Sooooo…What does “Avatar” have to do with what I would like to call “the perfect game” and what is “Limbo”?

Well,…just as the avatar machine technology allows Jake Sully to delve into an alternate reality, where he is stronger and faster, and able to walk, and experience a “new” world, to the point where it changes him, the perfect video game could, and should change how you see the world by allowing you to experience the results (long- and short-term) of your actions, and by enabling you to show, and share your emotions with other inhabitants of the “game world”, which I will call “Limbo”. In this article, I have tried to describe how such an alternate world should look and feel like, and how it can be realized without “avatar-level” technologies!

(There is of course another reason for using this image from the movie on this article: I just recently saw the movie, and it blew me away – there is just so much of what I’d love to see in persistent worlds in the “actual world” of the avatars!)

1. A Description of “Limbo”

If you like playing games as much as I do, there has definitely been a moment when you thought “I can make this game better!” More often than not, this happens after having played a really great game, and being totally immersed in that game for a long time, when suddenly you are jerked out of the experience because of some problem with the game UI, or because the game logic behaves in a really stupid way, or simply because an action gets so repetitive that you start to loose interest.

Humans are clever. We might not all know how to fix a certain gameplay problem, but we definitely know when something is wrong with a game. If we could channel that frustration into creative thinking, instead of just picking up some food and moving to the TV, we will learn a lot, because the games that we choose to play, and the things we dislike about them are indicators of the moods we are in.

When I play a shooter, most often I am in a “Bring it on!” or “Gimme me some action!” mood. But then at some point, and this is where disgust sets in, I always wish my enemies would behave  more intelligently, because it’s really no fun to blast away at a “weaker enemy”. For that same reason, I really enjoy good boss fights, or online multiplayer shooters, and in those kinds of games I never give up until I figure out the best way to beat that baddie. Unfortunately, after the boss-fight is over, or when you are in an online game with “noobish” players, the game pace slows down, you get some loot, a ranking screen or badly-written cut-scene, and return to the real world of mediocrity!

The fact that every single game I have ever played has its moments of painful mediocrity makes me think a lot about the concept of a “perfect game”: a game in which you can be immersed indefinitely, where you learn valuable things about yourself, and where interest never wanes. The “perfect game”, by definition, cannot have a linear structure forced on all players. It must adapt to your behavior, just as you adapt to it. In a way, it should be like an incubator universe; almost like the real world, but you should feel intensely and think intensely while you are in that world, and things must happen at a faster pace than in mundane life, while you are still protected from physical harm. In this article, I will share my fantasies about this “perfect” game with you, and try to describe how this game might be played. I will do so in narrative form, which is more conductive to the imagination than a bullet list of features.

[Limbo - Start]

Feeling totally wasted from the day, I power up my computer. My operating system asks me what I would like to do, and I tell it I would like to play a session of “limbo” – that is the game’s name.

The program starts up. A series of short movies recorded from my previous sessions show up successively while I get into gaming mood and the program loads. A single screen appears with a number of “playing “mood” options. From among “scared”, “depressed”, “angry”, “aggressive”, “thoughtful”, “calm”, “happy”, etc. I choose to play the “depressed” option, because that is how I feel right now. The game asks me to provide some keywords.

Off the top of my head, I say: “childhood”, “sad”, “dog”, “forest” – that should be enough to generate an entry point for the games thought mapping system. The game world appears. As always when I play in a negative mood, the view is in first-person, hiding my funny looking avatar in order not to break the emotional space the game creates. I am standing on a large grassy hilltop. There are tall dark mountains on the horizon. A strong wind is gusting, moving the tall grass around me violently. Overhead dark cumulus clouds are rolling in – blotting out sunlight almost completely. I walk around a bit, enjoying the feeling of the scenery. I take out the binoculars from my inventory and scan the horizon. I make out a heap of stones. I walk towards it.

It’s just a heap of stones. For the fun of it, I start rearranging the stones to spell out my name on the ground. I imagine what the other players in the system who come across this grassland will think when they see my name. They will probably be feeling the way I feel right now, and the message I will be sending them probably is “here walked a bored guy, low on creativity!” Again, I take out my binoculars. There must be something interesting around here, and there it is: what had seemed like a green patch near the mountains, turns out to be a forest. I click on the forest and I am teleported to the edge of it.

The dense canopy of the ancient trees blocks out all sunlight here. The forest seems calm, unaffected by the wind except for the occasional rustle. Strangely, I feel safer within the forest than on the exposed grasslands. I decide to enter the forest, when from the side of my eye I notice a funny looking guy dressed in magician robes casting some sort of fire spell on the entire forest! The guy is Instabanned and disappears in a flash of darkness. Griefers! Maybe I should have played in solo mode. I enter the forest.

Leaving behind the exposed windy grasslands feels good. Though dark, the forest gives me a sense of being protected. There are birds in the trees. I find a cluster of beautiful fly amanitas. I use the pocket knife in my inventory to cut a small one. I am sure I will enjoy defining the amanita’s properties in another session. I leave a few of the amanitas in a small cluster, a gift for the next traveler, and start checking their preliminary preset properties in my inventory – I hear barking.

The barking comes from far away. It does not sound like the bark of a huge dog, more like a small poodle. I try to find the source, but the sound reverberates in the forest. Finally, I find the dog. It is a small black poodle, bound to a tree with an unnecessarily strong chain. The poor poodle seems exhausted and helpless. It is trying to free itself, but it can barely lift the chain. I feel I can help. I am not carrying anything strong enough to cut through the chain though. I can’t use the C4 charge that I have been keeping in my inventory from the battle session I played yesterday, because the explosion might kill the dog – suddenly it hits me. I bind the dog to myself, and exploiting the game mechanics, I teleport to another location in the forest.

The dog is free! It runs around me and barks happily. Only now do I notice that months ago, I had defined a dog called “Sam” looking and behaving just like this one. Back then I was probably playing in “happy mood”. I am feeling better already,…drifting off into happy thoughts, when a girl appears before me.

The avatar of another depressed teenager I believe. Her name is Aurelie. She tells me: “you have a nice dog.” I look at her…the thought in my head: a small girl with a ruffled up poodle: a classic!…and though it took me some time back then to create the dog’s appearance and personality, I decide to give my dog to the girl. She accepts the gift, and thanks me…and off they go, the little girl and her small dog.

I am feeling a lot better now, seeing that even I can help people in “limbo”. The dog is happy, the girl is happy, and strangely, as a direct result it seems, I am happy! I guess the game AI has used up all four concepts I gave it for this session, and I am not really interested in re-exploring those keywords, or even linking them in the network (especially not in depressed mode); I terminate the session after choosing to update my thought-network profile, and doing binding some interesting effects I have purchased with my game points to the fly amanitas I found the forest earlier. Then I get ready for bed.

[Limbo – The End]

That was my idea of the perfect game, although I rather suspect that the concept will evolve as I think more about possible future technologies and new gameplay mechanics. As you noticed, “Limbo” is a loosely guided sandbox-type game. At any point, I could have chosen not to interact with what the game came up with, but it would be hard to do so, because all connections, objects and situations were chosen by the AI specifically to suit my current emotional state and the game’s history of my belief network that it had constructed over so many playing sessions. But can such advanced AI really be achieved? Where do the art assets come from, and more importantly, when will we be able to play such a game?

AI of this order should be achievable within 10-15 years. Human Behavior Modeling, Emotion Space Modeling, Neural Networks and Bayesian Belief Networks are good entry points for this type of AI to evolve. Of course a huge knowledgebase derived from the internet could provide the system with interesting new clauses. The art assets, in a similar fashion, will either be easily modeled by the player in a clay type environment with data gloves, or they will be automatically constructed from spoken descriptions. Alternatively, players can copy and mutate each others’ creations with permission from the owner, or simply purchase the object. All these technologies exist today, but need to be refined.

Data gloves need to be made cheaper and more versatile and computer memory and processing power would need to improve, but we already know that that will happen right? My rough estimate for computer hardware to reach this stage would be around 10 years. Then again, creating this game alone will probably take around 5-10 years, as it requires merging a lot of academic research together, and game companies aren’t particularly good at digesting academic findings. So, today is about the right time to start making the game…!

The way I see it, “Limbo” CAN be made, and it can be more than a game. A game like that can be a mirror for each one of us to know ourselves better. By letting the computer know what choices you would make in a simulated world, it can learn to help you know who you are. That is the real goal, but as you saw, the game can also provide loads of fun – and I played in depressed mood! Imagine how much fun it would be to battle in combative mood!

“Limbo” is my kind of game, but there is no one perfect game for all of us. Everyone might have their own ideas: some futuristic, and some more down to earth. I encourage you to share YOUR vision of “the perfect game” with others. Let me and others know how you would describe your perfect game, and whether you thing playing a game like “Limbo” might be fun,…or even beneficial!

2. The Psychology of “Limbo”

In part II of this article I will explore the similarities between playing the “Limbo” game and the state of dreaming, and discuss why it is important to develop this specific type of game. We begin with a discussion of what “dreaming” is.

Have you ever thought why you go to bed tired and wake up refreshed in the morning (at least up until you get stuck in morning traffic)? Have you wondered why a good night’s sleep can help you look at an apparently impenetrable problem from a new point of view? There is a lot of speculation about the effects of sleep, but the extraordinary importance of sleep is especially evident when you start to consider the effects of sleep deprivation. It is a known fact that about one week of sleep deprivation can lead to insanity, serious brain damage, heart failure, and in some cases death. My own record of 72 of hours work without any sleep resulted in almost complete disorientation, senseless mumbling while trying to talk, hallucinations, talking to myself, and catching a serious cold, even after I caught up with a 20-hour blackout-grade slumber.

One of the stranger effects of sleep deprivation that I have come to notice in myself, and have been able to confirm with others, is the fact that after about 48 hours of deprivation, you start to hear voices that don’t seem to be real. They hint at strange and often dark things. Many times a friend appears to be a foe. You start to read between the lines when people speak to you. Any simple thought turns into an endless line of seemingly irrelevant thought trains, akin to the effects reported by people who suffer from advanced stages of schizophrenia. Of course, I am not the first to have noticed this similarity and you do not need to take my word for it. There is plenty of research in this area.

The point I am trying to make is that sleeping somehow helps maintain the mental network of beliefs in our brains. This network is what we may see as a partitioning of thought-space, connected by lines of causality relationships we have built in the past, either by deduction or by induction.

Let me explain: We humans have word-images for almost everything around us. Word-images are images that arise in our mind when we hear a word. Generally, information is not stored in the form of words in our brains, but rather in the form of hyperlinked meta-tagged pictures or even movie snippets (which might contain other sensory information such as smell within them). These word-images usually carry along with them emotional, mental and moral tags. All of the concepts stored in our brains in this way divide the world around us into meaningful pieces that we can connect and use.

For example, if we had a cuddly little poodle during childhood, the word “dog” will carry a positive emotion for us, and a morally positive value such as “kindness” with it. Then there is also the concept of a “cat” that might be presented close to “dog” in our mind because they share a common group of concepts like pet, fur, small…

…until one day, out of the blue, the neighbors little poodle attacks you and chews off your finger. This is a traumatic experience, and over the course of the next few days or weeks your belief network will probably change, maybe radically!
This is how it might look like after the event:

Now imagine a gigantic network of all these inter-linked word-images, tagged on many dimensions, and evaluated by fuzzy values (not e.g. Booleans), and you get a good black-box model of the brain. It is the “simulated” causal links (I say simulated, because they might not necessarily be true causal relationships between the two phenomena in the real world) that begin to blur with sleep deprivation, either through biochemical and physiological effects, or through some other mechanisms of neuro-electrical or information processing nature that we have not discovered yet.

Whatever the reason may be, sleeping, and dreaming are both involved in this process. Dreaming, research has shown, has a direct relation to maintaining, rebuilding, and purging the huge belief network in our brains. It is a period of self-simulation and self-assessment for the brain, very similar to a POST (Power-On Self-Test, or “booting” as most call it nowadays) on a computer: a unique chance to check out what works, and what doesn’t. In our “angry poodle” example, the red links that were newly created e.g. might be re-evaluated and some might be discarded. Or concepts might be shifted around based on extraneous concepts that have links to the existing ones. Blood” “Barking” and “Pain”, our three new concepts, might be shifted to the left and away from “Cat”, as your brain tries to maintain it’s positive image of the cat. Positive here means: being closer to other “pleasant” things in the multidimensional space these concepts and connections create.

Luckily for us, in the case of our brain, if there is some totally illogical relation present in the network, it doesn’t cause a crash. Rather, that belief is either mended with other beliefs, resulting in pleasant dreams, or radically altered, resulting in “painful” dreams, where the brain is training itself through simulated pain to create huge changes in the fuzzy values of the relationship tags of the word-images and their links.

So far, I have asserted that individual human understanding is the building and refining of this network of image-words, and that we need a system which enables us to refine and test the network for discrepancies and fix them. This system exist, and it is known as “sleeping and dreaming” to most of us. Now, I believe that the perfect game can and should have the same healing effects towards our belief network that a good dream has, i.e. it must check your belief network for discrepancies by providing visual/sensory clues to it, and estimating your links and tags for each set of those images. Then, if a discrepancy is found, it should be changed by either inducing simulated “mental” pain, or reinforced through positive simulated mental “pleasure” as it happens in pleasant dreams. All of this should happen in a sensory-rich environment that keeps the brain interested for long periods of time, just as it happens in dreams (though, often when the brain looses interest, or starts to explore a totally different area of the belief network, dreams are broken up, and a new dream begins during the next dreaming cycle).

Why should a game resemble dreaming? Because we humans, over many centuries have been expanding our belief networks during our waking hours, at the expense of loosing sleep. Some of us cherish one night of good sleep. We are overloading the belief network with links that are not in accordance with the physical and logical truth of the world around us.

The world as a whole is starting to look like it can’t ever be understood. We feel lost in its complexity, and this feeling is debilitating many of us. The perfect game, as I have defined it is a chance to blow life into art and entertainment and perfect them in their role as ways to heal and purge our belief networks while providing an interesting latch for our awareness. The brain does not learn (adjust values in the belief network) when it is not aware of and interested in a specific situation. The beauty of art and the fun factor in games (along with all the hormones that back this fun) allow the brain to become aware and interested. Unfortunately, the next step, which consists of providing a way for our belief networks to be mended, expanded and purged, has been missing so far. To sum up: gaining knowledge of your “self” can and should be fun, and a game can provide a perfect medium to achieve this. I deeply believe in that last claim.

3. The Graphics of “Limbo”

In parts 1 and 2 of this article, I discussed the necessity of games that act as interfaces to our subconscious mind, and I presented the type of gameplay and interactions possible in a prototype of this game that I called “Limbo”. In this part, I will discuss how such a game can be actually made, and what disciplines and research areas need to developed to enable the creation of such games. First we will cover the area of computer graphics production for the game.

Having read the description of “Limbo” might make you think that such a complex game is impossible to build. You might argue that there is simply too much AI and guesswork involved; too much technology, and too varied an environment. Populating a graphical, physical and behaviorally correct database of all objects and people in the world around us seems impossible, etc. etc. Rome was not built in one day! As long as we are focused on a specific goal, and move in that direction, we will achieve that goal at some point.

If we analyze the problem at hand, we can break it into many parts, each of which isn’t out of reach at all, even if not in its perfect incarnation. As with most contemporary computer games, 3D environment and character animation production are two of the most costly assets in “Limbo”.

Problem 1. High Cost of Expense of Environment & Character Graphics Content: Due to the rush for newer technology, nowadays, almost every single game built provides its own custom-made graphical environment. Once we are past noticeable improvements in graphical fidelity, however, it will be possible for many companies to come together, and create huge databases of graphical objects. Middle-ware companies are starting to take a step in the right direction already by providing products such as SpeedTreeRT. Graphical object databases of the type described will be tagged with metadata, and a clever graphics-picking engine will be able to query the database over the internet to quickly put together believable scenes during the load time, or even during gameplay.

Problem 2. Right now productivity in the computer animation department is extremely low, by modern industrial standards: nothing is really automated, and everything is custom-built.

Let’s look at a common animation pipeline:

Step 1. A description of the animation is written,

Step 2. A 3D capture studio is hired, and a large number of people cooperate to create a few specific captured animations.

Step3. The data from the capture system is filtered, down-sampled, re-formatted and generally changed to meet the needs of the customer.

Step 4. The game company art department superimposes every single piece of animation with that of its character models, tweaks the animation to make it “fit” perfectly, updates timings, fixes skinning problems, fixes the start and end-point of the animation, etc.

Step 5. The animation is exported to an engine-specific format. There is a lot of stuff that can go

wrong here, because there is no industry-standard exporter. Unbelievably, there isn’t even a Microsoft-approved DirectX exporter!

Here is the more productive scheme that I recommend for the future, and specifically for making games like “Limbo”:

Step 1. 3^rd party Capture studios agree on one format, with an open well-defined and standardized set of biped sizes. They produce all manner of captures, with fixed beginning and endings and ready for import. Then many of them together use the services of an online publisher to sell their products through a huge online database that has integral tagging, searching and billing systems.

Step 2. The directors and animators in game companies have an integrated tool in their 3D creation environment, that allows them to access the capture database, preview the motion on their characters and purchase the motion. Because biped sizes are standardized, changes will be minimal and a lot of the skinning can be done automatically.

Step 3. An industry standard format for exporting along with free (and bug-free) exporting tools should be developed. Thus, exporting will be easy and pains-free!

Note that only the last two steps are performed in within game companies, and there is no need for unnecessary communications and individual contracts across companies. The good news is, that as motion-capture libraries grow, and online purchasing over the internet become the norm, this system is already materializing. If industry standards develop in time, animation will be one of the easiest parts of game development. The next step, would include a mechanism built into the game engine, that automatically finds animation sequences that correspond to the player’s choice of movement, downloads them from the online motion-capture system and displays them.

As connection speeds grow and bandwidth becomes cheaper, my estimate is that all of this can happen within the next ten years. Another solution would be to generate some of the motions procedurally by linking the game physics system with the code that is now integrated in  procedural routines being used to create realistic human and animal kinematics by some specialized software. K-3D and Houdini are two software suits for creating procedural animations that are available right now.

Once the graphics have been created, we will need hardware that can display it at the extremely high resolutions needed for spherical holographic projection (or direct retina projection for that matter). Real-time Graphics Display Technology: Spherical holographic 3D projection around the player is possible even today, though at very high cost. As the technology matures, however, it will be a viable display option for games. A direct projection on the players’ retinas is possible and the rudimentary technology already exists. This is as close as we can get the picture to the player without invasive surgery on eyes and brain, and should prove adequate for decades to come. One problem that is associated with both of these display methods is the necessity for having extremely fast vector processing units (a.k.a. DSP style processors).

Right now, most systems integrate these routines in their normal processors or graphic chips. DSP style processing is cropping up as an integrated solution though and the Sony PS3 makes use of the “cell-architecture” which is really a glorified name for a bunch of DSP chips working in unison.

Luckily, at the speed that vector graphic processing is growing in the telecommunication industry, we will just need to wait a few more years for the high-speed hardware to become available on global markets. Texas Instruments is one of the forerunners in this aspect, but AMD and Intel are looking at integrated DSP solutions too. To summarize, though graphics and animations look like two of the tougher development challenges in “Limbo”, they are actually the easiest one to overcome. Current technology and processes are moving in the right direction, and if standardization of formats and processes happens in time, it will not be hard to create the graphical infrastructure necessary for a game like “Limbo”.

(…in the next installment of this article I will discuss the AI and physics engines of “Limbo”.)