Using Psychophysiology in Games & Lunch with Mike Ambinder

While I was at GDC I had the opportunity to attend and talk by, and have lunch with, Valve Software’s Mike Ambinder. What became apparent from our conversation is that Valve and Mike are particularly interested in the application of psychopysiological measures to improve game play. The question they are targeting is: “Can games be made more engaging for users with psychopysiological feedback?”

Psychophysiological responses are things like heart-rate, skin conductance (the amount of sweat in the glands and pores), facial muscle movement (think the TV show Lie to Me), and even electrical activity in the brain. Since the mind is embedded in the body, processes in the brain can be measured by examining correlated processes in the body. For example, increased skin conductance and temporarily lower heart-rate correlate with heightened arousal.

In media studies psychophysiology is generally used to get at the unconscious mental and physical state of an individual. The thinking here is that this kind of measure gets at things that self-report often can’t (although interesting results can come from combining it with self-report). Most individuals can’t control their evolved physiological responses to stimuli, therefore using things like skin conductance to measure arousal is quite accurate. However, two difficulties of these types of measure are that they tend to be invasive (electrodes on the face and body) and noisy. Some, like skin conductance are less invasive or noisy, but in general they can invasive to collect and difficult to interpret without accurate tools and theory.
Background Regarding Psychophysiological Measures

When speaking of psychophysiology there are two concepts that I want to address. They are Valence and Arousal. There is often a bit of confusion between these two concepts how they are different/related. From my personal experience valence is thought of as an emotional construct and it can be positive or negative. It is related to the appetitive and aversive systems. Approach and avoid. Positive valence can be interpreted as some activation in the appetitive system, and negative valence as activation in the aversive system . Generally disgusting or scary things generate a negative valence, while attractive things generate a positive valence.

Valence differs from arousal in that arousal is a measure of the body’s alertness. High Arousal alone is neither positive or negative. It simply means that the body is ready and alert.

Arousal is typically captured through a combination of heart-rate and skin conductance. With games skin conductance is favored because it is less noisy with movement, but having both measures is best. Valence is more difficult with games, simply because it is invasive. Facial EMG requires sensors to be placed on various parts of the face and the signals are small. Facial recognition software is available, but it is currently prohibitively expensive.

So what good is all this to game designers and players?

Real time feedback from the field

The first thing that probably comes to mind when you think about using psychophysiology to make better games is using it in the home to detect player emotion/arousal and having feedback influence actual game play. This is certainly an intriguing possibility, but a lack of easy, accurate, noninvasive hardware currently prevents this approach. When I had lunch with Mike I asked him if this was something Valve would be interested in and his replay was affirmative. When I mentioned the lack of hardware, he said that he envisions more accessible hardware coming out down the line. He even commented that Valve might decide to make their own hardware if there was no one in the market, and they determined that psychophysiological feedback was a good business opportunity.

So, if we could gather direct physiological data from players as they played games what might be some of the uses? I’ll just sketch a few out with the help of my talk with Mike Ambinder and if anyone has any other ideas feel free to comment.

  • Direct feedback to make survival/horror games scarier. (Mike Ambinder)
  • Providing AI with insights into player emotion and attraction.
  • Designing more attractive or repulsive avatars/NPCs.
  • Making games easier/harder with feedback so that the player can remain in a “Flow” like state.
  • Providing researchers with large data sets that provide insights into human responses to media.
  • As a quantitative mechanism so players know the mental state of other players on their team or their opponents. (Mike Ambinder).
  • To provide quantitative feedback for the player, so that they know how they are responding in certain conditions (yoga, meditation, sports that require relaxation and concentration).

Designing a better director
Another way that psychophyisology can be used is to build better models. Even if we can’t gather information about players directly from the environment. We can gather it in the lab and try to generalize that data across different players. Research has indicated that certain individuals actually have different types of physiological responses to stimuli - for example sensation-seekers enjoy positive/arousing stimuli more than non-sensation-seekers. So, even if we can’t generalize to a population as a whole it maybe possible to generalize our predictions about physiological reactions across different groups.

One neat example of this that Mike Ambinder of Valve presented in his GDC talk was using skin conductance measures to make a better Left 4 Dead 2 Director. For those of you who are not familiar in the Left 4 Dead series players try to navigate a level that is infested with the undead. The idea behind the game is to make it to the end of the level while surviving large rushes of 100s of zombies and other undead with frightening superpowers.

What makes Left 4 Dead 2 particularly suited for incorporating psychophysiological feedback is that it is run by and AI program called The Director. The purpose of The Director is predict the mental state and behavior of the player and keep them on their toes. If the player has been idol for a long time then it might send some nasty surprise. If the player seems pretty stressed out it might let them coast for awhile.

With feedback from psychophysiology Valve can make a better Director. The can use the research from their lab to gain a better understanding of the type of behavior that relates to high/low arousal and positive/negative valence. By literally understanding what the player is thinking and how that data is correlated to play they can build a Director that has some insights into the mental state of the player. While this is not as good as direct feedback during play, it still provides a means for building a more predictive Director.

I am not certain whether or not these things will actually come to pass. Making hardware that can measure physiological responses and is non-invasive and accurate seems like a very difficult hill to climb. In the case of facial EMG it seems that facial recognition software may be paired with something like the connect to provide feedback about valence to the system. Heart-rate monitors are now becoming cheaper, but accuracy with noise still seems to be a problem. As for skin conductance, it might be hard to convince players that it is worth hooking even a noninvasive electrode up to them-self before they play their favorite game. Who knows maybe eventually we’ll be able to tap directly into the brain with some form of cybernetics and physiological correlates of brain processes won’t even be needed.

Using Psychophysiology in Games & Lunch with Mike Ambinder by Travis Ross, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

No Responses to “Using Psychophysiology in Games & Lunch with Mike Ambinder”

Trackbacks/Pingbacks

  1. Motivate. Play. - Better, Stronger, Faster: How Games Will Change What We’re Capable Of - [...] Adaptive computing designs will be most useful in cases where the emotional engagement of the user is a primary …
  2. The Origins of Flow | Motivate. Play. - [...] want to leverage games’ power to put us into a flow state to pursue social good, are using psychophysiological …
  3. | Motivate. Play. - [...] idea of implementing physiological sensing and dynamic player-system feedback loops in game design has been previously touched on here …
  4. Valve’s experimental psychologist discusses sweat detection and eye tracking for games | GamesBeat - [...] said Valve has conducted experiments where it has measured your sweat and correlated that to your level of arousal …
  5. Valve’s experimental psychologist discusses sweat detection and eye-tracking for games | Tech. - [...] said Valve has conducted experiments in which it has measured players’ sweat and correlated that to their level of arousal …

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>