The Future of Virtual World Research

Over the last 5 years I have been lucky enough to be involved in some of the early attempts to create a virtual world for research purposes.  In 2006-2007 I was a part-time producer for Edward Castronova work on Arden: World of William Shakespeare.  A game that was unfortunately unable to live up to the hype due its lofty expectations, the inexperience of our team, and some problems with middleware.  Still, it was an extremely valuable experience and yielded some terrific insights into how to – and how not to – build a virtual world for research.

Later in 2009, with a few lessons learned, I was the lead programmer on Greenland a browser based multiplayer game that would to run in a seasonal format.  The idea being that every season would last 3-4 months and the end of the season would conclude the experiment.  Greenland was more successful than Arden.  We actually hit the target we aimed at (special thanks to Rory Starks and Edward Castronova).  The game launched and ran with about 300 players in a convenience sample, and the gameplay was somewhat similar to civilization.  Players would gather resources and control property in order to succeed in the game.

The major problem with Greenland though was that we didn’t fully consider the implications of our operationalizations, which in the science of virtual worlds translates back to game design.  In one specific instance we allowed players to eliminate one another from the game.  In Greenland players began to off one another and as play went on our sample size drastically dropped.  Had we been intending to study this type of behavior the results would have been interesting.  Perhaps an insightful question would have been: In a limited resource environment what conditions prompt competition and the elimination of other players?  However, we were not asking this type of question.  We were instead asking a question about the emergence of money and trade, which made a large number of players especially important.  The bad news for us was that because of the design decisions we made early on, players killed each other off and the level of trade required did not occur.

Now you might be saying to yourself, “Interesting (or boring).  Now what’s the point?” The point is that like any other research project, success hinges on the conceptual and operational definitions developed early on in the research process.  The next section will discuss some of the implications of conceptual and operational definitions in virtual worlds.  For those with more questions about virtual world research in general see my papers here and here for a primer.

In a recent piece in the journal Communication Theory Dmitri Williams (2010) discusses the problem of mapping.  The idea is fairly strait forward.  In order to use virtual worlds for research purposes researchers have to understand how behavior in a virtual environment maps onto behavior in the real world.  Now, while the idea itself is simple Williams correctly identifies that the problem is not.  Williams agrees with Castonova (2008) and Bainbridge (2007) that virtual worlds can provide test beds for social scientific research.  In doing so he points out a body of research from the Stanford Department of Communication that demonstrates that many behaviors produced in virtual worlds are identical to real world behavior (Nass & Reeves,1996; Yee & Bailenson, 2007).  Williams also not notes that researchers must consider problems of internal validity (are we measuring what we say we are measuring) and external validity (can we make predictions about the real world) when thinking about how and if virtual worlds can be used as test beds for research.

While I agree with Williams conclusions I do want to point out that when considering virtual world research he fails to take into full consideration that virtual worlds are designed spaces.  Returning to what I said earlier not only must we consider the actual behavior of players in the world, but we must also keep a sharp eye on how we structure our conceptual and operational definitions.  Williams purposes that we can use communication theory to found virtual world research.  While I certainly agree that theory should found research, I do not agree that communication theory is equipped to do so.  I believe that the solution to the problem of conceptualizing and operationalizing research in virtual worlds will come from institutional analysis, games theory, and the institutionalized knowledge of the game industry.

In behavioral economics the motto is often: keep the experiment as simple as possible. For years experimental economists have striven to design experiments that have the highest possible internal validity, under the assumption that the experimental window into human behavior cannot be successful if not carefully controlled.  More complexity only adds additional problems with internal validity.  The reason being that each additional component maybe controlled easily enough, but each additional variable added to the experiment presents another possible interaction with structures that already exist.  The important lesson here for virtual worlds research is that each additional feature of a game adds additional complexity.  Now, it may be that a new feature doesn’t have any interaction with other existing components of the game. For example, introducing a simple change like a hadrosaurus skin for the mount system in the game might not change the way that players behave.  But on the other hand if that skin is particularly difficult to obtain and becomes a symbol of status it may very well alter behavior and the results of different levels of unit of analysis.

Virtual worlds are complex systems.  It is their nature to be path dependent. Without careful consideration of design small perturbations can lead to big changes.  If we start two virtual worlds side by side with the same parameters depending on the way the world is designed the actions of one player could have drastic implications for virtual society.  In fact many early commercial virtual worlds ran into this exact problem.  How does one go about creating a world where individuals feel autonomous in regards to manipulating the state of the world, yet protect the world so that the actions of an individual doesn’t alter the state of the world for all other players?  The designers of virtual worlds seem to have solved this problem, as one particular player has a difficult time shifting the world off its stabilized path.  In modern society real world institutions also strive solve this problem, the goal being to create a relatively stable society while providing freedom or at least an illusion of freedom.

The point here is that virtual worlds provide macro-level research environments prime for manipulation. This is something that has never existed.  However, these environments are complex and their complexity is what makes them powerful.  The motto of experimental economics “keep it simple” can’t really apply to virtual worlds.  In order to actually use these worlds for research it is important that we think of them as institutions and complex systems.  Theories of institutional analysis can provide predictions about how alterations in the world should lead to changes in behavior.  Combine this with complex and dynamic systems theory and we might begin to understand how closely the designed features of the world are coupled.  Do simple changes lead to big effects?  Or can we manipulate features of the world without having to worry about the implications that might have for other aspects of the game. I don’t claim to have the solutions to the problems, and it seems to me that systematic research will be required in order to determine how to provide large scale research environments.

What do you think?

The Future of Virtual World Research by Travis Ross, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

No Responses to “The Future of Virtual World Research”


  1. Motivate. Play. - Revisiting Virtual Worlds Reseach - [...] between many online behaviors and their offline counterparts and Travis had pointed out in a previous post this line …

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>