An Interview with Raph Koster on Designing a COVID-10 Sim

Posting my questions and Raph Koster's replies (dated September 30, 2020 via email) as part of the documentation of "In the Time of Pandemia."

Khail: Could you describe how you went about crafting the game design sketch that became the basis of "In the Time of Pandemia" (ITOP)?

Raph: In early March of this year, I was spending a lot of time trying to educate people online about how viruses spread, why social distancing mattered, how deadly Covid might be… literally hours and hours arguing on social media. And people just didn’t get it. They had trouble picturing exponential growth (which is known to be a classic human cognitive challenge). They had trouble understanding the concept of repeated exposure, and how it pushed up the odds they would catch it.

Worse, they were really misunderstanding the tradeoffs on costs of lockdowns versus deaths. After all, epidemiology tells us that if everyone in the entire planet just stayed home and saw absolutely no one else for six weeks, the entire epidemic would be over.

I thought to myself, you know, if they played a game about this where they were put in the position of managing things, their process of optimizing for a high score would naturally lead them to certain conclusions.

But I didn’t have time to actually implement this. I have a new startup company, and we were dealing with the day to day impact of managing lockdown, our milestones, and all the rest. So I ended up just doing the design sketch and posting it on Facebook.

Khail: What were the challenges you encountered and how were you able to solve them?

Raph: The biggest was just picking the right variables to represent without making the whole thing over-complex. For example, at first, “diagnosed” was a state, a level of how sick you were. But commenters on the thread pointed out that a patient could die without ever being diagnosed, and a patient might also recover without being diagnosed, so I added a separate axis for it. I was very reluctant to add too many different axes to the game, because epidemiology is actually very subtle and complex.

Khail: Given the complexity of the SARS-CoV-2 pandemic, how were you able to pinpoint the elements to focus on?

Raph: It was all about the key lessons I wanted players to think about. Asymptomatic transmission was basically “the enemy” along with the length of the period in which people are contagious. The cost of lockdowns and the cost of testing. I wanted it to be a direct conflict between money and lives.

From there, the choices fell out pretty naturally. It led very naturally to things like including comorbidities, since one of the things that people were not clear on was the relative levels of risk for different people. Getting that across meant that I had to make the little colored circles in the sketch feel like people, so that there was some emotional impact to it. That’s why they have names, and not just ages and health stats.

Khail: Considering the real suffering associated with the subject, how were you able to ascertain that players would appreciate the resulting game? (An early tester rocked us a bit by saying, "I play games for escapism and not to be reminded of the impending pandemic." ITOP's average rating did end up in the top 3% of 224 simulators accepted by the Newgrounds community so far this year.)

Raph: Players can easily get turned off by a fantasy that isn’t something they want to engage in, and certainly, in stressful times many people prefer escapism. But I knew that plenty of people were obsessively following sites like the Johns Hopkins map, so I figured people like that were likely to be interested. Giving players a sense of control over the situation, where the real world very much left us all feeling like we had no control, seemed like it would also make a big difference.

Khail: A major factor why we were able to develop ITOP despite the limited time, zero budget, and no experience in the genre was that we had high confidence that the design you outlined was going to work. We were able to skip a significant amount of trial and error that comes naturally with a new game design. How could we generalize the game design process you described for resource-challenged contexts such as game dev start-ups, student capstone projects, and university research?

Raph: When I create game designs from scratch, I’m basically trying to create the sort of complex intersection of mathematical curves that something like a pandemic naturally has. I mean, if you think about the pandemic and the math of epidemiology, it’s very much a complex puzzle.

• You have limited resources to take action. This is literally money, and of course, we put money in games all the time as a resource.
• You also are working against the clock; the slower you take action, the more things happen, so you face time pressure, which is a classic game element.
• You have a “landscape” that is the population, which is basically a complex moving graph of connections. Game designers often look at social webs as graphs of connections, but we also look at level design that way. You can abstractly think of the little circles changing color as they change state as being the same thing as turning nodes on a graph to different colors – which how a game of capturing territory, like Othello, works.
• The disease makes “moves” of its own, captures some of that territory, and it has hidden information (you can’t tell which territory it has captured, not until the dot changes color so you know it is sick). That kind of hidden information is just like your hand in poker, secrets that the player is trying to deduce.
• And of course, we give players tools they can spend their resources on. Since time, hidden information, and territory capture are the “tools of the opponent,” we give tools that counter each one: you can try to slow down time with lockdown, to uncover the “map” or hidden information using tests, and to recapture territory by hospitalizing.

This sort of abstract way of thinking is how I approach game systems design, and it’s a very powerful way of arriving at systems that will work. Once you have built this “machine” to speak, it’s easy to add small variations. It is also very important to tune the numbers – you can have a great machine, but with bad numbers the game could end up unplayable. In the case of the sketch, I actually started with the best available real world numbers at the time.

It’s also easy to think about how the player learns from what they are seeing. For example, asking yourself the question “how does the player learn about the captured territory?” In this case that’s the stages of sickness that the infected citizens go through. You want there to be a signal to the player about the severity, how fast it’s happening, where they can play defense. At that point, it’s about the user experience, and even the initial sketch I wrote had a lot of UX design elements in it, focused on teaching the player what the system was doing under the hood.

Khail: For [the last set of questions], we take ITOP to be an instance of game-based learning or educational games. The latter two terms are here treated as interchangeable.

Recent meta-analyses of game-based learning find that it is indeed more effective than traditional methods in terms of cognitive outcomes. However, not quite in agreement with the early optimism of advocates, the effect sizes thus far are less than moderate. As crucial, many studies show that game-based learning is not more motivating than conventional means. What is your take on this issue?

Raph: I think that any studies down those lines really need to have an expert game designer come in and evaluate the games used for the study to see if they are any good! There is a lot of terrible game-based learning content out there.

Khail: Among Newgrounds' top games of the day for August 2020, ITOP ranks 16th out of 28 games. Being able to stand toe-to-toe with top-rated games mostly built for entertainment can be considered as a counterexample for the claim "educational games fail as games". What do you think ITOP did right?

Raph: The overall game presentation is solid, the mechanics are fairly easy to learn, and the actual pacing – which is critical – is at the right speed for players to be able to pick up on what is going on. The right amount of “dressing” is there – meaning, audio and visual elements that make it not feel like a dry simulation. Players need that in order to make the feedback the game gives them feel rewarding.

Khail: Any ideas on how to improve the cognitive and motivational outcomes of educational games?

Raph: Educational games teach best when the lesson they teach is not mandatory, not the point of the game, but simply the best strategy for solving the problem. Players arrive at the strategy by trial and error. They persuade themselves, and teach themselves, because it’s the way to beat the game. But way too much game-based learning material instead makes the lesson the point of the game, or an obstacle to be overcome. The fun is then just small bits of reward for basically doing homework. That is rarely fun, and experts in the field call this “chocolate covered broccoli.”

***

Thank you, Raph, for sharing with us your insights and for providing us with the inspiration and opportunity to serve our communities in the time of pandemia.

-- Khail Santia, ITOP Lead Game Developer