My AGC talk on Retention and Systems Math

Here it is, in all its mathy glory: Gritty Systems Design for Retention.

I barely pulled off getting this one done in time. I knew what I wanted the shape of it to be, largely prompted by some of the design choices I saw in Pokemon Go. But I also knew it would involve an awful lot of spreadsheet work and an awful lot of graphs. And I wanted to make those graphs real, not just sketches, so that people could walk through the math and see how it worked.

So — I had notes, but then worked from 10pm to 3am the night before, and then from 9am to 2pm the day it was supposed to be delivered. I don’t recommend cutting it quite this close (the talk was at 4:45, so I finished with not quite three hours to spare).

Photo credit Austin Game Conference

But the graphs are all real! In particular I suspect that folks who have been reading the blog for a long while (Tobold springs to mind) might be interested in the opening section, where I walk through the precise math of “open big” that was referenced in a blog post long long ago. People who are interested in progression system design may want to look at slide 43, which describes the way advancement systems can be based on “number of successful encounters per level,” which leads to a predictable pace of leveling and numbers and curves that can be tuned without massive disruption to the whole game. There’s also descriptions of Monte Carlo modeling, and of the ways in which pure random tables unintuitively lead to very inconsistent pacing of encounters and rewards.

One of my favorite moments from the session was in the Q&A, where veteran designers from a variety of games stood up to add anecdotes and data from their experiences. The most impactful was probably when Dave Rickey described how a small kink in the XP curve in Dark Age of Camelot caused player churn at that level to change from 1-3% to 22%.

The reason that one sticks in my mind is because I did have a fair number of people tell me, over the course of the last few months, that as long as a product is profitable, surely we can’t complain too much about its design? But I think that hopefully the missed financial opportunity these graphs convey should serve to put that to rest; you might be profitable, but it’s a lot harder to increase margins when your basic design is broken. Wouldn’t it be nicer to have a solid foundation to start with? It doesn’t take significantly more effort than the broken values do.

Like many designers, I spend a lot of time in Excel. But I don’t think I have ever shown very much of that sort of work here on the blog before. I also wonder sometimes whether some of this stuff is “a lost art” to some degree, as people engage in a sort of “cargo cult” design approach of simply copying what came before without understanding the principles under the formulas. Maybe this will give a bit more of a glimpse of what a system designer’s life is actually like, for those who don’t know.