Welcome to Raph Koster's personal website: MMOs, gaming, writing, art, music, books.
Welcome to Raph Koster's personal website: MMOs, gaming, writing, art, music, books.

Links
Essays
Talks/Interviews
Snippets
Laws
Timeline
Book

Design Patterns for Persisting Maps

There are basically three choices for architecting map persistence. They range from the static database, which has the most rigid structure, yet also offers the most control to the administrators; to the dynamically generated world, which has virtually no structure, but abdicates all control to the whim of the algorithms that construct the map.

¨   The Static Map Database

The most basic form of map persistence is to draw the map from a static database. Since it is static, by its very nature it does not change and is therefore persistent. Most text mud systems like this are based on the concept of area files, which are text files that act like a flat file database. These files have a rigid structure which forces building the map to be a fill-in-the-blank exercise. This structure usually forces certain compromises: for example, exits to other rooms must be only in the cardinal directions—whatever directions the file format supports. Many muds have added support for directions other than the standard six,[1] but you’re not going to get exits like “towards the castle” as you would under systems that are not template based. Your only choices are going to be those directions that the original designers of the file format supported. Most area file formats have come to encompass a wide range of options, but it is still a particularly constraining limit to have to work within.

Some advanced area files support the addition of blocks of code (generally in an embedded scripting language) to rooms to detect events and respond to them. Most are not that flexible, however, and require that instead scripted objects of some sort be placed in the room (these are sometimes termed daemons after the Unix terminology).

Most template-based map systems require that the entire game server be restarted in order to load new database entries into the game. Even those servers which can do it on the fly do so by essentially running their startup routine without dropping active connections.  Some area files require compilation into binary form before use.

By their very nature, template-based systems demand a high level of centralized control. They preclude expansion of the world by ordinary players, as there is a degree of access to the server and its execution space required in order to add an area. Many muds have added systems for online map creation, usually termed OLC, but all these do is use the mud itself as a medium for creating the static data file and placing it on the server. A template-based system, by its very nature, does not encourage community contribution to the database.

It may seem that template or static database systems are overly restrictive and that there’s no way that a modern mud architecture would make use of them. However, it’s worth considering that the nature of graphical environments is such that they are ideally suited for exactly this sort of system. In fact, most of them rely on exactly this sort of static data file existing on both client and server,[2] making the task of updating the map even more difficult. In particular, 3d environments often require some form of compilation in order to optimize them.

The central control paradigm required by template-based systems makes them ideal for servers where a degree of thematic consistency is required, since they require an administrator to install them, and they are easily checked since they are wrapped up in a tidy little package. This also means that it is fairly easy to put together a template-based mud by simply obtaining individual area files which one likes, and putting them together in a database. This has led to the much-decried phenomenon of stock muds, which are muds built entirely out of other people’s work, with little to no glimmering of originality. However much we might dislike this phenomenon, it must also be admitted that the easy-to-use nature of template files has opened up mud design and administration to a wider range of people than would have otherwise been able to engage in it.

¨   The Aggregated World

Usually, of course, a template-based mud is built up of the aggregation of many area files. A more sophisticated map system embraces the concept of the aggregated world and allows aggregation in smaller units, down to the room. Mud servers with a more object-oriented design such as MOO, MUSH, and LP, treat rooms as merely being an object. Objects can be added on the fly, resulting in sections of the map that appear to be coherent expressions of a single creator but which are in fact not, and are most likely not contiguous in the database either. The flexibility in such a system is immediately apparent; however, so are the weaknesses. As a result, there are basically two ways in which aggregated worlds are handled, and they are quite independent of the code implementation of the map system; the choice between a pre-designed and carefully managed aggregated world, and a user-contributed space defines the nature of the mud in a more fundamental way than the database implementation.

Most goal-oriented muds are centrally managed. In aggregated map systems, this means that creation of new space must be limited to only those individuals with special permissions granted by the mud administrators—generally, only the administrators themselves. In addition, for true coherency, you have to have some sort of central building authority that controls the quality of what is built. Most MUSHes set in popular fictional settings tightly control this sort of thing in order to make the virtual world emulate the fictional one as closely as possible.

Other systems prefer to let players simply build. This is the pattern of a social MOO, such as TinyTIM or LambdaMOO. However, because it is difficult to separate the wheat from the chaff, what typically happens on such a mud is an incoherent sprawl of database entries. This has led to the creation of central authorities even on free building environments.[3] Since many aggregated systems tend to store everything in a giant flat file database, the entire world must exist in memory simultaneously, which can result in performance problems.

While it may seem that the aggregated world has much to recommend it, it suffers a great deal when the community that is building does not share from a single coherent vision. Even the most techno-anarchist of users might cringe at the idea of a virtual space that has no spatiality, no sense of scale whatsoever, no thematic coherence, and thus may be un-navigable except by the most diehard and patient of people. It is extremely common to see free-building MOOs declare a theme from the outset (HotelMOO, for example) and request that all subsequent building be done in the context of the world’s theme.

It’s worth noting that in general, aggregated systems are still treated as static. Once the data has been aggregated, it continues to exist. It’s not particularly portable from world to world, and it only goes away if some form of garbage clean-up is done on the database to remove spaces that are rarely used.

Aggregated systems are common in the text world, but not well suited to the graphical environments that are becoming more popular. The only major try at an aggregated graphical system was AlphaWorld. In this system, a gigantic coordinate space was built, and players were offered building tools to construct any buildings they chose on the surface.

¨   The Dynamic World

The most exciting concepts in mud map design have never actually been implemented in publicly available servers. They all have to do with solving the problem of having an aggregated or dynamic space, but without the problems of fictional coherency, memory space bloat, or overly rigid templates. These sorts of systems are for continuous maps, and rely on one or both of the concepts of fixed random seeds and sparse arrays.

The concept of sparse arrays is simple: instead of having a database entry for every location on your grid, only have ones for those locations which have anything of interest. Then nearby locations can be constructed dynamically by looking at what is nearby on the coordinate grid. Sparse arrays lend themselves to environments such as wilderness or other natural areas where the majority of the environment can be assumed to consist of the same thing; then only those elements which are different can be superimposed upon the assumed default. Another way to think of this is that the sparse array serves as a delta, or Text Box:  list of changes, on a base map that does not actually exist Text Box: Add in a sparse array entry for an irregularly shaped mesa…but which is assumed to be completely uniform.

By itself, a sparse array serves to massively economize on the data storage required for a map. It also makes the tedious task of assembling a map much easier. But more exciting, perhaps, is the fact that the sparse array also can be far more dynamic than the traditional continuous map model. Since each entry in the database is identified with a key and a set of information defining what the entry involves, it is easy to move elements within the sparse array around on the map. This means that it is easy to make such a map dynamic. As an example, Ultima Online stored objects on the map in a sparse array, by simply keying each object to a location. But since one class of objects was actually buildings, it was possible to denote an entire structure on the map with a single database entry. And it could easily be moved or removed. In the case of a well-constructed textual mud, forests could move, trenches be dug, and other such radical alterations occur in the geography.

In a graphical system, sparse arrays leave something to be desired, since the level of detail required to make a compelling graphical environment is not really suited to a sparse array. You could certainly economize a great deal on your map storage, but you’re not likely to get appealing results unless the default against which the sparse array is placed is complex, rather than simple. And it has to be a complexity that is simply described. This is where we can introduce another concept that complements that of the sparse array.

The other concept is that of the fixed random seed. A good definition by J. C. Lawrence from the MUD-Dev glossary:

Fixed random seeding

Using a fixed value (such as a character's unique ID, or the character's position in XYZ space) to seed the random number generator, assuring that the same random number will always be rolled if the circumstances are exactly the same, but requiring no storage. This allows parts of the world or its behaviours to be dynamically generated from the seed value as needed, and yet to have each "new copy" be the same as all the others because the seed value hasn't changed.[4]

To cut through the gobbledygook: the allegedly random numbers generated by a computer are not in fact random. They give predictable output given a seed number. Most computers get around this by rapidly rotating the seed values so that from moment to moment, the random number generator will actually give different results. But at best these are pseudo-random. And if you pass in a fixed seed rather than letting the computer pick a seed, then you can get predictable results for that seed every time. And if you generate your map from a fixed number so that you always get the same output, then you are in effect storing a full map with only one piece of data: the seed.

The basic principle in both sparse arrays and fixed random seeding is that of reproducible results from an algorithm given very little data. An example of the utility of fixed random seeding merged with a sparse array is demonstrated by Privateer Online, which planned to have literally millions of individual rooms, each with a continuous map fractally generated from a seed. Then buildings and alterations to the geometry are stored in a sparse array on top of that, permitting the world to be dynamic within the constraints of what the permissible alterations are. Naturally, a given entry in the sparse array might actually cover a fairly wide area. An element like “a radial circle of standing stones extending across 50 feet” might be represented by a single entry in the sparse array, and the code responsible for placing that element on the map could automatically handle such niceties as smoothing the terrain for the stones to be placed on—without actually having to store each vertex on the map.

The concept of adding detail as the map is examined more closely is of course highly suitable for both sparse arrays and fractal systems based on fixed seeds. Consider the system proposed by Lawrence for gradually populating a sparse array using a seed, based on frequency of usage of the area:

…the game-world and all its details are (generally) a random-seed generated thing with the depth of that simulation (and thus the expense of the modelling) being proportional to player activity there.

 The first player sees an endless flat gray plain without distinguishing features, landmarks, interruption, or any other observable entity in existance other than himself.

  As that player goes about trying to do things, and as other players join him, the world gradually (or rapidly) forms about them, growing out of the entropic raw chaos into something definite, knowable, and of specific type.

Player built constructs would then assume the same requires-regular-human-refreshing-to-persist models, so that, for instance, the road between a distant village and a local town would likely be a narrow path thru the formless void, with only the path and its immediate surrounds being well defined.  The pleasing side-effect being, that should regular traffic between the town and village fall off for some reason, the path would tend to disappear, and the village might well become permanently "lost".[5]

Privateer Online was making use of similar effects, by allowing players themselves to expand the map (which really means letting them toss a new seed onto the pile). Cleaning up the database (which is far smaller a database than otherwise required) is then a matter of whether or not seeds are being instantiated into maps regularly—if not, then do not persist the greater detail. It is not a stretch to imagine a forest entry placed in the sparse array; and when you walk to a point in the forest, a fractal routine being used to add more detail, which can then be stored in the sparse array, but tagged as a higher level of detail so that it can be prioritized for recycling into unused memory.

There is little doubt in my mind that the future of online worlds is going to rely on concepts like fixed random seeding and sparse arrays, and others that are not yet developed. As of now, only a few ventures have attempted to make use even of these concepts,[6] so it is likely they will be state-of-the-art for quite some time.

 



[1] Up, down, north, south, east, and west are the standard suite.

[2] This is done so that the map data, which in a graphical environment can be quite a lot of data, does not need to be streamed down over the wire. This results in lower costs for the company running the game. As commercial “massively multiplayer” games grow, so do the map sizes required by market pressures—the result is that the next generation of games is likely to exceed the storage capabilities of a CD by far, requiring footprints well above a gigabyte in size.

[3] Such as the Architecture Review Board on LambdaMOO.

[4] This definition can be found at http://www.kanga.nu/FAQs/MUD-Dev-L/34.html.

[6] Hero’s Journey from Simutronics and Privateer Online, which was cancelled by Origin, both depended on fixed random seeding; the latter also made extensive use of sparse arrays layered on top of the generated map. In the text world, Discworld is a good example.

Child's Play


A Theory of Fun
for Game Design

Cover of A Theory of Fun

Press

Excerpts

Buy from Amazon


After the Flood

Cover for After the Flood CD

Available on CD
$14.99


More stuff to buy

Gratuitous Penguin 2006 Wall Calendar

Gratuitous Penguin 2006 Wall Calendar
$18.99


Receive CafePress Updates!

LegendMUD

click here to visit the Legend website

"The world the way they thought it was..."