Database “sharding” came from UO?
Lessons Learned: Sharding for startups is a technical post about database scalability. What caught my eye was the term. What an odd term — “sharding.” Why would a database be described that way?
So I started reading a bit about it. It basically means running a bunch of parallel databases and looking into the right one, rather than trying to cram everything into one.
Near as I can tell, a quick Google seems to say that the term came about because of a guy who worked at Friendster and Flickr, and seems to . Wikipedia has only had an article for a little while. In the comment thread at Lessons Learned, there’s mention of the term being used in 2006.
Flickr, of course, was born as an MMO called Game Neverending. In fact, I was quoted in Ludicorp’s business plan, and Stewart Butterfield had asked if I could be an advisor, but I couldn’t do it at the time because of my contract with Sony. Sigh. Anyway, I would be shocked if the term “shard” hadn’t been thrown around those offices… because in MMOs, of course, “shards” has a very specific meaning and history.
It means database partitioning — of worlds. Parallel worlds each running the same static template database source, but evolving different runtime databases. But these were just called “servers” — like, Meridian 59 had bunches of them, and they had numbers instead of the common practice of names that is in use today.
A snippet from the UO intro movie
No, “shards” came about specifically because when we realized we would need to run multiple whole copies of Ultima Online for users to connect to, we needed to come up with a fiction for it. I went off and read a whole mess of stuff about early Ultima lore and tried to come up with a fictional justification. What I ended up with is described here pretty well: that the evil wizard Mondain had attempted to gain control over Sosaria by trapping its essence in a crystal. When the Stranger at the end of Ultima I defeated Mondain and shattered the crystal, the crystal shards each held a refracted copy of Sosaria.
It was a very very specific word chosen because, well, it was a piece of a crystal, which was a completely fictional invention. If Mondain had captured Sosaria on a parchment or in a painting, I would have said “a tatter” or a “fragment” or some such. But in the original U1, it specifically said he had used a crystal to gain power. We even talked about terms like “multiverse” and the like at the time and dismissed them as comic-book geeky and not really Ultima-flavored… so “shard” it was.
Now, from there time kept marching forward as each parallel Sosaria evolved in tandem. (UO was supposed to be between U3 and U4, in terms of chronology). The difference is, some of them got the Avatar (sent by the Time Lord) and some didn’t. Some of them were captured by The Guardian, and we invented the notion that Shadowlords were essentially evil beings created from shards he had captured. In fact, the beta test shard eventually was captured in this way — if you read up on it, you’ll find that really, there should be a fourth Shadowlord running around now.
Original planned UO map, photo by Cory Doctorow, CC BY-SA
(Originally, the landmass of Second Age was supposed to be Ambrosia from Ultima III, and there’s actually a spot up north that is where Exodus is supposed to go. We even made the art for the whirlpool that is supposed to go there, and then just never put it in. But that’s a whole other story…)
(Oh… and then why does the Stranger in the original UO intro movie have an Ankh on his chest? Because U9 was in development aready, and nobody had time to make a new model. 🙂 So it’s the same 3d model as was used in U9, which didn’t ship until years later. So expedience led to a fictional glitch.)
In any case, we called parallel servers “shards” and it became a term used occasionally though not universally as a term of art within the field. You’ll hear folks who worked on MMOs in the 90s use server and shard interchangeably — sometimes saying “shard” to reference a parallel server cluster rather than a physical server.
So, did this database term come from a doc that I dashed off one afternoon in 1996? Umm… I am not sure. Seems like an interesting coincidence, if not.
I wonder if I still have that doc…
14 Responses to “Database “sharding” came from UO?”
Sorry, the comment form is closed at this time.
Cool! I had kind of wondered about that myself…
Quick! Post the doc and reference it on Wikipedia! Then sue others for patent infringement! 😛
As an aside, you had the art for the whirlpool completed?! Is there any way at all, any possibility for any sort of screenshot of the thing? If not, no big deal, but I’m suddenly intensely curious what you folks had it looking like.
It was a whirlpool rendered in 3DSMax, ready to be chopped up into terrain tiles — which I don’t think ever happened.
Ahh. Well, cool to know it was made manifest for at least a little while.
Shred those shards, Jim.
You have successfully stirred up my long-dormant Ultima nerdlove!
It would have been *so freaking cool* if Ultima III had been tied into UO as you describe (Ambrosia!!!). Back when I played, it felt as if UO (while enjoyable on its own) was an online game set in a shadow of the true Ultima lineage rather than an integral part of it.
Wasn’t that “The Lost Lands”? What was the thinking in surrounding it in mountains, as opposed to sea?
UO had a lot of carefully put together lore that tied to the Ultima series. Then each subsequent team pretty much moved it in different directions because there was not adequate documentation of why things were a given way, etc.
For example — originally the Virtues, in UO’s time period, were not established — they are trying to be established firmly by LB, but he faces political opposition from Blackthorn (despite their friendship). Later teams added things like Virtue Quests which suggested that they were in fact established.
http://www.usecode.org/ultima/u3/ambrosia.png
http://uo.stratics.com/content/guides/champions/t2aspawn.gif
Lost Lands is a fragment of Ambrosia, just a small portion of the northern end. Minax in both cases was supposed to be near the top.
[…] Read | Permalink | Email this | Linking Blogs | Comments […]
Back in 1998-99 I helped found a company called BarrysWorld (http://en.wikipedia.org/wiki/Barrysworld), a game service provider based in the UK. Between 1999 and 2001 I was dealing with requests to host unofficial UO ‘shards’ on an almost daily basis, and yes, the word ‘shard’ was used to explicitly describe the function of hosting a private copy of the game world.
When I started reading about database ‘sharding’, the first thing that came to mind was those days at BarrysWorld and the idea of private MMO ‘shards’ – so, I agree that it is quite likely the term ‘shard’ in this context came from UO…
[…] web games end up sharding to cope with the volume of players. By careful game and system design, I think I can build a single […]
[…] Jede Instanz der Applikation ist ein potentielles Spiel (Sharding) […]
Is Ultima Online the origin of the word “shard” in distributed systems?…
As a loan word from Ultima Online, “sharding” makes sense: splitting players across different servers was already a common technique, but UO popularized it, being the first massively popular MMO. And the term “sharding” is unique to UO’s game fict…
[…] called “shards” (a term that apocryphal rumor I desperately hope is true attributes to the MMO Ultima Online ). If you are using cloud infrastructure and you’re not a giant company like Amazon, you […]