Building the human algorithm
Some academics (including Albert-Laszlo Barabasi, author of the excellent Linked: How Everything Is Connected to Everything Else and What It Means) have been doing analysis of human movements based on where people are making cell phone calls from.
BBC NEWS | Science/Nature | Mobile phones expose human habits
The results showed that most people’s movements follow a precise mathematical relationship – known as a power law.
“That was the first surprise,” he told BBC News.
Is it really a surprise anymore when something happens to have a power-law distribution?
In any case, it does seem like we are inching ever closer to Asimov’s psychohistory. Given enough data, why wouldn’t we be able to build predictive algorithms for large-scale human populations and social trends?
29 Responses to “Building the human algorithm”
Sorry, the comment form is closed at this time.
No, it isn’t a surprise that people find the power-law when they try very hard to find it… It does produce beefy headlines though. Barabasi is very entertaining, but he is a bit over the top IMO.
Given enough data, why wouldn’t we be able to build predictive algorithms for large-scale human populations and social trends?
For the same reason that making random investments in the stockmarket yield the same results as putting your money in funds? In other words, define “predict”. 😉
I call being the Mule.
“Given enough data, why wouldn’t we be able to build predictive algorithms for large-scale human populations and social trends?”
Because then we would all be Karl Rove.
I just knew Starbucks was really Fibonacci.
You’d need to augment your algorithms with some math to help with discontinuous changes, something like the ‘catastrophe theory’ models from topology in the 80’s. Unfortunately that stuff didn’t pan out very well.
If you’re really interested in mathematical models for large scale social trends read some demography. Not as flashy as a physicist doing pop culture sociology of course….
You don’t need much data to predict some trends. EG, “Them that has shall gets, them that’s not shall lose.”
I think processing power is a big issue honestly. 😛
Depending on the interaction of various variables for large scale trends, you may end up with alogirthms that require exponentially greater amount of time to solve for every new variable being included, and if you need a LOT of them to have a good predictive algorithm you may end up with something that takes longer to compute than it does to simply sit back and watch.
Wouldn’t applying predictive algorithms for large-scale human populations and social trends modify the reactions of the subjects?
@relee: it depends on what you mean by ‘applying’. Classic behavioarl modification theory requires some coupling of the output of the algorithm to the input. That kind of modification can be weak, strong, and noisy, but yes as long as the predictive algorithm output can be fed back through the system by modifying the value being measured and predicted, or by modifying a value that is coupled to the first value, it can change the outcomes. On the other hand, there may be other corrective algorithms that push the predicted value back to its mean, and that coupler may be hidden or not reachable even if observable.
This is the essential truth in the phrase, “you can’t cheat an honest man”. Someone with strong values is a tough sale unless you meet those values and in that case, the pigeon can become the power, IOW, the scientist is modified instead of the test subject.
There are many good bits of literature with that theme such as the play and movie “Charlie” and the Don Juan spoof with Brando and Depp.
Methinks you need to re-read the Foundation Trilogy. 🙂 That was indeed one of the considerations…
Although Asimov was not far off (except the bit about the center of the galaxy), games and behavior in games do appear to be the petri dish for that line of thinking these days.
You game designers, what say you? Have you been able to predict and modify behavior through games?
I think the answer to that is “hell yeah,” often to disturbing degrees.
If anything, we learn our trade by unintentionally manipulating large groups by not knowing what levers we are pulling. Then we learn how to pull them better.
Actually… I’ve been thinking about this some more, and I have to wonder if large scale behavioral prediction for aggregate populations is actually useful at all. I believe it’s the second Foundation novel which highlights an inherent problem with psychohistory, namely that it cannot account for freak occurrences or individuals that go beyond the typical boundaries of the aggregate population. But I think that in terms of human existence, one person can make a huge enough impact that any such predictive methods would inherently be flawed, since they can never be taken into account. You might be able to highlight things that would spiral out of control if left alone, but you could never account for individual factors that push those problems to a head faster or manage to defuse them entirely.
Would the Civil Rights movement have ended up the same way as it did if Martin Luther King Jr hadn’t been born? Would we be in the same place we are now if he hadn’t been killed? You could’ve been able to see that the civil rights movement would’ve happened, regardless of any specific individuals, but the time frame and methods of resolution could’ve varied quite a bit if just a few things had been different; things that had nothing to do with the aggregate as a whole.
So given enough data why wouldn’t we be able to build a predictive algorithm for large scale populations and social trends? Because single individuals can still make too much of an impact on even the largest population, and those individuals are too random in their appearance to predict.
@eorlin: The trick is to make the prediction wide but shallow enough. As I recall from the Foundation series, the laws of large numbers are reasonably accurate. Fidelity drops off as you get closer to both real time and specific points on the time line. Chaos/complexity theory takes that a step forward and deals with nested scaling where topology changes at scale (the dot becomes a sphere, the sphere a wrapped cylinder, the cylinder a point set and so on) and then points out the problems of interdimensional scaling when encapsulation breaks down and an error or false number crosses the dimension (inherits or propagates) and warps the manifold. This enables one to account for fidelity/view issues. It is a good idea to study semiotics for the practical applications to human behavior of those theories.
Did you think the Democratic nominee would be a half-black half-white man of Arabic origin with a potential woman VP? The mass numbers would predict otherwise and they should be right. Why aren’t they?
What the predictions can’t see is the effects of small groups of determined disciplined people with enough savvy to manipulate events/signal in the mass amplifier. In this case, Nancy Pelosi set this up from the beginning (See the Atlantic Monthly’s article on the Amazing Money Machine). The web is simply a very large amplifier array. That’s all the Long Tail really means. The second trick to know is semiotics. That is the piece the Foundation trilogy barely touches on.
That’s the mathematical take. The political/semiotic take belongs on another blog because it takes up motivations and that is an incindiary.
Why would anyone expect people to generally go elsewhere than their homes, jobs, or schools? Predictable?
Obama hired 400 bloggers to convince Clinton voters to stay party line. Predictable?
len, I see the biggest problem being that last bit you mention, and that’s what I was kinda getting at. Becaues it can’t see concerned efforts by individuals to alter the equation, it’ll fail the second it comes into contact with those efforts (if they were successful anyway), and once that happened, future predictions become increasingly inaccurate, depending on how important the errors were to those predictions. Those individual actions could very well end up distorting any truly long term (100+ years out) predictions, and depending on how much of a part the individual, rather than the aggregate, ends up actually playing in developing social trends, you may end up with very difficult problems in actually managing accurate trends. You can’t even really self-correct for that, because the outcome is pretty much unknowable and the predictions likely require that the variables are accurate, at least to a degree. If the variables change unexpectedly because of efforts by things that the algorithm can’t see, the errors will disrupt the entire thing after a certain length of time.
Question is of course, how long is that length of time?
But this even happens in the books, and Hari Seldon’s predictions that he recorded for the First Foundation end up being completely useless after a single unforseeable event occurs (of course he realized this was a potential problem and had a contingency plan, but that’s besides the point). I’m just questioning whether or not those unforseeable events would occur vastly more often in the real world for similar time frames; i.e. whether or not simple individuality as part of the equation would be enough to do the same as a mutant that can do things outside the bounds of human capabilities. Are the Martin Luther Kings, Ghandis, and Hitlers of the real world enough to disrupt things as much as the Mule was?
… and Einstiens and Guttenbergs really. But psychohistory for the time scale of Foundation explicitly only worked because of the assumption that there wouldn’t be any significant new technological discoveries beyond what was known when Seldon was working the thing out, something that should be damning enough on it’s own really. It’d take quite a bad hit for humanity to stop advancing technologically.
@eorlin: Individuals absolutely make a difference. I hung around the crowd that invented the cores of the systems we are using to communicate in this medium and individuals were the biggest discriminator of results.
That said, a piece of what seems like contradictory knowledge: the ones who made the most difference in my opinion got the least credit, the least payoff and kept their heads down because that was how they could keep making a difference at the lowest cost. Anonymity or at least relative obscurity is a powerful ally when you discover the key is not prediction but flow. If you know how and where a river goes it is easy to put a bottle in it or to step in it even if you can’t put it down twice. The trick is the signal itself. A word in the right ear at the right time without a thought for the reward is the most powerful force for good in the universe.
Two weeks ago, Ernst Stuhlinger died. Find out who he was. He sent me a book once with the signatures of his friends and colleagues. It is a treasured possession. If you don’t believe in what a man like that can do, go outside one night and look up. Even in the heart of darkness, you can matter if you can survive.
@Amaranthar: Yes there are 400 hundred of them and 18 million to convince. But today gas hit $139 a gallon. Who will make the most difference to the future: Pelosi and Reid or the kids who build cheap solar for charging batteries for cars that weigh 800 lbs fully loaded?
Never give up. Never give in. Dream about a world with abundance through intelligence, heart and sweat equity.
And listen to Bo Diddley. 🙂
@raph:
I was looking at your slides. Excellent work. I’ve blogged/listed it at some other places. That said you said:
“We DO need to lick the “story problem” in online games.”
What precisely is the ‘story problem’?
Mastering large-scale participatory storytelling. Arguably, since this was done, TV and ARGs have made far greater strides than the game or virtual world industries have on this issue, though it has morphed into getting called “transmedia.”
For more on that, try this completely separate presentation:
https://www.raphkoster.com/gaming/gdc_2002_Storytelling.htm
BTW, ALL the presentations are here:
https://www.raphkoster.com/gaming/pres.shtml
Len, Politicians will make the most difference. The discoveries will come no matter what. The politicians will determine the obstacles.
@len, I think I agree with you there honestly, so I don’t need much convincing 🙂
But it makes it hard to have a useful predicitve algorithm when someone can shift the flow like that. It might be useful to use a predicitve algorithm TO shift the flow though I suppose. It’d just be really bad at actually predicting anything accurately. The variables would be shifting under its feet too often.
len wrote:
Who will make the most difference? Why, the folks with nukes, of course!
Check out this article in the June 2008 issue of Wired.
Magazine cover: http://www.wired.com/images/press/pdf/1606cover.pdf
There’s also a counterpoint among those links.
The comments make for a wasteful but funny read, too.
Amaranthar wrote:
Or become them.
“The discoveries will come no matter what.”
Right, because what scientists and engineers do is pretty straightforward, all it takes is enough turns for the little Erlenmeyer flasks to fill the box. 😉
“If we knew what we were doing, we wouldn’t call it research” — Richard Feynman.
Discoveries about something will come, we just don’t quite know what. Thermal Depolymerization may become feasible someday. Or maybe not. Eventually Moore’s Law will break. When? No idea. When will the commodities bubble burst? If you can figure that out, you’ll make billions.
Any of these events would cause what one of my engineering friends used to call a “deweighting bomb” into your model. In other words, you have to reconsider all of your assumptions and predictions in light of that new data. How much do you have to reconsider them? Well, that’s another unknown. As is when.
And also consider — any model that can relate any one of its factors to every other of its factors increases in complexity quadratically. (Yes, you can choke even today’s computers on a relatively small square matrix — 1k by 1k, say, if you decide you have to operate on it often enough.)
Then consider the formation of the planets themselves. Scientists have tried to model this by setting up computer simulations with a multitude of particles orbiting a star. Change the location of one particle by a few million miles, and you’ve gone from an end state of a planetary system like our own, to a system of five planets where the massive ones are closest to the Sun.
Strange attractors are always possible, of course, but we run the risk that these are too strange to be guessed at.
@raph and eorlin:
Raph notes in his slides the means to influence change where one has managed to arrest the will of the players into the scenario: control of the fixed media (eg, radio or TV) signals where the player has little influence, aka, the power of the editing suite over a herd.
1. Reality series aren’t. Real time events are sampled, edited, then fed back into the series to resonate with predetermined mythical roles.
2. A simulation that can’t predict events from a single sample is trivial. Feedback control is based on multiple samples over time. This is the low-energy orbital technique where small influences are used to influence a path to a node where the fallover into another attractor only requires a small thump. Then the path is easy to predict. Still, the key is small predictions for short term effects that aggregate over time.
3. Politicians don’t have the most effect. Editors do. Every major election in the US for example, since 1960 has worked on media control. The glaring power of it began to show in the 1990s. What we are seeing this time is the loss of the backstory as a means to hide it. We are accepting it without resistance because we are accepting the meme that all we are seeing is fiction and games. This evaporates the political center leading to a sports framework. It becomes a game of short passes for yardage to be in position to make bigger plays. It is more expensive but easier to control. In the current election, the use of gerrymandering and mau-mau in the caucuses was very effective. Network techniques were applied and worked. The losers are studying and preparing for the next round. That research/imitation cycle is why the different sides tend to emulate the last winner (professionals fight the last war).
These cycles lead to a collapse in values similar to the behaviors we see when online lists begin to go to hell. “As the twig is bent, so grows the tree.” Values then migrate to new organizations. Game players migrate to new games. The power of migration is the next meme in the series to understand.
@morgan: “Who will make the most difference? Why, the folks with nukes, of course!”
That is already happening here in the Deep South. 18 reactors will come online in the next five years.
The first major commercial accident that almost was happened about ten miles from where I sit as I type this: Browns Ferry. I was a solo singer in a local restaurant. The plant manager was my customer. After that event, he rolled in pale and in need of a drink. He told me how it happened: dumb people. Take a look at the three major incidents and find out which were systems failures and which were humans in the loop. The humans are terrible. Why? Cheap labor. If we allowed airlines to do what nuclear plants were allowed to do vis a vis labor, we’d all take trains. Take a look at the French systems.
That said, the energy crisis is not about a single solution or Manhattan project. It is about learning to use multiple systems in interlocking cascades. That has typically been the case, eg, water power to drive turbines, wind to drive turbines, solar to drive hydrogen, and so on. We can achieve energy independence in the US but we have to change attitudes. To do that, we have to recapture the media and push it to the center. The wild swings of our electoral cycles destroys the coherence of long term strategies.
And learning to clean the condenser drains on our HVAC. 🙂
That is already happening here in the Deep South. 18 reactors will come online in the next five years.
Wait, what? Completely unrelated interest, but could you link me, please?
It was in the local news paper over the weekend. I noticed it because it seemed like an unusually high number. Let me see if I can find that and make sure I had it right.
Some were reactors in already established facilities where they have multiple systems (eg, Bellefonte).
I hope there will always remain people who resist the human insect hive mind.
People trying to predict these movements are doing so in order to control them, and that needs to be understood and resisted.
Prok, did it occur to you that sometimes people want to know how things work simply because they want to know how they work? Curosity is naturally human. It doesn’t have to go hand in hand with a desire for control.
I’ve vastly less worry for the researchers than for the abuse of their research.