The Internet as existential threat
Some days I wonder if we are completely screwed. So today’s post is a perhaps slightly hysterical outburst.
The news is not paying enough attention to the Petya/NotPetya ransomware, and the effects it is having on the Ukraine and on a bunch of businesses worldwide. I think it may be a harbinger of how the Internet could kill us all.
Based on what little I have read so far… A piece of widely used tax software — one used by the Ukrainian government — did its usual “phone home” to check for updates. Instead of getting back a few hundred bytes of acknowledgement, it got a viral payload. Basically, this tax software served as a means of auto-updating the virus to thousands of targets. The result is not just accounting systems down, though. It’s gas stations and point of sale systems in grocery stores.
This kind of thing basically makes me wonder how long we’ll have the Internet.
The whole premise of the Internet is the connecting of disparate networks. It started out by only connecting computer networks. But today it connects networks of vastly different sorts: computers, yes, but also financial networks, distribution networks, road networks, water networks, power networks, communication networks, social networks. It truly is “Inter” now.
As we rush towards putting more and more things “in the cloud,” as we rush towards an Internet of Things with no governance beyond profit motive and anarchy, what we’re effectively doing is creating a massive single point of failure for every system we put in it.
Think of a house with an alarm system on the doors, and a phone system, and power coming into the house, and water pipes, and so on. In your house these are probably all separate connections to separate networks. If the water stops running, you don’t tend to think that your phone will go down too. But you know that cutting the power at the mains renders the house vulnerable in a host of ways, because so many things do connect to the electricity.
Well, even without going so far as to buy Internet-enabled juicers, quite a lot of that stuff actually has been connected to one point of failure, and it’s not necessarily all things we term “critical infrastructure.”
What we are building is basically a perfect scenario for collapse, where a commons is consumed by actors who either don’t care or don’t understand the collective damage that is possible in a connected system, and the tipping points that can ensue.
Most networks we come across in the real world follow power-law distributions, and are what we term “scale-free networks.” Basically, this is where most nodes on the network aren’t that important, but there’s a preferential attachment thing going on, where some nodes are super-connectors. They’re really hard to destroy; you have to take out the biggest nodes, all at once. But if a power-law network is co-opted, you have a real problem. The Internet is basically our biggest node now.
Most of the big virus scares lately have been traced in one way or another back to state actors; Petya is based on an exploit the NSA kept secret, that was then leaked to the general public, and weaponized by hackers. As huge as their effects have been, consider that this implies fairly limited use. But picture a world where these tools of state actors are actually in the hands of random people, and released at the frequencies that random people would engage in. I remember being in South Korea in the mid-2000s, and watching a colleague’s laptop get owned instantly just from connecting to hotel wifi without firewalls up. Within ten seconds, the laptop was completely useless, locked up, conquered totally. Picture an Internet like that. In such a world, the only people who can connect would be the ones with the wherewithal to do so, the money and the savvy and the ability to actually harden security.
But just as critically, governments and state actors seem to be the source of so many of the problems precisely because the Internet is now too many forms of critical infrastructure, and therefore too juicy a target. If software eats everything, then the ability to kill software is the ability to kill anything. Net connectivity becomes the single point of failure for every system connected to it.
Even if the Net itself is designed to route around damage, that doesn’t help if it is the single vector of attack that can take down any given target. It’s too juicy a target for the military, too juicy a target for terror, too juicy a target for criminal ransom.
The old adage goes “when they came for this, I said nothing. When they came for that…” — we all know it. Consider that the more we hand gleefully over to the cloud because we want convenience, big data, personalization, and on, we’re creating a single thing that can be taken from us in an instant. We’ve decided to subscribe to everything, instead of owning it. When they came for your MP3s, your DVDs, fine,. not “critical infrastructure.” When they came for your resumes, OK, getting closer.
Your juicers? Whatever, we can laugh at that because it seems ludicrous, but it’s not. A typical US city only has three days of food within the city limits, because the Internet has enabled just-in-time delivery of foodstuffs. Economic optimization within a network tends to imply specialization, which means that even those lovely rural communities that in theory grow their own food don’t grow balanced diets locally. And you’re laughing at an Internet connected juicer? Your juicer is already Internet-connected. If that goes down, you don’t get any more juice! It’s just connected in a way you can’t see.
Now that gas stations play video ads on a loop above the station, now that every cash register is replaced with an Internet-connected device, losing Internet means no gas and no groceries. No gas means no trucks delivering the groceries. Especially if we make them into self-driving trucks! We think of critical infrastructure in terms of government-owned or controlled utilities… but the food trucking fleet is “critical infrastructure.” It’s owned by a massive patchwork of private entities, and actually is networked into the air fleet and the shipping fleet as well via databases of shipping container IDs. Wanna paralyze the world economy? Corrupt that ID database.
If you have a “smart wifi lightbulb” that’s critical infrastructure because it can be owned by a botnet and used to attack. Hyperbolic? In a world where we take actual damage when something digital is attacked, any CPU is basically a weapon, and leaving Internet connected CPUs unattended is basically leaving armory doors open.
Take the example of the solar panels on my home. They are similar to the IoT lightbulb, but the point is more pertinent.
The solar system controller phones home in a variety of ways to provide information to me on how it is performing, but also to inform the grid about the power I am generating . Because there is no battery in my home, any excess power beyond my consumption must be fed back to the grid. Should solar panels feed more power into the grid than the grid can actually handle, this power must be offloaded elsewhere — typically California pays neighboring states to take it. If the power utilities failed to do so, the grid would actually explode. Literally. Explode. The result could be a cascading power failure covering several states.
By connecting this solar controller to the Internet, we have actually put a portion of the critical infrastructure of the entire power grid in the cloud where it is vulnerable. Is that the most direct vector of attack? No, of course not. I suspect you can’t actually tell my solar controller to do anything much, it’s pretty stupid as smart devices go. But I have every expectation that someone wants to make direct bidirectional control possible, because it’s “cool.” (Presumably, regulation is stopping them. Yay, regulation. Please don’t let Congress notice your existence).
The only difference between my solar panels and a hydroelectric dam is scale. To the grid, they are all just nodes, with differing power outputs. Yes, you could cut off my panel. You could cut off a hydroelectric plant too. The issue isn’t whether the node in the network is severable. The issue is whether we are increasing the fragility of the system and thereby increasing the likelihood of cascade effects.
Network connecting solar panels opens the possibility of things like malware attacks designed to cause them all to misreport, say… luckily, the electrical grid has redundancies, fuses, switches. Physical lines to sever. We can measure power flows independent of using the Internet. So let’s consider another example.
Our medical systems have terrible Internet security… MRI machines you can connect to with USB that still have “admin:password” to gain root access. That’s horrifying, sure, but that’s not an attack at scale. More frightening: we’re busily uploading all our medical records to the cloud. Take down that cloud, and no patients can be treated, because nobody will know what they have, what meds they are on. Software swallows your insulin pumps and your pacemakers. To kill people, all you need is to hack that database, or simply erase it or block access to it. After all, we don’t tend to realize that in an Internet of Things, humans are just Things too.
As this software monster has encroached on stuff like election systems, the common reaction has been to go back to paper. So let’s consider a less obvious example. We should be going back to paper for our libraries too! We’ve outsourced so much of our knowledge to digital that the amount of knowledge available in analog has dropped notably. There are less librarians in the fewer libraries with smaller collections than there used to be. If the net goes down, how much reference material is simply not accessible that was thirty years ago? Google Search is “critical cultural infrastructure.” How much redundancy do we actually have? Could a disconnected town actually educate its children?
How critical is Google as a whole? If Google went down for a month, I am pretty sure we would see worldwide economic collapse. How much of the world economy passes through Google hosting? How much of it is in GMail? How much is dependent on Google Search, Google Images, Google Docs? The answer is a LOT. And because financial systems are now also JIT, ten thousand corporate blips where real estate agencies and local car washes and a huge pile of software companies and a gaggle of universities and so on are suddenly 100% unable to function digitally (no payroll! no insurance verification!) would absolutely have ripple effects into their suppliers and their customers, and thence to the worldwide economic market. Because interconnection without redundancy increases odds of cascades.
It’s actually NORMAL for complex systems to go through collapse cascades. It is part of how they grow and develop. We just won’t like it when one happens to us.
In the current economic climate, there’s this romance with the idea of monopoly. VCs like Peter Thiel speak approvingly of not funding anything unless it has a shot at monopoly. Some great achievements of technology probably wouldn’t have happened without the monopolies that are currently enjoyed by most of the big names in tech. The usual arguments against monopolies are generally around how they stifle competition and hurt consumers. Consumers are OK with the tech monopolies because they largely see benefits right now.
But the single biggest downside to these monopolies is actually lack of redundancy. If AWS went down for longer than the brief interval it did a while back, is there even enough capacity elsewhere? I have no idea — probably there is — but what happens when instead of it being a minor inconvenience it’s actually gone? That’s more like losing the hydroelectric dam than losing the solar panel.
We should be thinking now about how we create redundancy, resilience, in all these systems. “The cloud” isn’t it. Big Data isn’t what we need. Small replicated data is.
This is not solely a technological problem. I’ve often wanted to sit down with Mark Zuckerberg and argue with him about Facebook. It is premised on the notion that “connecting everyone” is an unmitigated good. But it’s not, and for the exact same reasons as the above. We don’t have opinions, we share the opinions of those we know. We think and decide things like politics via viral mechanisms — the old school meaning of “meme.” Nodes can be infected, can even be high-profile nodes, and they will have cascading effects on far larger populations. Actors who don’t understand what they’re doing — like say, billionaire political activists — can basically release ideological malware into the population not realizing the cascade effects, because predicting chaotic systems is hard, and by connecting everyone we’re actually intentionally removing the firewalls and the fuses and the airlocks. Attacks on the idea of the value of expertise are like taking down the immune system while giving the patient a cold.
Right now, we’ve got shit in the water supply.
It’s possible the water gets so dirty that no one can drink from it anymore. This would be all of us saying the net is too dangerous to connect to.
It’s possible we all keep guzzling away and all die.
Or maybe we can start getting smart and diversifying our water supply, getting smarter about cross-contamination, drill separate wells and avoid tapping the same water table.
This sort of problem is what birthed modern epidemiology, long ago, when Dr. Snow figured out a cholera epidemic’s source in London. Facebook is like all of us drinking from the same well.
In general, I’ve come to believe that the norm for systems is to interconnect, to form larger networks, and for sub-areas in that network to evolve into specialization. In the process, they lose autonomy. Eventually, they end up as appendages — sometimes vital, sometimes optional — to the larger organism. The larger network is almost certainly more powerful, more likely to survive, capable of greater things. But when it goes, everything in it goes too. Bits and bobs survive, or dissolve back into constituent parts. Anything over-specialized at that point is almost certainly going to perish, to be used as building blocks for a different network.
We’re fine with this when we are the larger network. Paring our fingernails is no big deal, and the fingernails don’t get a vote. When we are in the larger network, though… it’s likely to our individual benefit not to permit it to reach too high a level of interconnection, specialization and sophistication. It simply means we’re each more vulnerable to the failure of some strongly interconnected node way up the line — just like the tendon in our toe is screwed if our nervous system gets shut down.
Anyway. Pay attention to Petya. Think about how much of your life is online. Assume every connected service will some day shutter. Consider your personal strategies, and contemplate the larger scale. I’m not a radical individualist, not by a long shot… not the sort to say we should hoard gold and have self-sufficient farms in our back yards. But I am someone who more than once has built entire complex communities with hundreds of thousands of nodes — technological and human nodes — and watched them fall prey to single points of failure.
This isn’t about cute Internet of Shit jokes anymore. It’s about how gangrene spreads.
24 Responses to “The Internet as existential threat”
Sorry, the comment form is closed at this time.
You don’t even sound hysterical. It all sounds very plausible. But then I’ve been paying attention to this stuff for ages, probably as long as you. Maybe we’re both hysterical?
Reminds me of all that Black Swan stuff from Nasim Taleb. The internet was designed to survive a nuclear war and yet is making everything more fragile.
Reminiscent of a short story I wrote a while ago:
http://www.sixminutestory.com/stories/the-day-the-net-stood-still
We’re going to have some interesting conversations when you come visit next Raph.
Hi Raph –
We should totally dive deep into this. As you know, I’ve worked at the intersection of games and cybersecurity for years. It’s FASCINATING stuff – I love talking about it.
Many things you say are true – at a macro level, attacking technology will continue to grow. No question. We’ve only just seen the beginning of WannaCrys and Petyas. The world went nuclear when Stuxnet happened – state actors openly sanctioning this kind of thing.
But the reality is that while tech is a power law population – there are fantastically large populations of similar tech, then homogeneous blobs of decreasing size – the system as a whole is a LOT less consistent than you think.
I think the next time we get together, we should set up a Defcon-style Capture-the-Flag, and you’ll see how getting through one armory door doesn’t get you into the next one. It’s more like a shopping mall – yeah, it’s pretty easy to smash any of the glass doors. But sometimes you get into Macys, and sometimes you get into Hot Topic. And just because you’re amongst the novelty tees doesn’t mean you’re also in Macys too.
I’ve also had the pleasure of working with some brilliant folks who are working to counter all this. Whether it be locking the front door in the first place, or getting things back up and running after an event does happen, there is significant attention being put into the problem. We’re probably behind the bad guys, but that’s history for you. Someone builds something. They assume good intentions. Someone comes along with bad intentions. They break it. We fix it. It becomes useful again.
There will be major events, and they’ll get bigger. And they’ll be more and more costly. I just don’t see a runaway cascade like you’re talking about happening ahead of our ability to fix it. I think we qualitatively agree, but perhaps not as much quantitatively. It’s just not existential, as in humanity, as a species, is doomed.
This is a great conversation for the next time we have beers. 🙂
Aaron
PS – ironic footnote – I couldn’t log in with Twitter, Facebook, or G+ to post this. I see Twitter gave access, but your blog post page is not logging me in. 🙂
Well, I do want to say, ROM chips still exist, and at some point device makers are going to take an attitude for a lot of smaller things that’s basically “Our device runs code out of this ROM, and that’s it, period. We have no facility for automatic updates or running code sent to the device over the web for any reason, ever. If this tiny, cheap device needs updating in the future they can buy a new one and update that way”.
This “EVERYTHING is dynamically reconfigurable through downloaded code” fetish will eventually become more sane, with some things able to be updated and some not, and more controls over when and how many of the configurable things allow outside code to be run. (I think Microsoft played a large role in making our current infrastructure as vulnerable as it is, but they’re not the only ones.)
Of course humans being how they are, this will happen AFTER some large catastrophe(s) have happened, in response to them, rather than before. At least I can say *I* have always designed and built software architecture on the basis of continually asking myself “Knowing my system inside-out, how could *I* hack it and what could I do? Ok, let me change that so it can’t be hacked that way. NOW how can I hack the system with my modified plan? Ok, let me fix it some more and reconsider again…”
It will take time for that attitude to become more widespread among coders. But I have to say, it’s already more common & with more people competent at it today than was the case 20 years ago. I think we’re getting there. My current Furcadia coding team are all more security-focused than I am, which is probably a good thing. As long as they keep getting development done & shipped too. Which they do. 🙂
Also I would keep a non-electric can-opener on hand right now. Just in case. 😀
[…] wonder why I didn’t address things like privacy concerns or the possibility of cyber attacks turning the devices against us. I am not an analyst, despite the fact that I have a blog and write a whole lot of words without […]
This is why we don’t see alien civilizations among the stars. Everyone gets to this point and implodes. 🙂
[…] An essay on the dangers of the Internet of Things and single points of failure […]
Bruce Schneier writes about this kind of concern periodically, such as this one which originally was published in January: https://www.schneier.com/blog/archives/2017/02/security_and_th.html
You are right to worry, and there are plenty of people who are worrying along with you. But why this kind of worry isn’t getting a larger voice that forces companies to listen, ensuring that their devices won’t make the Internet a worse place, I don’t know.
These kinds of concerns are why I won’t immediately sign up for a new service, even if it seems amazingly useful. When my favorite todo list app had an automatic update that locked me out of my list unless I signed up for their AI-powered service that was suddenly required, I gave up the app.
And more and more I am looking for self-hosted or installable software for the things I depend on regularly. I would love to switch to webmail for my business email, but I don’t want to use the popular option of forwarding through GMail because Google already controls my calendar and various other pieces of my life. I don’t want a single point of failure.
But when it comes to society as a whole? There’s only so much I can do as an individual to protect myself from the fallout that could result from a hacked/attacked power grid or healthcare system or government database. And I’m not sure exactly what I can do that would be effectual in that regard.
Ralph – good article. I’d suggest you remove the ‘first they came’ reference, as it’s not an old adage. It refers to the Germans exterminating jews. There is ZERO equivalency between “first they came for my pics on my iphone,” with “then they came for the jews.” Zero. I can only hope that you’re not aware of what you wrote, vs being horribly insensitive.
That reference pollutes what’s otherwise a brilliant and thoughtful piece.
I’m quite aware of what it’s from and what it references (and, it IS old at this point — it dates to around 70 years ago!).
The end point I am referencing in my essay is not “pics on the iPhone” — that is the *starting* point. The whole point of Niemöller’s original is the escalation, the gradual encroaching (depending on version, the original starts with “the Socialists” or “the Communists”). That what seem like acceptable steps open the door to less acceptable ones.
The end point I am talking about here is potentially in fact genocide-level calamity, potentially in fact the deaths of millions. I used the phrase “existential threat” in the title. That phrase usually means “the death of all humanity.”
It’s a fair point that the level of malice aforethought is different. This is more of a blind stumbling into evil than one directed by people with an evil agenda. But the magnitude of the evil is comparable.
That said, certainly no offense was intended. It was a reference made quite deliberately to underscore the kind of risks I see.
I’ve been doing a lot of thinking about stuff like this. I have a lot of friends that just don’t think about the world they live in, and don’t realize how fragile everything is.
I’ve been trying to come up with something that can fix this. The biggest problem I run into is that whatever solution I come up with, it’s going to cost money for people to implement and it’s going to see resistance from those who benefit from centralized systems. But, once I have cleared some of my current projects off my plate (and hopefully built up some resources of my own from them), I want to really dig in and get to work making it real – I may not be able to fix everything but hopefully enough people will like it and use it that our worst case scenario won’t be so bad.
[…] The Internet as existential threat (raphkoster.com, 3) In the wake of this week’s major cyber attack targeting Ukraine but causing crashed systems across the globe, it’s definitely time to consider the dependency of an increasing number of critical infrastructures on a properly working Internet an existential threat. And I really dig the term “ideological malware” mentioned in the this important text. In a connected age, the term “critical infrastructure” actually could be extended to the human mind. […]
Brittle systems break and then they get tolerated or fixed.
Is global GDP tied to the complexity of our global networks?
To me this seems to be the case, the system grows with more nodes around the edges. To stay specialized within some field of expertise you need to update your skills at an ever growing pace. To make a profit from innovation you mostly increase the complexity and rarely remove old and obsolete links.
Our rescue will be to invest more of this effort into stabilizing the system. Basically developing insurance. At least this is what my idealistic wishes tell me. 🙂
Wow, nicely done, Raph; excellent and thought-provoking.
Also: Now I’m scared shitless of my coffee pot and am considering a covert op to cleanse my kitchen.
This is the most sensible article I’ve read in a very long time. Thank you for crystallizing so many of my concerns in such a concise fashion.
I kept waiting for a new game announcement at the end that you were working on with the aforementioned content =P
Great article, and I particularly like the analogies used to illustrate the threats and potential downsides for the nontechnical audience.
About 25 years ago, Hurricane Andrew devastated much of south Florida, and a very important thing I learned from an article about it was how quickly – 3 days – civilization descended into chaos. You had to stay awake all night guarding your property, fuel, food, and water with firearms. It was every man for himself, in three days. Most people have no clue that we are closer to that horror than they realize, and even worse, getting closer all the time, for all the reasons you mention. And we call this “progress”.
Interesting timing on your post. Canada’s entire banking system, slightly larger than 10% of the American system, got paralyzed for nearly 48 hours starting Friday afternoon thanks to its “Interac” internet based payment system.
Nearly all Canadian banks, trust companies and credit unions use Interac for e-Transfer of funds. It has almost completely replaced checks in Canada. It’s faster, 20 minutes, saves a trip to the bank, cheaper for both bank and customer, so cheap that banks have eliminated fees for such transfers. Everybody uses it, both business and personal payments. Including a lot of payroll. Friday was the lay before the Canadian Canada Day long weekend and the day before the first of the month, when rent is due and most mortgage payments get paid.
A lot of people did not get their Friday pay checks. Canadians live as much hand to mouth as Americans do. Personal rent and mortgages didn’t get paid, In turn landlords couldn’t pay their mortgages. Long weekend trips got canceled because people literally could not buy gas or food.
Catastrophe ensued.
IOT to me is a classic case of the gold rush to make money in a technology or process, while willfully ignoring safety concerns or environmental impact. Other recent examples are Uber, Vaping and Frakking. The lure of cash and the lure of being a first mover and dealing with regulations and paying for it later is too strong.
Great article Raph. I think the more people understand about interconnected complex systems the more they should be rightly concerned. Having worked for two critical utilities before games the margin of safety before cascade collapse is razor thin. People should be very concerned about their individual responsibility for cirtical supplies – and the infrastructure companies have to be investing now in redundancies. Warnings like yours are so timely. -Mark
Well, thanks for that great post, Raph, which taught me a lot. Years ago, when the Internet of Things began to be prototyped in Second Life by various people including Joi Ito, I protested and said if those things ever came to RL I would stamp on them and crush them or pour cans of Coke on them. They wanted to put these RFID things everywhere which they called spimes, for space-time, you see. Groovy. They were going to help stop environmental pollution. That it would enable more spying and griefing wasn’t of concern.
As I followed the Internet of Things, I began to conceive of the title for a book I might do some day: “Why I Ran Screaming in Knock-Kneed Terror from the Internet of Things”. My beef with it aside from the obvious security/griefing issues was that it communalized property like a voracious Communist system. That is, you could buy a couch or a coffee maker but it wasn’t “yours” because it was attached to the IOT which enabled its features to work, and the coders who ran those wires essentially owned your thing — they could turn it on or off or render it useless, by design or mistake or maliciously. Things no longer had intrinsic value and were your property, even your house — the thermostat and door lock and such were all hooked up to this network that other forces would used to control life and make money. Capitalism for me, communism for thee, you would wind up with a lot of broken stuff. Let’s say some nerd didn’t like what you wrote on Twitter, he could cut off your coffee pot. Or a thousand variations.
To me, the key problem with the IOT is the collectivization. Everybody hates when I talk about communism, and you only come at it indirectly, but trust me, it really is the issue. It’s about whether you can really own and control your refrigerator so you won’t starve.
Of course, Trump and Putin are going to make some invisible cyber thing so I’m sure it will be fine.