Why are our water and power utilities connected to the internet? Is it so that the employees controlling them can work from home? If so they are accepting too much risk relative to the benefit.
Because other replies aren't really stating it explicitly, let me add...
The water and power utilities are themselves large distributed systems. They need communications between elements just to function properly. They don't exist in a single location where people can go locally manage them in some air-gapped, offline fashion.
There is no option of not having a communication network to monitor and manage these geographically distributed elements. The question is which communication network you would use, and how you would secure it. Whether it is telephones, radio links, or people running around as messengers, it is still a communications network.
Will some "dedicated" network be any safer? If anything, I imagine the fantasy of a private network will lead to even less security. You cannot physically secure the entire signal path. You really need to treat it as untrusted and build your security on top with encryption, authentication, authorization, etc.
You really need to treat it as untrusted and build your security on top with encryption, authentication, authorization, etc
Infrastructure usually has a long lifetime.
Things become much more vulnerable as time rolls on e.g. we should be worried about AI hacking of smart meter firmware (hard to secure and expensive to upgrade).
Finally, I poked at something that wasn’t connected over USB but WiFi instead, the Elgato Key Light Mini. This one turned out to be way more interesting than I expected: it’s the only one with meaningful firmware integrity protection.
Unfortunately, while that’s an improvement over all of the other devices we’ve looked at, it protects the firmware at exactly one point in time: when an update is happening. It’s not a boot time check enforced by the bootloader or any other kind of secure boot scheme, and the updater happens to be running while everything else in the device is still operating, meaning there’s huge attack surface to try to disable that signature validation. I asked Claude to look for an exploit that might enable this, and it found a doozy
The water and power utilities are themselves large distributed systems.
For power I can understand. For water supply it is harder to understand. Does water supply have similar characteristics to power, for example can you turn on a reservoir when there is an “outage” in another?
think about a grid responding to demand. The old model (like the titanic) had a remote manager phoning the site lead who phoned the control room who phoned the engineering room.
Remote management has clear benefits, don’t be obtuse. I totally agree on the security risks, but the benefits are obvious.
I'm not seeing why it has to be connected to the internet. Make it a private, air-gapped intranet for all (or most) of the benefits of being "connected", but with no entry point for someone sitting on another continent to turn off the water.
Are you talking about the media layer or the application layer? What would be the alternative to using the internet media layer? Every utility running their own private media ? So a duplicate network of media, as broad physically as the internet, that’s not connected to the internet? That hackers could tap into, and with poorer security, because it wouldn’t be constantly probed.
I’m glad you raised this point. What are some of the better VPNs you’ve seen for secure industrial .
I asked because of books I had read about the bad ones, where unsecured industrial control protocols were exposed wirelessly , or via vpns. And I’ve been curious if any good ones are out there .
In addition to remote monitoring, central control over large systems clearly has benefits: quickly and automatically spinning up a power generation in one location in response to an outage or just increased demand in another one, or conversely spooling down generation in response to a large demand spike going away (many factories need to notify the grid before starting up or shutting down), shutting down water/gas flow upstream of a detected leak, or yes, working from home, which lots of people here regularly argue is a good thing.
You could absolutely make the case that it isn't worth the risk, but that isn't the same as not having benefits.
No, and it's not strictly impossible to correctly build out a SOC / SIEM and ingest all the logs you want and pay for the right engineers to make sure it all works correctly. But damned if anyone manages to do a great job in this area. It's too complex and too expensive, so almost everyone settles for "best effort."
A lot of problems are easy conceptually, but we can't manage to tackle them.
Comments
Why are our water and power utilities connected to the internet? Is it so that the employees controlling them can work from home? If so they are accepting too much risk relative to the benefit.
Because other replies aren't really stating it explicitly, let me add...
The water and power utilities are themselves large distributed systems. They need communications between elements just to function properly. They don't exist in a single location where people can go locally manage them in some air-gapped, offline fashion.
There is no option of not having a communication network to monitor and manage these geographically distributed elements. The question is which communication network you would use, and how you would secure it. Whether it is telephones, radio links, or people running around as messengers, it is still a communications network.
Will some "dedicated" network be any safer? If anything, I imagine the fantasy of a private network will lead to even less security. You cannot physically secure the entire signal path. You really need to treat it as untrusted and build your security on top with encryption, authentication, authorization, etc.
Infrastructure usually has a long lifetime.
Things become much more vulnerable as time rolls on e.g. we should be worried about AI hacking of smart meter firmware (hard to secure and expensive to upgrade).
Today's secure system is tomorrow's insecure system. E.g. https://news.ycombinator.com/item?id=49413320 :
For power I can understand. For water supply it is harder to understand. Does water supply have similar characteristics to power, for example can you turn on a reservoir when there is an “outage” in another?
That explains why they are networked, not why internet access is required.
The problem of secure networking has been solved long ago but there is no incentive for OT solution architects to get it right.
In the end real reason is always cost. Connecting straight to internet is extremely cheap. Anything else is somewhat more expensive.
think about a grid responding to demand. The old model (like the titanic) had a remote manager phoning the site lead who phoned the control room who phoned the engineering room.
Remote management has clear benefits, don’t be obtuse. I totally agree on the security risks, but the benefits are obvious.
I'm not seeing why it has to be connected to the internet. Make it a private, air-gapped intranet for all (or most) of the benefits of being "connected", but with no entry point for someone sitting on another continent to turn off the water.
Are you talking about the media layer or the application layer? What would be the alternative to using the internet media layer? Every utility running their own private media ? So a duplicate network of media, as broad physically as the internet, that’s not connected to the internet? That hackers could tap into, and with poorer security, because it wouldn’t be constantly probed.
mostly, because setting up a separate network is harder.
a virtual network built upon the regular internet is much easier.
And yes, mostly done wrong
I’m glad you raised this point. What are some of the better VPNs you’ve seen for secure industrial .
I asked because of books I had read about the bad ones, where unsecured industrial control protocols were exposed wirelessly , or via vpns. And I’ve been curious if any good ones are out there .
It makes sense to have them connected for a lot of reasons.
which reasons?
In addition to remote monitoring, central control over large systems clearly has benefits: quickly and automatically spinning up a power generation in one location in response to an outage or just increased demand in another one, or conversely spooling down generation in response to a large demand spike going away (many factories need to notify the grid before starting up or shutting down), shutting down water/gas flow upstream of a detected leak, or yes, working from home, which lots of people here regularly argue is a good thing.
You could absolutely make the case that it isn't worth the risk, but that isn't the same as not having benefits.
Remote monitoring is one of them presumably
But is it completely impossible to separate the monitoring and equipment/management?
Stupid example like the equipment itself is not networked, but you could watch via a webcam and/or get metrics via image recognition.
Maybe it's safer just to have somebody work the night shift.
No, and it's not strictly impossible to correctly build out a SOC / SIEM and ingest all the logs you want and pay for the right engineers to make sure it all works correctly. But damned if anyone manages to do a great job in this area. It's too complex and too expensive, so almost everyone settles for "best effort."
A lot of problems are easy conceptually, but we can't manage to tackle them.
Sounds like a good application of mesh networks.
Not really practical when the nodes are separated by large distances.