The Internet of Things is sparking debate around physical building networks and where best to implement enterprise solutions that touch both IT and OT. At OTI, we have:
- worked with IT to add OT devices and building systems within existing IT networks
- implemented OT networks completely outside the context of IT
- added OT networks to the IT network to create a converged IoT network
The real debate is not around proving there is one right way to implement an IoT strategy for any one network – there are use cases where it’s clear which one of the three options is best. And just because one network architecture works in one implementation doesn’t mean it will make sense for the next one. The question to be answered when considering where IoT solutions should plug into a network is how will the human interaction be impacted when building devices communicate over IP networks rather than RS-485 networks? Once we understand the human side of the equation, we can more accurately define how the network should be architected and how IT and Facilities (also known as operational technology, or OT) should engage with the project.
The ground floor – technicians and controllers. Technicians need continual access building devices in a convenient, efficient way. When controllers are installed on an RS-485 network like BACnet MSTP, technicians have unencumbered access to devices for programming, data sharing, and commissioning of the systems. When these devices include IP connections, they need to be added to a network that resembles an IT network.
In most cases, when a technician today needs to create a network for an IP-connected building device, they bypass IT and install CAT5 or CAT6 cables and cheap, unmanaged switches to go back and forth between controllers. They do this because ease of connectivity is integral to their jobs. They need to do continuous programming and commissioning on building devices and bothering IT to open a port every few days is untenable for both parties. While the CAT5 workaround provides the technician the access they need, it can open up the corporate IT network to unwanted and unnecessary security risks.
The network layer – new solutions, new problems. Many enterprise organizations around the world are working to solve the secure OT network problem, and several already have workable solutions available on the market. In all honesty though, the most effective solutions have mostly moved the burden to IT. This does solve the connectivity and security problems, but it adds a whole host of issues for both teams.
In an existing operation, it is straight forward to get new switches and ports assigned from IT for OT systems. The problem is not in the complexity but in the delivery. In my experience, there are often significant delays in getting integration projects completed because of IT-related hold-ups. This is mostly due to lack of experience with and knowledge of the OT devices, operating systems, personnel and services required to integrate building systems. Over time, the working relationship is bound to improve, though I am not sure it will ever be completely copacetic.
For a new construction project, the problem is exacerbated by the fact that IT isn’t put in place until the building is ready for occupancy. This is typically weeks or months after the building systems are required to be online and communicating. OTI has been involved in several projects where 80-90% of the devices are connected via IP and need to be online well before the IT staff is ready for them. We have managed through the project implementations and have worked with IT groups to make sure we are installing products and cabling they will be prepared to support once they are on site, but this is far from a perfect process so far.
- How will IT and Facilities work together to maintain these networks?
- How will IT respond to the service needs of OT?
- Will OT be able to control their own destiny or will they be tied to IT for all support and troubleshooting?
I was moderating a session at IBCon in San Diego earlier this year where the “One Building One Network” question was a leading topic. I said something to the effect that OT needs to the own their network and control their destiny. This was taken out of context by some in the audience, so I will take this opportunity to explain the nuance of my comments. There are two problems I have identified that must be addressed as we consider the proper technical backbone for both IT and OT networks.
- Technicians and operators responsible to maintain building systems have been successfully handling operations for years without the need for additional resources to manage the network. They have become accustomed to diagnosing problems with RS-485 and Lon networks and have accumulated a lot of expertise in troubleshooting these systems.
- The methodology used by the majority of IT departments requiring one port per connected device will need to change in order to cost effectively implement large scale deployments of OT devices.
There is no doubt it makes sense to manage one network infrastructure for all things connected to the IT network. It also makes sense that the IT professionals should manage the network, at all levels. The part where I typically deviate from the rest of the “One Network” pack is when it comes to the applications living on the network. I believe that the OT staff needs to be in control of the section of the network related to the devices and systems defined as Operational Technology. These systems are HVAC controls, lighting, and anything else that could be considered part of the operation of a building or campus.
This means the IT group must provide tools and access to the OT staff to manage and operate the OT portions of the network. This provides some challenges for IT and OT groups given there is so much specialty knowledge required to effectively manage an IT infrastructure. It can be very complex to allow access to certain management tools without creating security risks to other aspects of the IT network.
The new future – why “us against them” is the wrong way to go. This is where new innovations are hard at work to eliminate these problems. The product we use is Optigo Connect by Optigo Networks, which employs passive optical networking (PON) to allow the OT segments of the IT backbone to be installed in a much more cost-effective way than traditional fiber infrastructure. The user experience is also fairly intuitive and can be understood by most OT professionals in the field. It allows the OT group to manage ports, port vLan assignments, and PoE. They can monitor the bandwidth and connection status to make sure devices are behaving properly and sharing data across the network.
The IT group still manages access, routing, security, firewall rules, and other traditional IT responsibilities but the OT staff is empowered to “own” and operate the building systems in a way they are accustomed.
The second part of this IT/OT backbone conversation is about ideology more than technical ability. To explain, let’s get technical for a minute with an example: Consider a floor with 30 VAV controllers serving conditioned air to the offices and open areas on a typical building floor. Manufacturers like Distech Controls and KMC have created VAV controllers that connect using IP cables. When used in combination with the Optigo Connect products, the ethernet switch in the controller supports the Rapid Spanning Tree Protocol (RSTP), as well as a ring monitoring function to automatically switch off redundant paths, and a broadcast storm protection function. This provides some redundancy in the network to keep the devices online, even when the connection is broken in the middle of the floor.
If we were to use the traditional IT paradigm for this representative scenario, we would install 30 CAT5 cables that would all terminate in a single port on a network switch. This adds a lot of cost to the overall implementation and is not likely to be performed at scale.
In the new paradigm, the CAT5 cables would be installed in a daisy chain fashion requiring only 2 CAT5 cables that terminate into 2 network switch ports. The only cost impact is the two ports and the material. The labor is identical. The advantages for network performance, data access, and stability are tremendous.
This is just one example. To evolve with the Internet of Things certainly presents daily challenges for IT, OT and the points at which they need to overlap. Rather than thinking of it as one network against the other, the proper backbone design and operation to meet the demands of the IoT requires new thinking on the parts of both teams and the ability to find solutions that help everyone meet in the middle.