Buildings IoT

Convergence is here, so what does that mean for IT and OT teams?

By Brian Turner | May 3, 2018

iot-convergence-hero

In the interview with OTI’s new VP of Information Technology, we outlined the dynamic conversation surrounding OT networks and IT collaboration in a building project. That same week, I sat on a panel session at Niagara Summit called “The Value of OT Networks.” It quickly turned into the “IT vs. OT” debate we outlined in that blog article. In my view, we’re well past that. We’re no longer looking at OT devices just sitting on the IT network, or two completely separate networks. It’s now a truly converged IoT network where each team needs to understand and work together on a bigger picture.

What is Convergence in Smart Buildings?

For buildings and OT, convergence has accelerated as the price of enabling technologies has decreased. Controllers the size of your hand now come with IP connectivity and they’re purpose-built for building automation. Fiber has advanced so that it’s now cost-competitive with Cat5 cabling and much more scalable. On the information technology side, IT teams are faced with larger, more connected networks that offer access holes to malicious parties at every turn.

On the Buildings IoT “One Network” Debate and What’s Been Missing from the Conversation

When OT comes to the table for a building network strategy session, we’re talking to a room full of IT people about network and fiber like it’s new. IT has been doing this for a long time. The only thing that’s new is the application. In order to better work together, OT needs to help IT know what the device is, what network ports it is listening to and how often, how much bandwidth it needs and most importantly, which device starts the conversation.

When a Switch is Not Just a Switch

While OT is leveling-up on the technology side, IT needs to reframe the way it thinks about building devices.

“To IT, it’s just a light switch,” says Rich Miller, VP of Information Technology at OTI. “It doesn’t need very much attention. Instead, IT is focused on making sure all users have email access or that the personal devices people bring onto the office network are not posing threats to overall security.”

As convergence deepens, building devices are no longer just light switches. Devices are WiFi enabled, IP connected, power over ethernet, data sharing smart devices. They are part of a larger effort to improve operations, maximize profitability and keep people comfortable. They’re flexible and programmable because they’re not inherently secure.

Enterprise case study on truly converged IoT Network for GGP mall properties

“The crucial point that both OT and IT teams need to understand,” Miller says, “is that if you don’t secure every one of these OT building devices, it could be the jump-off point that ends up getting a whole office or corporate enterprise hacked. It is everyone’s mutual responsibility to ensure that doesn’t happen.”

How to Make Convergence Work Better for Everyone

Traditionally, IT has seen facilities as outside their network purview and OT has seen information technology teams as roadblocks to on-time and in-budget project delivery. Convergence has arrived and the internet of things is only going to continue intertwining devices, teams and networks. Everyone needs to work together otherwise networks will have security holes, machine learning will halt and entire industries will fall behind. Here’s how both IT and OT groups can better work together.

1.) Stop dreading the conversation

There’s a huge misconception that working with IT can push an OT project behind by three to six months.

“The mentality in the buildings industry has been ‘we don’t want to deal with IT, they’re a road block, we just want to do our own thing,’” explains Clint Bradford, head of operations at OTI.

The truth is, a clear path can be agreed upon in one to two meetings. Calling IT at the last minute saying “I need 100 ports open for my VLAN” is never a successful strategy. Neither is working around IT and placing a router anywhere that OT teams need access to a platform front-end. The first path is unfair and the second is not secure.

Data center case study on working with IT for 352 pieces of equipment

2.) Talk early and at key project points

“It always comes down to the relationship,” Miller says. “The more someone trusts you and the work you do, the more they’re willing to work with you to accomplish a mutual goal.”

Convergence means that IT is part of the OT network, OT is part of the IT network and everything works together for the sake of the IoT.  With this in mind, it only makes sense to have both IT and OT start working together early in the project process.

OTI is going all-in on IT/OT collaboration

3.) Outline exactly what you need

Building controllers and network devices have advanced so much that it’s hard for even industry veterans to keep up. Imagine how out-of-the-loop IT is. There are a few specific reasons IT must be involved in OT network integrations. Conversations should clearly outline these needs. First, discuss bandwidth needed for each device on the network. Then explain the important features and functions of the system and latency/up-time required. Finally, determine port assignments, data transfer and storage methodologies.

Convergence means that it’s not just OT devices sitting on an IT network. It’s a true IoT network where all applications share relevant data important to each application. With everyone on board, projects will go smoother, networks will be more secure and people will be more successful.

OTI News

Thrilled to welcome Rob Vandenberg as Director of Managed Services to OTI

By Brian Turner | February 15, 2018

rob-blog

This week we announced the latest new member of our team. Rob Vandenberg joins to build out our managed services. He comes to us from the airline industry, where he developed a leading software platform for asset and maintenance management.

Read details about Rob and why we’re so excited to have him on our team in the press release. Share your own warm wishes with Rob on LinkedIn and keep up with OTI on the go by signing up for our newsletter. February edition coming soon!

Buildings IoT

The Agony of Making an IOT Decision

By Brian Turner | October 3, 2017

Consider this scenario: A corporate real-estate executive is contemplating a presentation to the Board of Directors. She is experienced at operating efficient class A properties, often improving the bottom line and increasing revenue for the buildings in her portfolio. It’s clear to her that IoT is a buzz word. In her personal life, she’s very tech-savvy, accustomed to accessing everything within a few touches on a screen. Everything, that is, except for the operational technology assets in her buildings.

At home, she can let a visitor in while sitting at her favorite coffee shop. She can  watch them enter and exit via IP-connected cameras. Why she doesn’t have this technology available in all her buildings? Her company has invested millions of dollars, under her leadership. Something isn’t right.

This exec wants her office buildings to be the ideal location for all tenants. She wants to provide  the latest technology for security, parking, elevators, guest access, Wi-Fi, heating, AC and lighting. The latest in comfort and healthy environments as prescribed by industry groups like LEED and ASHRAE are crucial too.

Sound familiar? Now…how does she get started? And how does she get it right?

Where the conversation starts and sometimes stops

This scenario is playing out for busy real estate executives around the world every day. Somewhat paradoxically, technology and the expanding list of innovative IoT manufacturers are making it harder to answer that very first question – where do I begin on revolutionary projects for commercial office buildings?

Once these real estate execs start to look for the best solutions to fit their needs, it seems everyone and their brother has one to offer. Some solutions come from household names, while others come from agile businesses working to bring the next big technology to the industry. They’ve all got the best interface and the easiest to use integration platform. Plus all of their employees know how to solve any problem, no matter how obscure. There seems to be no end to the amount of money or energy that can be saved when these technologies are installed. Great! I’ll take three! Right…

Let’s try to make sense of all of this. As a master systems integrator, I must make sure my team knows about the new latest and greatest technology available, so I attend several conferences per year to see what the market has to offer. When I look out on the composite of these show floors, I see hundreds of access control vendors, hundreds of HVAC control manufacturers, metering solutions from big and small vendors, thousands of smart equipment and device manufacturers and hundreds more lighting control manufacturers. Each of the products and solutions have merit, but there is a lot of cross-over in function and benefit.

Promising platforms

The second, more complex issue, is that integration means different things to different people and thus produces a wide variety of results and solutions. Artificial Intelligence is promising, and providing, in some cases, fantastic results in buildings. Building analytics platforms are also getting better at giving users the information they really need, in a form that helps them act, again aided in some applications by AI.

Remote connectivity is another area where innovations are starting to see positive returns, helping to get the building automation industry closer to IT standards of security and management.

I have read a lot of stories lamenting failed IoT projects and how one technology or process could have solved the problem. Some of the solutions sound reasonable and some are a stretch. At the end of the day, we are all human, trying to solve human problems. What’s great and also extremely difficult about right now is we have more tools than ever before to address these problems. The overwhelming abundance of shiny new things is, in some ways, paralyzing us.

The Good News

The good news is that we don’t have very far to look for the answers. For those busy real estate execs, the IT group has some of the answers. The facilities, or OT group has some of the answers. The OT master systems integrator (OT MSI) has some of the answers.

Together, as the IoT team, we will get it mostly right. These three entities are the keystones to making solid, well informed IoT decisions. You likely already have your IT team, so a first step will be to find a good OT MSI. Look for one who takes the time to understand your business goals and who will work with your IT and OT teams to make decisions together. By working as an IoT team, the agony of each IoT decision reduces while the likelihood for success improves.

Now go forth and make an IoT decision!

Master Systems Integrators

The Role of the OT Master Systems Integrator

By Brian Turner | September 22, 2017

A good Master Systems Integrator (MSI) can do a lot of things. Primarily, MSIs help building owners and IT groups navigate the IoT. The MSI should offer a host of system-specific knowledge. This includes deep understanding of HVAC controls, lighting controls, or energy monitoring in any number of combinations.

The most successful MSIs also have some IT expertise on staff to help with the design and implementation of the OT network.

master systems integrator for building controls projects

Challenges

The OT challenge for IT is never the connection. The process to connect OT devices is no different than the process for most IT devices. The biggest knowledge gap between IT and OT teams is the application of the devices. Also, how the OT systems transmit and use the building data.

This chasm will not close quickly because OT devices like sensors are rapidly multiplying within buildings’ OT systems. In these discussions, the MSI is a consultant to the building owner. MSIs point out the right course for optimal OT functionality within the IT purview. They also represent the owner in discussions with the specific OT vendors.

A key thing to note: it is rare for MSIs to actually install the OT systems connected to the IT backbone. This remains the domain of controls contractors, and rightfully so.

Rethinking the MSI Role

So far, the MSI has not been elevated to the role of an architect, security consultant, network provider, or ERP. It is common for enterprises to standardize on technology like Cisco for network infrastructure and operating systems. It is also common to standardize on access control and video platforms, even contractors. But there is little standardization among electrical distribution systems, HVAC control, or lighting control.

Given the growing role of data, semantic tagging, analytics, network access, and customer specific knowledge required for successful IoT projects, it is now time to consider building-wide standardization for operational technology systems, and to rethink the owner/operator relationship with master systems integrators.

Most MSIs today grew out of HVAC controls companies. So in many cases, HVAC controls are still in their DNA. This knowledge, when applied to OT is invaluable. But that controls background has made it difficult for some MSIs to put the controls business in the back seat as they work to determine what makes sense for a building owner and their IT team. This history can also make it difficult to be objective when an owner wants to use a technology other than the one the MSI has most familiarity with.

Evolve or Perish

While the term Master Systems Integrator may be a new one for the building automation industry, those who call themselves MSIs today don’t just have former HVAC controls contractors to worry about as competition.

There is a growing crop of IT consulting firms working on building problems. They have full knowledge of how to perform the MSI role without legacy loyalties to specific products. These IT MSIs also don’t get bogged down by implementing OT sub-systems. Those IT consulting firms don’t yet have OT fully figured out, but you can bet they’re working on it, through acquisitions, field experience or both.

While there is a bit of consulting in the MSI role, master systems integrators must provide more than an analyst’s-eye-view of challenges. We will need to engage with clients for many years, with active roles and solid seats at the decision-making table. Where I believe MSIs can offer the most value is in OT infrastructure, integrated databases, building analytics and data visualization.

These areas are increasingly relevant to IT groups and the C-suite they answer to. While the MSIs from legacy IT consultancies get up to speed on OT systems, those with controls experience will rise. That is, if they’re able to translate system-specific knowledge to broader understanding of a whole-building network.

With all of these changes and new opportunities, it’s an exciting time to be in the expanding building automation industry.

Buildings IoT

On the Buildings IoT “One Network” Debate and What’s Been Missing from the Conversation

By Brian Turner | September 14, 2017

Buildings IoT and its broad counterpart the Internet of Things is sparking debate around physical building networks and where best to implement enterprise solutions that touch both IT and OT. At OTI, we have:

The Right Questions

The real debate is not around proving there is one right way to implement an IoT strategy for any one network – there are use cases where it’s clear which one of the three options is best. And just because one network architecture works in one implementation doesn’t mean it will make sense for the next one.

The question to be answered when considering where IoT solutions should plug into a network is how will the human interaction be impacted when building devices communicate over IP networks rather than RS-485 networks? Once we understand the human side of the equation, we can more accurately define how the network should be architected and how IT and Facilities (also known as operational technology, or OT) should engage with the project.

Programmer holding laptop and checking machine

The ground floor – technicians and controllers.

Technicians need continual access building devices in a convenient, efficient way. When controllers are on an RS-485 network like BACnet MSTP, technicians have unencumbered access for programming, data sharing, and commissioning. When these devices include IP connections, they need an IT network.

In most cases, when a technician today needs to create a network for an IP-connected building device, they bypass IT and install CAT5 or CAT6 cables and cheap, unmanaged switches to go back and forth between controllers. They do this because ease of connectivity is integral to their jobs. They need to do continuous programming and commissioning on building devices and bothering IT to open a port every few days is untenable for both parties.

While the CAT5 workaround provides the technician the access they need, it can open up the corporate IT network to unwanted and unnecessary security risks.

The network layer – new solutions, new problems.

Many enterprise organizations around the world are working to solve the secure OT network problem, and several already have workable solutions available on the market. In all honesty though, the most effective solutions have mostly moved the burden to IT. This does solve the connectivity and security problems, but it adds a whole host of issues for both teams.

In an existing operation, it is straight forward to get new switches and ports assigned from IT for OT systems. The problem is not in the complexity but in the delivery. In my experience, there are often significant delays to integration projects because of IT-related hold-ups. This is mostly due to lack of experience with and knowledge of the OT devices, operating systems, personnel and services required to integrate building systems.

The problem is exacerbated in new constructions because IT isn’t there until the building is occupied. This can be weeks or months after the building systems need to be online. OTI has been involved in several projects where 80-90% of the devices are connected via IP and need to be online well before the IT staff is ready for them.

We have managed through the project implementations and have worked with IT groups to make sure we are installing products and cabling they will be prepared to support once they are on site, but this is far from a perfect process so far.

Maintenance Difficulties

  1. How will IT and Facilities work together to maintain these networks?
  2. How will IT respond to the service needs of OT?
  3. Will OT control their own destiny or will they be tied to IT for all support and troubleshooting?

I was moderating a session at IBCon in San Diego earlier this year where the “One Building One Network” question was a leading topic. I said something to the effect that OT needs to the own their network and control their destiny. This was taken out of context by some so I will take this opportunity to explain the nuance.

Two problems must be addressed as we consider the proper technical backbone for both IT and OT networks:

  1. Technicians and operators maintaining building systems have been handling operations for years without additional resources. They have become accustomed to diagnosing problems with RS-485 and Lon networks. They’ve accumulated a lot of expertise in troubleshooting these systems.
  2. The IT requirement of one port per connected device. This will need to change in order to cost effectively implement large scale deployments of OT devices.

There is no doubt it makes sense to manage one network infrastructure for all things connected to the IT network. It also makes sense that the IT professionals should manage the network, at all levels. The part where I deviate from the “One Network” pack is applications on the network.

I believe OT staff needs to be in control of network related to devices and systems defined as Operational Technology. These systems are HVAC controls, lighting – anything considered part of the operation of a building or campus.

AAEAAQAAAAAAAAsoAAAAJDZjMzY4ZTEwLTcwMjctNDk4YS04ZDczLTRmNjNiOTlhY2E3Zg

This means IT must provide tools and access to OT staff. It can be very complex to grant access to certain management tools without creating security risks on the IT network.

The new future – why “us against them” is the wrong way to go.

This is where new innovations are hard at work to eliminate these problems. The product we use is Optigo Connect by Optigo Networks, which employs passive optical networking (PON) to allow the OT segments of the IT backbone to be installed in a much more cost-effective way than traditional fiber infrastructure.

The user experience is also fairly intuitive. It allows the OT group to manage ports, port VLan assignments, and PoE. They can monitor the bandwidth and connection status to make sure devices are behaving properly and sharing data across the network.

The IT group still manages access, routing, security, firewall rules, and other traditional IT responsibilities. But the OT staff is empowered to “own” and operate the building systems.

It/OT Backbone part 2

The second part of this IT/OT backbone conversation is about ideology more than technical ability. To explain, let’s get technical for a minute with an example. Consider a floor with 30 VAV controllers serving conditioned air to offices and open areas on a typical floor. Manufacturers like Distech Controls and KMC have created VAV controllers that connect using IP cables.

When used in combination with the Optigo Connect, the ethernet switch supports the Rapid Spanning Tree Protocol (RSTP) as well as a ring monitoring function to automatically switch off redundant paths, and a broadcast storm protection function. This creates some redundancy if a connection is broken in the middle of the floor.

New vs. Traditional

If we were to use the traditional IT paradigm for this scenario, we would install 30 CAT5 cables that terminate in a single port on a network switch. This adds a lot of cost to the overall implementation and is not likely to perform at scale.

In the new paradigm, CAT5 cables are installed in a daisy chain requiring only 2 cables that terminate into 2 network switches. The only cost impact is the two ports and the material. The labor is identical. The advantages for network performance, data access, and stability are tremendous.

This is just one example. To evolve with the IoT presents daily challenges for IT, OT and the points at which they overlap. Rather than thinking of it as one network against the other, the IoT requires new thinking on the parts of both teams. The ability to find solutions that help everyone meet in the middle.