At Optigo Networks, we firmly believe IT and OT teams can work together in any number of ways. Your network could be separate or converged, and you could assign responsibilities based on expertise or device type. The only “wrong” way to work together is not at all.
With that in mind, we’re exploring all the ways you and your colleagues can team up. What workflows can you put in place? How can you assign responsibilities? How We Work is a series where we dig into the technical and social sides of managing building networks.
We chatted with Adam Boltz, controls engineer, and Kevin Gift, associate director of critical infrastructure from Princeton University about their IT/OT workflows. We found out how they’ve designed a system that works, and why a small data centre fire helped bring them closer together.
Read on for the recap, and watch the recording below!
Kevin and Adam have worked together from day one, in a way that’s unique to Princeton University. That’s because Kevin’s role in IT was created specifically to support the Operational Technology (OT) team. From the very beginning, he was reaching out and working with OT, facilities, and controls engineering to help them on projects. They collaborate on all sorts of initiatives whether big (evacuating smoke after a server fire) or small (assigning IP addresses to new devices). Whoever’s driving a new project will go to the other team for support, whatever they need.
Kevin’s background is heavy in IT, but he also studied mechanical engineering and worked in robotics. His diverse experience certainly helps him as he works with the different departments that fall under critical infrastructure.
Even more significantly, he understands how important the campus’ OT is, and that IT and OT are dependent on one another for their success. They both need each other to do their jobs well, so it’s a collaborative, give-and-take relationship. Kevin works very closely with the controls engineering department on their data centres in particular, to make sure everything’s up and running properly.
There has of course been work on both sides to learn more about each other’s differences, and how they can better support one another.
Adam explained that it came up a while back that they needed to create a private VLAN for the building automation system at Princeton. It would be easier to maintain, especially with broadcast messaging on their massive network. They had a meeting with the heads of all the departments to explain how important these VLANs were, so the network wasn’t in danger of being overwhelmed with broadcasts. Now, not everyone understands just how big of a role building automation plays, from lights and HVAC to life safety, animal health, and more. But through that conversation, it was quickly clear to everyone how important it is that the building automation system was reliable and easy to maintain.
A small server fire in their data centre back in 2016 also solidified their relationship — though they don’t recommend starting any fires to get IT and OT working together! Fortunately, the fire didn’t cause any damage to the data centre, and they already had processes in place for emergencies. The experience did expose some gaps in their processes, though, which they’ve since collaborated on fixing.
One of the biggest gaps was in responding to a smoky environment. Where there’s fire, there will be smoke (even if it’s a relatively small fire), and the data centre wasn’t set up to evacuate smoke quickly. Kevin started working on a vendor quote for the data centre to reduce risks and get rid of smoke efficiently in the event of another fire. He poked his head into the controls engineering office and caught Adam and his colleagues at a rare quiet moment to ask for help. Well, within 15 minutes, Adam and the controls engineering team had figured out a way to get a valve working so they didn’t even need help from the vendor.
This just goes to show how invaluable it is for IT and OT to understand one another, and to work together on an ongoing basis. The departments might have separate projects from day to day, but they are dependent on one another for their success. If Kevin and Adam hadn’t made an effort to see each other’s values, priorities, and differences, they might not be able to work so well together. Beyond that original effort, though, they’re conscientious to this day about keeping the lines of communication open. Kevin actually “hotels” in the OT office once a week. All of this helps to strengthen their working relationship. If they hadn’t built that relationship, odds are when Kevin “poked his head in to ask about the smoke,” it probably wouldn’t have gone over well. And, when Adam’s team needed help with VLANs and IP addresses, they might not have gotten support as quickly.
Now, they can both bring up a problem and get the help they need, even if it just starts with a casual conversation. Whether it’s VLANs or a smoky data center, the IT and OT teams at Princeton always have each other’s backs.
