Interoperability: Why Policy Makers Should Make It Their Personal Mission
In the fall of 1999, while working at Portland General Electric, I was given a unique opportunity to start building a dream that several of us in the industry had for many years: Build a virtual peaking power plant using customer owned backup generators. This program, eventually called the Dispatchable Standby Generation (DSG) program, has become a very successful program for PGE and recently exceeded its original goal of acquiring 100 MW nameplate of peaking capacity for the utility.
For the initial pilot project in late 1999 and early 2000, however, we began with only one 500 kW Katolight generator as part of the program. That generator was located at a state facility, the MacLaren Youth Correctional Facility. That little generator created many headaches for our team, but, together with the generator control system we purchased from EnCorp, it also taught us many hard lessons related to interoperability.
That fall and winter of 1999-2000 was extremely cold and wet, and the generator and the switchgear/generator-controller sat outdoors on a small piece of property just outside MacLaren’s security fence. To keep relatively dry, we built a rag-tag shelter with tarps to keep the rain off. For a short time, we had the luxury of a small, unheated construction trailer left behind by EC Power after it installed the generator.
Our first challenge, after getting grid power to the installation, was to get the EnCorp Generator Controller to communicate with the Katolight generator. We followed the manufacturers’ recommendations to the letter, using an expensive grounded communication cable—“the fancy cable,” we called it. We wired up the system and set about testing. It only “sort of worked.” It was inconsistent—sometimes it worked, sometimes it didn’t. We kept testing, our frustration growing every day as we tried to fight off hypothermia. After weeks of trial and error, we learned that the system did not work because of inductance issues on the fancy cable; we discovered that substituting a simple low cost twisted pair of wires—speaker wire, I called it—did the trick.
That was only the beginning. Next we had issues with the generator controller not talking with the software system due to interface problems. My boss wondered why we were spending several months in the field and why costs were rising on the installation.
Fast forward twelve years to 2012 for PGE’s Salem Smart Power Project. Once again inductance and interface issues made communication between smart inverters and the 5 MW lithium-Ion battery storage system challenging
for our team. In our brave new world of inverter-based distributed energy resources (DER), plug and play is still the dream – not the reality.
Interoperability should be simple. When you purchase a piece of equipment that’s designed to communicate with another piece of equipment, they should work the first time. There are probably some energy policy makers out there that believe that’s the way things actually work: you plug it in and “there you go.”
I can tell you that is not how it works. Rarely, in the distributed generation or demand response equipment areas, do systems inter-operate. In fact, each generator or demand response site poses its own unique challenges to inter-device communication.
Let me give another example; Kettle Foods in Salem, Oregon, a major potato and other chip maker, installed one of the first greater-than 100 kW solar arrays in the Pacific Northwest. I wanted to obtain the data from that system. PGE had recently been experimenting with a new Ethernet radio system that seemed to offer simple wireless communication between field DER and our fairly new control system at PGE’s headquarters. We call that control system “GenOnSys.” We could also use a PGE SCADA (System Control and Data Acquisition) radio communications tower in West Salem to receive signals from Kettle Foods. This could ultimately provide a slick system for smaller generators in the Salem area to participate in the dispatchable standby generation program. In addition, solar or demand response systems could potentially utilize that approach in the future.
Once again, the ugly reality of non- interoperability raised its head. The solar array’s inverter at Kettle Foods did not offer a way to communicate. Generally it is best to get information directly from the solar inverter, the device that converts DC power from the solar panels to AC power for the utility. That inverter was not capable of communicating with PGE. The meter used on the project, a SquareD PowerLogic meter, should have made communication easy. I’d had a lot of experience with that system, so it should have been a piece of cake. Not so: the radios we were using were Ethernet-based, similar to what you might find on your home wireless internet system. The meter only communicated via serial Modbus. That meant we needed a separate black box, a Modbus to Ethernet Converter, to communicate. Once again, several of us tried, carefully following the manuals, to make the systems talk. It took too long to try to make it work. Finally, PGE’s most experienced SCADA tech, who had worked his entire career on serial communications, got the system to work, but only after a trial and error process of rewiring each pin connection of the cables several times to find the right combination for the serial ports.
After this experience, along with several years of having to re-wire serial communication pin-outs, I started speaking at conferences about the need to standardize with Ethernet communications for distributed energy resource (DER) installations. To this day, many generators and solar inverters only offer serial communications. Why? Because it’s cheaper for the manufacturer. Once the manufacturer sets up the system, their techs can use it over and over and it works pretty well. Also, by using this type of proprietary system, the manufacturer believes it forces you to use its products and services for communications. Which, by the way, works: Most utilities or industrial customers will stick with a particular manufacturer and make its products their standard—thus achieving interoperability, but at the cost of forgoing more robust competition.
For the many of us stuck with having to integrate different manufacturers’ equipment, it requires learning how each manufacturer’s devices communicate and adapt to their various protocols—each situation a one- off—which causes much wasted labor and longer project completion times.
The hidden cost of non-interoperable equipment? Millions daily in time and labor
By now you may dismiss this issue as being too technical. I agree, device communications should be simple. The fact is, on every single generator, solar array inverter, fuel cell, battery storage system, thermostat, communicating electric meter and the myriad converter boxes I’ve tried to connect is that they don’t speak a common language or don’t connect in a common way. I believe millions of dollars in this country and the world are wasted each day in trying to make grid devices communicate with each other. Based on my experience, a good one-third to half of the installation costs of a grid DER system is the labor to allow the systems to communicate with each other. In my observation, the three biggest issues causing cost overruns on projects are equipment shipping delays, software modifications and interoperability issues.
The bottom line is this: If we are to have a successful smart grid world or Internet of Things (IoT), this lack of interoperability cannot continue. Instead, energy policies must be established that embrace and nurture device communication interoperability.
Now that I’ve retired from PGE, my biggest consulting mission is working to understand the root causes of interoperability issues and solving them. I’m currently supporting the National Institute of Standards and Technology (NIST) and their Smart Grid Interoperability Panel (SGIP) and working with a company called QualityLogic that understands how to reduce interoperability problems. Their testing and certification systems have helped eliminate issues we had in the 1980s and 1990s of getting our PCs to talk simply and easily with our printers, so that fonts and layouts that you see on the screen show up on the printed page. QualityLogic has now set its sights on smart grid devices.
Part of my recent work for Quality Logic was with the Pacific Northwest Smart Grid Demonstration Project. Each of the 11 utilities participating in this ambitious project documented interoperability issues via a detailed survey that we conducted. Based on survey results, we learned that most of the system interfaces used between devices were proprietary. This means that each utility and almost all devices—including smart appliances, water heaters, generators, battery storage systems, lighting systems, thermostat systems, etc.—had a unique way of communicating. Each device had to be de- bugged, reconfigured, re-wired, had special black boxes installed that converted one communication protocol to another, all in a customized way that likely can’t be used again. In several cases, the device manufacturers actually claimed that their equipment’s communication was open- standard.