Imagine you are in intensive care in a hospital and your breathing is being assisted by an electrically operated ventilator that is quietly humming next to your bed. Suddenly the humming ceases because the ventilator has stopped working, and you begin struggling for air. The ventilator begins again for a few seconds—and then stops completely. This frightening situation was reality for a number of patients at one hospital. This article describes a hospital power outage and discusses what could have been done to prevent it.
A lengthy chain of events led to this critical point, but it all started with overcurrent devices for the facility that were not selectively coordinated and a motor winding faulted to ground. This hospital had completed a coordination study some time ago, but its implementation had been delayed due to various department managers' refusal to permit a scheduled, area shutdown during the evening. The areas experiencing an outage would be transferred to emergency power during the outage time or, in the case of emergency circuits, they would continue to have normal power but emergency power would be unavailable during the time required to set the breaker trip units.
The problem began when the windings of 20-hp induction motor faulted to ground. The motor happened to be driving a fan in an air handling unit and was connected to the equipment system branch of the essential electrical system. The associated motor starter was located in a multisection motor control center that served the essential mechanical systems for several floors of one wing of the hospital. The heater element in the starter was sized larger than appropriate for the motor full-load current, so the overloads did not disconnect the motor starter when the winding faulted. In addition, the motor branch circuit, short-circuit, and ground-fault protective device did not detect the fault and, therefore, did not trip the motor and starter off the electrical system.
As the ground fault traveled upstream, the feeder breaker in the distribution panel was a standard breaker (without ground fault), so it also did not detect the ground fault in the motor. The lower level fault current in the phase conductor was too small to trip this feeder breaker, so the ground fault continued upstream.
The substation feeding this portion of the distribution system was a typical, liquid-filled 5 kV 480/277 Vac transformer with both main and feeder breakers, which are both power-air breakers. When the feeder breaker saw the ground fault, the fault current was too small to be detected by the breaker tripping elements. The substation was equipped with two-level ground-fault detection in accordance with National Electric Code (NEC) 517.17 (B ), which requires that both the main breaker and the first set of overcurrent protective devices (OPD) downstream from the main have ground fault. The random factory breaker trip settings were still in place, so the trip levels and timing on the ground fault detection for the feeder breaker and the main breaker were reversed. Thus, the main breaker tripped on ground fault, shutting off power to all the normal and essential electrical system loads served by the substation.
The transfer switches for the essential electrical system detected an absence of normal voltage and transmitted a signal to the emergency generators, which promptly started. These generators were connected to an automatic paralleling system, which closed onto the first generator that came up to operational voltage and then paralleled the remaining generators onto the emergency bus. Power was then restored to the emergency side of the transfer switches, which had lost power due to the substation main-breaker trip. The transfer switches sequentially transferred from the de-energized normal source to the live, emergency source, with the life safety branch transferring first within its requisite 10 sec. Emergency, exit, and egress lighting were immediately restored. The critical branch transfer switch then timed out and transferred to emergency power, which reenergized the nurse call system, ventilators, and other critical care loads. Finally, the equipment system branch transfer switch transferred to emergency power, but this is when other problems started to occur.
When power was restored to the motor control center, the starters began automatically restarting their respective motors. However, when the starter for the faulted motor closed, the winding ground fault was still there because neither the overloads nor the starter OPD had taken the circuit offline. Back went the ground fault through the distribution panels, transfer switches, emergency distribution, and finally to the main paralleling equipment.
The feeder breaker from the paralleling equipment had been designed or was supplied with ground fault tripping and, upon sensing a ground fault, immediately tripped, removing all emergency power to the essential electrical system loads. The reasoning behind NEC Article 700.26 concerning the use of ground-fault tripping on emergency circuits is apparent.
More than two hours later, power was restored to the four floors that had been completely black. The hospital personnel had to restore power by identifying the faulted motor and taking it off line. Once the fault was cleared, emergency power was restored to the area so that the chance of harm to the patients was minimized. Then the feeder breakers in the substation were turned off so that the initial power surge would not damage any equipment.
After energizing the substation, the feeder breakers were sequentially closed re-energizing each of the distribution panels and motor control centers.
As normal power returned, the transfer switch controls initiated the timed sequence for their loads to return to normal power, and the generator cool-down procedure started. So that patients on electrically powered ventilators could continue to breathe, all available nurses were called from other parts of the hospital to provide manual ventilation during the outage. Due to these efforts, there were no patient deaths or injuries.
The Code Perspective
The NEC requires that overcurrent devices in emergency distribution systems be selectively coordinated. Specifically, NEC Article 700.27 states, “Emergency system(s) overcurrent devices shall be selectively coordinated with all supply side overcurrent protective devices.” This sentence refers to health care emergency systems, whether the life safety branch or the critical branch overcurrent devices (see Figure 1).
Figure 1: Typical equipment/loads on life safety branches and critical branches. Source: Kenneth Lovorn
If the phrase, “Emergency distribution system” had been “Essential electrical system,” then the breakers feeding the equipment system branch also would have to be included under this requirement.
The life safety loads include:
Illumination of means of egress
Alarm and alerting systems such as fire alarm and medical gases
Generator set room lighting
Elevators (at least one per elevator lobby) and their associated communications and controls
Automatic doors that must be energized to allow them to function.
The critical branch loads include:
Critical task illumination in anesthetizing gas locations
Selected receptacles associated with anesthetizing gases
Isolated power systems
Selected illumination and receptacles in patient care areas
Psychiatric bed area illumination
Nurse station illumination and selected receptacles
Nurse call systems
Blood, bone, and tissue banks
Telephone equipment and rooms
Specialized task illumination and critical power in selected treatment and diagnostic areas
Additional illumination and selected power circuits not previously listed required to assure effective hospital operation.
Article 517.17 of the NEC , paragraph (C)—Selectivity, requires that “Ground-fault protection (GFP) for operation of the service and feeder disconnecting means shall be fully selective such that the feeder device, but not the service device, shall open on ground faults on the load side of the feeder device.” This section appears to require that the GFP be selectively coordinated for emergency systems as well, as it does not delimit this requirement to any one system.
We find that NEC Article 700.267 specifically addresses this situation when it says, “…emergency systems shall not be required to have ground-fault protection.”
If the motor overload element had been closely sized to the motor characteristics, if the substation ground fault protection was properly coordinated, or if the ground fault elements were not included in the emergency distribution system, this incident never would have occurred.
However, performing a full, selective coordination study is not enough; implementing the study so that all of the settings are made on the respective breakers and the fuse types are adjusted so that the system is fully, selectively coordinated.
If maintenance outages on the power system cannot be scheduled, a facility inevitably will be performing maintenance during outages. In this case, the coordination study settings were implemented once the hospital could schedule the electrical service organization to do so. Everything was adjusted within two months of the incident.
Lovorn is president of Lovorn Engineering Assocs. He has 39 years of design and engineering management experience with architect-engineers and consulting engineers designing electrical systems. He is a member of Consulting-Specifying Engineer's editorial advisory board.
For Further reading, access these articles at www.csemag.com:
Isolated power systems for healthcare, Keith Lane, PE, Jan. 2008
Top five NEC 708 guidelines, Tommy Buford, PE, Sept. 2009
Coordination conflict, Patrick Lynch, et. al. Nov. 2009
Selective coordination increases reliability of emergency systems, Tim Crnko, Dec. 2009