In May of 2014, the National Public Safety Telecommunications Council (NPSTC) published a report entitled, “Defining Public Safety Grade Systems and Facilities.” I was fortunate to be part of the group that worked on the site portion of this document. If you have not read it, I suggest you do. Our report, which discusses radio sites and what is needed, differences between geographic areas, and recommendations, was submitted to the Public Safety Advisory Committee (PSAC), which represents the public-safety community and reports to FirstNet the Authority.
There are sections addressing seismic events, wildland fires, flooding, wind events, ice storms, power-grid failures, and geographical-specific events. In this Advocate I will report on Section 9, Site Hardening. This chapter was created by the Association of Public-Safety Officials (APCO) broadband committee and, as noted, it was edited to fit the form of the report. I was vice-chair when we worked on this section, led by Bill Schrier who now works for the FirstNet Authority. As part of our research, we compiled a list of previously-published documents about site hardening.
We went beyond what other documents recommended and hopefully were able to translate standard jargon to practical advice for those retrofitting or building new communications sites. The general requirements section includes the following table:
“The following requirements are general in nature and apply across all requirements sections.
120. Critical communications sites, such as public safety sites, SHALL adhere to all legally applicable local and national standards and practices as defined by the local and state building, electrical, fire, and other applicable codes.
121. In the case where legally applicable codes differ from the standards, practices, and other requirements within this document, the more stringent or rigorous requirements SHALL be applied.
122. In the case where the legally applicable codes are in conflict with the requirements within this document, the legally applicable code SHALL be followed.
123. In the case where the legally applicable code is in conflict with the requirements within this document the site designer/developer SHOULD attempt to meet the intent of the requirements of this document without violating the legally applicable code requirement. Alternatively, the site designer/developer may attempt gain an exemption or waiver of the legally applicable code requirement.
124. If the referenced standards (e.g., TIA 222 Rev G) are updated or amended, the newest revision SHALL apply unless otherwise noted. However, sites SHOULD only have to comply with the most current standard at the time of construction or site modification.”
Section 9.6.6 then delves deeper with discussions of site security issues and damage to a site’s electrical feeds, telco (fiber) lines, and antenna lines. The section on power states the following:
“Clean, reliable electrical power is paramount to highly available wireless communications sites. Availability of power to communications equipment is the fundamental limiting factor regarding the in-service state of the equipment. The causes of loss in commercial power can vary from natural events such as ice storms and high winds to manmade failures such as overload of the power grid. When failures occur, they often persist for several hours or days until downed lines can be restored. As a result, communications systems, to be highly resilient, must have immediate and long-term backup sources. And finally, the power systems themselves must include redundant components to protect against failures as well as include components that protect the power systems from upstream power system deficiencies. Sites each have different levels of criticality that would affect the operations of the network. Each site must be addressed individually based on the importance in regards to the network”
You will note I highlighted the “several hours or days” section. Many of today’s natural and man-made incidents last weeks instead of days. Further, something we did not even conceive of was that power companies would turn off power to millions of homes and businesses, and in some cases, these man-made power outages can last a week or more. Below is a chart of requirements from the report. It is important to understand that SHALL means MUST and SHOULD is a suggestion. This terminology is used throughout the entire report.
“259. AC power systems SHALL be designed installed and maintained adhering to the current NFPA 70/NEC46 codes and/or local jurisdictional codes utilizing the most stringent.
260. Future expansion of the site SHALL be considered during the electrical systems design.
261. Operating loads SHALL not be more than 80% of the electrical systems capacity.
262. The current edition of NFPA 70- Article 220 and Article 310-15 or the local jurisdiction’s code, whichever is more stringent SHALL be considered for circuit / feeder design and conductor selections
263. At all sites, there is either or both a main service disconnect and a fused disconnect. A main service disconnect may be located at a meter location away from the building. A main disconnect located within the shelter, equipment room, or area may be fed by a feeder circuit originating at a main service disconnect located in an electrical room in a different location in the building or even a separate building. Typically, the neutral and ground conductors are bonded in the main service disconnect. When the main service disconnect is located remotely from the equipment room or area, a separately derived system SHOULD be installed in the equipment room. (See NFPA 70, Article 250.30 and 250.32 for additional information.) One of the reasons for the separately derived system is to reestablish the neutral/ground bond, thereby improving the effectiveness of normal mode suppression.
264. Circuit breakers SHALL be sized to protect the conductor attached to them and not the load (Current edition of NFPA 70, Article 240.4)
265. A panel schedule SHALL be filled out. (Current edition of NFPA 70, Article 408.4)
266. All branch conductors SHALL be copper to reduce corrosion and impedance due to dissimilar materials.
267. Branch conductors SHALL have an allowable ampacity equal or greater to the non-continuous load plus 125% of the continuous load. (Current edition of NFPA 70, Article 210.19(A)(1)
268. A ground conductor of the same size as the current conductor SHALL be installed in all branch circuits.
269. Extension cords SHALL not be used to power permanent equipment.
270. All interior surface mounted building wiring SHALL be in rigid electrical metallic tubing (EMT) or raceways. (Current edition of NFPA 70, Article 358)
271. Conduit SHALL not be used as the AC equipment ground (ACEG) conductor.
272. A fused disconnect SHALL always be installed before all other panels and equipment, including a generator transfer switch.
273. The following are required thresholds when testing AC power quality in most single-phase and three-phase configurations (IEEE STD 1100-1999). The actual thresholds used SHALL be based on the installation requirements of the connected equipment, as the connected equipment may have more stringent requirements. See IEEE STD 1159-R2001 for additional information. Phase Voltage Testing Thresholds
• Frequency Deviation SHALL not exceed ± 0.5 Hz
• High Frequency Noise SHALL not exceed approximately 1% of the phase neutral voltage
• Voltage Sags SHALL be less than –10% of nominal supply voltage (108 V for a 120 VAC circuit)
• Voltage Swells SHALL not be more than +5% of nominal supply voltage (126 V for a 120 VAC circuit)
• Transients SHALL not exceed approximately 100 V over the nominal phase-neutral voltage’
• Distortion SHALL not be more than 5% Total Harmonic Distortion (THD) – the voltage distortion level at which loads may be affected Neutral-ground Voltage Testing Thresholds
• High Frequency Noise SHALL not exceed 2-3 peak volts
• Voltage Swells SHALL not exceed 1% to 2.5% of nominal phase-neutral voltage
274. Diverse and redundant power feeds delivered from a minimum of two different power systems from the utility SHOULD be considered for locations with extreme criticality affecting the operation of the network to ensure 99.999% availability of the network and applications”
The first recommendation in 261 above states that electrical service should not be more than 80-percent of the electrical system’s capacity. This is important because often as more and more equipment is added to a site the batteries and generators are not upgraded to match the new power demands. Several notable examples of this include a recent event on the west coast where the batteries and generator functioned correctly but did not provide the capacity needed to run the entire site and the air conditioning. In this case, the fans kicked on and the site was almost destroyed when it filled with dust and ashes from the fire.
In Houston during Hurricane Harvey, the generators worked and some could be refueled, but because they had to run for long periods of time, the oil that cools the motors was depleted and the generators failed due to the lack of oil. The final recommendation, 274, states power redundancy SHOULD be considered for locations susceptible to extremely critical conditions that may affect the operation of the network to ensure 99.999-percent availability of the network and applications. Yet during an outage due to a power company cutting power to an area during high fire alerts, dual AC power feeds will not provide the expected redundancy because we are now faced with this new reality of power companies mandating power outages covering large areas.
The report goes on to address sites without access to standard AC that rely on solar, wind, batteries, and backup generators. Under the section covering long-term backup power sources, the report provides the following recommendations:
“There are a number of long term fixed and mobile backup power sources. This section addresses these units and fuel types available.
277. Each site SHALL have a backup power generation with a power supply duration sized to power the site until it can be refueled to maintain 99.999% availability.
278. The fuel source for the generator SHALL be chosen to provide reliable generator operation given the site’s climate and other environmental factors.
279. Fixed generators with onsite fuel storage SHALL have an adequate sized storage to allow for the unit to operate at full load for the longest expected runtime given distance from supplies and the potential for transportation disruption during a disaster or power outage. System(s) SHALL have adequate capacity to carry ALL loads at full capacity when sizing the long-term power source equipment.
281. System(s) SHOULD have adequate capacity to carry ALL loads at full capacity plus 30% expansion factor when sizing the long-term power source equipment.
282. When sizing long-term power source equipment, derating per manufacturer’s specifications SHALL be calculated for altitude, installation location, voltage/phase configuration, and fuel and load type.
283. Based on the environmental characteristics of a site, generators and fuel storage SHALL be located in an area protecting it from flooding and should address other physical hazards: blowing derbies [debris], falling ice, and frequent extreme weather.
284. In areas prone to seismic activity fuel lines and connections SHALL meet local seismic codes for the fuel type utilized.
285. Generators SHALL be equipped with an engine high temperature and low oil pressure alarm/shutdown.
286. The capability to view oil pressure and engine temperature SHALL be installed on generator.
287. Voltage, amperage, and frequency meter(s) SHALL be installed either at the generator, transfer switch or both.
288. Any additional alarms or indicators SHOULD be considered to provide early detection of impending issues.
289. Engine, stator, control panel, and battery heaters SHOULD also be considered on the location of the unit.”
The recommendations in 285 and 286 are important points in this new normal of long power outages initiated by utility companies and/or caused by major incidents. Today, batteries and generators are required for Land Mobile Radio (LMR) sites and those used by FirstNet and classified as primary sites. Historically, secondary sites for cellular often have batteries but not generators, although many have a place to plug in a generator.
In at least one instance I know of, a town has permitted attachment of small cells to power poles but residents did not like the battery boxes mounted on the ground. This entire system is without backup power. As a result, people in the town who use this carrier will not have any cell-phone coverage during power outages. (This is not a FirstNet (Built with AT&T) system.)
In California, this situation has led Cal Fire to issue a call for more generators at more sites than ever before. Some major communications sites already have generators but the facility has added equipment over time without regard to the limitations of the existing generator, and most sites I am familiar with do not have oil coolers installed on the generators. In today’s world, many more sites will fail during “planned” power outages and in areas where a storm or earthquake has disrupted AC power and it cannot be restored for an extended period of time.
In 2014 when this report was published, the current state of affairs was not even imagined. Today, wildfire season is 365 days a year, hurricanes are stronger and do more damage, and normal weather patterns are no longer normal. It is imperative that we invest in keeping the communications infrastructure up and running to the greatest extent possible. What is happening today in California should be a wake-up call for all site managers to re-examine their existing power backup capacity and capabilities, and to invest in upgrades.
I had intended to write about several new devices I saw at IACP but the piece about backup power ran long, so here I will mention one more that caught my attention: the L3Harris XL-200M multiband P25 mobile radio that also supports analog FM. These are to mobiles what multiband LMR radios are to handhelds. This radio is packaged in a compact chassis with a removable control head for remote mounting. The connection between the control head and the radio is by standard Ethernet cable. Anyone who has installed an in-vehicle remote radio knows what a pain it is to run massive cables from the dashboard to the trunk or rear of a vehicle.
The radio offers VHF, UHF, 700/800 and 900 MHz and is field-upgradable, meaning LTE broadband can be added at a future date. The XL-185M is a single-band baby-brother version of the 200 that supports analog FM, P25 conventional and P25 trunked and it is available for vehicle, motorcycle, or desktop use. This radio will make a great command-vehicle radio or a radio for vehicles that need multiple bands and future built-in FirstNet. Last week I mentioned I was impressed with XL-200 multiband and FirstNet (LTE) handheld capabilities and now there is a mobile to match. It would be nice to be able to remove my three radios (VHF, UHF, 700/800) and replace them with this one unit. Nicely done, L3Harris!
The Phoenix Fire Department is on a citywide 800-trunked system but it also has a slew of VHF simplex channels for off-network communications. Law enforcement is on the same shared 800 system, but the the Department of Public Safety (DPS, highway patrol) currently uses UHF spectrum. All these agencies are either using or sampling FirstNet and at a recent meeting, a senior DPS staffer stated that in the north end of the state they have poor LMR coverage but FirstNet works great. FirstNet is the common network that will enable all these agencies to work together and to communicate with any other first responders who are also on FirstNet IF their push-to-talk choice is the same as what is being used in Phoenix. We still need to solve that interoperability issue.
I hope all of you on daylight savings time remembered to turned your clocks back last weekend. Phoenix does not switch to DST, which makes things interesting, especially when setting up conference calls. Now, with DST expired elsewhere, we are an hour later than California and on the same time as Denver. In the summer, we are on the same time as California and an hour behind Denver. I am hopeful that at some point we will settle on either DST all year or no DST at all. For now, I am back to juggling time zones.
Until next week…
Andrew M. Seybold
©2019, Andrew Seybold, Inc.