Part Four of a Multi-Part Series
This segment of my ongoing series about building the Nationwide Public Safety Broadband Network (NPSBN) will focus on the elements needed for the network itself from the eNodeBs (cell sites) through backhaul, to the Evolved Packet Core (EPC), which is the heart of the network. Without all of these elements properly deployed and operational, the network will not function and devices that are supposed to connect to it will not be able to do so.
This article is based on the current state of LTE as it is being deployed within commercial networks around the world. In the future there may, in fact, be some enhancements that might address some of the issues I am raising, but they are still being worked on in standards bodies such as 3GPP, or they are ideas and thoughts about how to remedy some of the issues. FirstNet must move forward with this network and we must be realistic about what is available today verses what may or may not become available in the future. Therefore, my comments are based on LTE as it is today and what has already been accepted and designed by the standards bodies—not what some may see on the horizon.
First, it is imperative that both the Public Safety community and those involved in commercial LTE deployments understand the key differences between Land Mobile Radio (LMR) as used by Public Safety and LTE broadband networks. The most important differences include the following:
- LMR networks and devices are designed to be able to function in several fallback modes. The networks can be degraded and still operate, the devices can change modes of operation, and when there is no network available, either due to network failure or being out of range of the network, the LMR devices can be used for one-to-many simplex or peer-to-peer off-network communications.
- The brains of an LTE network reside within the network and, to some extent, at the edges of the network and in the devices themselves. The devices are 100% reliant on the network being up and operating and if the network fails or users are out of range, the LTE devices they are carrying will not be able to communicate; not even to another device a short distance away. Therefore, the LTE network must be built with this premise in mind: If there is a failure in the network, the devices will no longer be able to communicate.
- Further, if an individual LTE cell site loses connectivity with the network, users within that cell site’s coverage won’t be able to communicate with anyone.
- If the connectivity to the LTE EPC is disrupted, e.g., if there is a power failure or a cell site is damaged, the area covered by that cell site or group of sites will not have network access. Once again, the field devices will not function.
- Using LTE deployables to fill in these gaps during times of failure or network overload is an option. However, they must be transported to the area of the incident and they must have access to the network by some means—fiber, microwave, or satellite. Since time is a critical element during major incidents or disasters, relying on deployables to solve connectivity problems may not be realistic.
- LMR systems make use of high-powered base stations, mobile units, and portable radios. LTE systems use moderate-powered base stations (eNodeBs) and very-low-powered devices (usually less than ¼ of a watt). LMR devices have some form of external antenna while today’s LTE devices have antennas embedded into them. The difference in RF transmission and reception capabilities is significant.
- LMR coverage area on a per-tower or site basis is much greater than that of an LTE cell site. Further, many LMR systems include additional receive-only sites to enhance the talkback range of the LMR radios.
- Many LMR systems have supervisory override so control can be taken of the voice channel if needed. There are various Quality of Service and priority levels with LTE but it does not appear that priority access is always available depending on the circumstances.
- LMR voice systems provide voice capabilities throughout the coverage area of the system. LTE networks and devices have different data rates and capacity characteristic depending on how far devices are from the center of the cell site. The further the LTE device is from the cell center, the less data capacity and speed there is available to the device.
This is a partial list of the differences between LMR and LTE systems, but for our purposes it is sufficient to point these out and then to address what LTE can do as well as what it cannot. While these differences are significant, the NPSBN offers capabilities that are either not available to first responders today or that are only partially available by using existing commercial networks. Note that commercial networks do not offer any form of priority to the Public Safety community and they tend to become overloaded and unusable for first responders during major events such as the Boston Bombings, Hurricane Sandy, major fires, accidents, and incidents that cover large or small areas where demand for commercial services is heavy.
Public Safety needs this new network, but it must be as robust and reliable as we can make it. The more I work with LTE experts, the more I am convinced that it will be a very long time, if ever, before there are enough fallback and other capabilities built into this system to assure that it is as robust and reliable as today’s LMR networks. Recently, the National Public Safety Telecommunications Council (NPSTC) published a document about voice over LTE networks. This document is worth reading and passing along to elected officials. So my view of this new network is that it will serve Public Safety well and it will become more robust and failure-resistant over time, but it will be years before I would ever recommend decommissioning existing LMR systems.
LTE is an all Internet Protocol (IP)-based network. From the user device to the core of the network and beyond, the data and video sent over it is encapsulated in packets. The back-end of the system (the EPC) is made up of a number of computers, switches, and routers. Perhaps one of the best ways to understand how it functions is to track a request for services through the network:
A user in the field or dispatch center initiates a request for a data or video session across the network. This request can be to send information to another user, or hopefully in the near future, to a group of users, or to request access to a database or even to the Internet. The device sends out a request for network connection. The closest eNodeB receives the request and routes it to specific computers within the EPC. At that point, the user is verified, the request is processed, and it is sent on to its destination.
This is an over-simplification of what transpires in a fraction of a second but it does describe the actions taken by the user and then how the network handles and processes the request. The computers within the EPC can also verify the network priority level of the user, the data rate capability of the device (determined by its signal strength into the network), and even the type of device and its display capabilities. All of this is important information used in sending the requested information back to the device in the correct format.
For the duration of the session, if the device is moving from one cell to another the network tracks these movements and hands off the session to another cell site, keeping the session alive. During the session the user may move from a prime coverage area with access to the full capabilities of the network to an area that is further away from the center of the cell. The network will adjust the unit’s transmit power and change the data rate to compensate for a weaker signal to and from the device. While all of this is happening, the device and the network are talking to each other to ensure that the session is handled properly, that any dropped packets are quickly retransmitted, and that the session is completed.
Issues Faced by FirstNet
- How many eNodeBs (cell sites) will be needed to cover the United States?
- How many will be needed within a specific city, county, or region to be able to cover the geography and handle the load on the network? LTE capacity is a function of how many users there are in a single cell sector or site. Where demand will be heaviest, more cells will need to be built closer together.
- How many EPC or cores will be built and placed where within the network? For the sake of redundancy, more than one will be needed, but what is the correct number? Further, how far will the nearest core be from a major metro area?
- If the nearest core to Los Angeles, New York, or other major metro area is a thousand or more miles away, and even if the backhaul is redundant fiber connections, there is the possibility of a communications failure between the core and the city. If this happens, and Los Angeles, for example, loses connectivity with the core, we would have on the order of 180 useless cell sites and no service in that area until connectivity was reestablished. I believe that in addition to the nationwide core(s), we may need to permit major metro and regional areas to install their own cores, closer to their own area of operation. More than a year ago there was a sub-committee of waiver recipients that analyzed this issue and concluded that multiple cores could be employed. The recommendation was that in addition to the nationwide core(s), if a given area wanted to have its own core it would have to fund its operation and its integration into the network. I am in full agreement on both points.
- What sites will be selected? Will they all be commercial sites or will FirstNet also employ some key Public Safety sites as well?
- My belief is that wherever possible, FirstNet should use Public Safety sites first and then use commercial sites to fill in coverage and capacity as needed. If the system is designed to use the usually more hardened Public Safety sites and there are failures at the commercial sites, there would at least be a backbone of key sites over which users could communicate. Many commercial sites already have shared towers and facilities with all network operators, so if there is a failure and a site(s) is not working, none of the commercial operators would have service. If the NPSBN is also located on these sites, Public Safety will lose service to the same geographic area. I know FirstNet does not have a lot of money for this network but it can make wise choices, for example, in my area, the same hilltop or mountaintop hosts a county installation and a commercial cell site installation. In cases like this, I believe that the location of the eNodeBs should be on the Public Safety site—not on the commercial site.
- Commercial networks are designed with at least three levels of sites depending upon the population they serve and the area they cover. Top-level sites are full-blown sites with back-up batteries and generators. Second and third tier sites are used to fill in for capacity and coverage. Many times these sites do not have the same power back-up capabilities since the main sites can, in times of failure, provide some basic coverage for sites that are out of commission. By balancing Public Safety and commercial sites, FirstNet should be able provide even better coverage during times of disaster.
Next up is perhaps one of the most important aspects of this network. I am assuming here that in order to afford this network, FirstNet will rely heavily on commercial network partners and piggyback on their facilities, use their backhaul where needed, and in other ways maximize the commercial network operators’ assets. I see no way around this, and in return, the network operators will be able to use NPSBN excess capacity to help manage their own networks and demand for services. However, as I wrote in a recent article for Urgent Communications, I have some serious concerns about network sharing. This started as a discussion among a group of us about sharing the network with commercial users and the requirement that some form of absolute and pre-emptive priority be given to the Public Safety community.
Unlike commercial networks where capacity can be predicted (e.g., Times Square before and after the theater), incidents that will require first responders and where they will occur cannot be predicted. A quiet neighborhood can suddenly erupt in a major incident as easily as in a known trouble area of a city. A major accident can occur along any stretch of highway anywhere in the nation, and hurricanes, tornadoes, and wild fires can hit almost anywhere. Therefore, when planning excess capacity availability, FirstNet must ensure that there really is a way for Public Safety to gain full and immediate access to all of the network capacity at any time in any area of their operation.
In the Urgent article, I started with the premise that the cellular networks in Boston after the bombings had been shut down, which was an early assumption that was not correct. However, what did happen was that the networks were so overloaded with traffic that it appeared to many that they were not operational and thus must have been shut down. The result was the same. The public, reporters, and Public Safety did not have assured access to any of the commercial networks.
The next logical point was to wonder whether commercial network customers who would have sharing rights with the Public Safety network would have been moved to the NPSBN, thus adding to its congestion. Moreover, if the commercial networks had been shut down to foil a bomber’s attempt to set off a bomb using cellular technology, would the NPSBN have been shut down too, just when it was needed the most, because it had commercial users on it?
Based on these discussions, I went back to a committee of LTE experts that is run by a very smart consultant. This is a unique committee made up of some of the best LTE engineering minds in the nation. Members of this group work for different, competing companies. They have come together for the benefit of the Public Safety community and have shared their expertise with many organizations including the Public Safety Spectrum Trust, the waiver recipients, and the APCO broadband committee. They give freely of their time and as a group they are involved in every phase of LTE from working with the standards bodies to working with their own companies and thus with the network operators. They have studied and learned as much as they can about LTE, and in some cases have been involved with LTE before it became a standard. So after my discussions, I decided to ask this group to comment on two assumptions that need to be answered in order to ensure that Public Safety will, in fact, have complete and total access to the NPSBN whenever and wherever it is needed. The two assumptions I started with were:
Assumption #1: If the signaling channel is overloaded, a User (UE) with maximum priority and pre-emptive rights may not be able to access the network.
The short answer: This is essentially a true statement, especially in a network that is shared with commercial users.
Assumption #2: LTE provides a way around this problem (as stated in Assumption #1) that can be implemented to ensure full priority access when needed.
The Short Answer: Mitigation tools exist in the 3GPP standards, but due to a wide range of potential scenarios and causes, to characterize this as solved would be an over-simplification.
The team’s full report will be made available in the near future but the bottom line is this:
Using today’s LTE, if the signaling channel for an eNodeB or group of eNodeBs receives too many requests for connection, the signaling channel (RACH) will, in fact, become overloaded and the eNodeB will not even know that a user with any level of priority is trying to access the network. Further, if a number of eNodeBs all have their signaling channel overloaded, it is also possible that the part of the core network that processes priority service requests will be overloaded and will not be able to process the requests.
Are there solutions on the horizon? The group concluded this was perhaps the case, but as mentioned at the beginning of this segment, we need to deal with what is here and available today, having some faith that enhancements will be developed over time, passed by the standards bodies, tested, and proven to mitigate some of the issues. In the meantime, it will be important for FirstNet to find ways to load-balance the network between Public Safety and secondary users. This information should be used to balance the amount of secondary traffic on the network with the need for Public Safety’s absolute need to access the network at any given time.
How to Proceed
What all of the above means is that the number of secondary users permitted on the network will need to be carefully controlled. While this may, to some degree, limit FirstNet’s ability to convince commercial network partners to contribute fully to the construction of the network, we must be careful to keep its primary purpose in the forefront. This network is to be a nationwide broadband network for the Public Safety community. I am hopeful that there will be a way to turn off access to the NPSBN for secondary users when the capacity is needed for Public Safety. The issue, I believe, will be more political than technical and some form of agreement will have to be put in place so that full access to all of the network capacity can be quickly and easily assigned to the Public Safety community on an as-needed basis.
During an incident such as the Boston Bombings, the number of Public Safety users will increase rapidly. It will start with local law enforcement, fire, and EMS responders, but soon federal agencies will arrive, other first responders from out of the area will descend on the scene, and the number of Public Safety users on the NPSBN in and around the incident area will grow quickly. By the same token, if the commercial networks are jammed, and they have access to the NPSBN for overflow traffic, some way of curtailing that access will be required. Further, in major metro areas I am not at all sure that anyone can predict how much of the spectrum will be available for sharing at any given point in time. Perhaps it makes more sense to enter into partnerships in suburban and rural areas and use the federal funds to build out the major metro areas. Once the network is operational and populated, we can see what types of devices and applications come to be, and we will have a track record for Public Safety usage in large metro areas. Then the issue of sharing spectrum in these areas can be revisited.
Considering the number of first responders across the country—which is, indeed, a small number of network users compared to commercial network users—it is, I am sure, easy to believe that there should be plenty of available spectrum for secondary users most of the time, and this may, in fact, be true. However, the issue for Public Safety is that when there is an incident, even a bank robbery, a hostage situation, or multiple-car accident, the number of first responders within a small geographic area will increase dramatically. Unless there is a way to ensure full and complete access to the first responder community, secondary users could take up needed bandwidth. How many incidents occur in a given metro area on a daily basis? The number is known within the local jurisdictions but, of course, their locations are not known in advance. The safe course of action then, is to limit the number of secondary users permitted to share the spectrum within the major metro areas and re-evaluate spectrum usage after Public Safety has some experience with the network and usage patterns.
The $7 Billion Public Safety is supposed to receive from the incentive spectrum auctions is certainly not enough to build out the network, thus partners will be needed. However, FirstNet must ensure that where secondary usage is permitted, first responders come first 100% of the time. Speaking about funding, we need to think about where the money is coming from because there are other actions by the federal government that could jeopardize that funding. The $7 Billion allocated in the ACT is based on the proceeds from the Incentive auctions for TV spectrum. Unlike many laws, there are no provisions for a secondary source of funding. The auctions are sure to raise the money when spectrum is in demand from commercial network operators large and small, right? Well, there are a few issues that could come into play and over which the Public Safety community has little or no control.
When the spectrum is auctioned, the TV stations that agreed in the reverse auction to vacate the spectrum will receive the first portion of the revenue. The auctions are supposed to generate between $25 and $28 Billion but there is a danger for Public Safety that they will not generate this level of revenue. Recently, the Department of Justice recommended to the FCC that both AT&T and Verizon be precluded from bidding on this spectrum. The reason stated was that these two networks already own the bulk of the spectrum below 1 GHz. If the FCC agrees with the DOJ, fewer companies may show up to bid, and they may not be able to pay higher prices for the spectrum. The result could be an auction that does not raise enough money to pay the TV stations and fund the $7 Billion that is to be diverted to Public Safety, let alone feed the government’s coffers. A recent economic analysis of the impact these bidding restrictions will have in funding confirms that, in fact, there could be 40% less money raised at auction, which will certainly have an impact on the funding of FirstNet. Therefore, the issue of funding won’t be settled until the Incentive auction dollars are tallied. If the auction does not bring in sufficient funding for the network, Public Safety will, once again, have to walk the halls of Congress looking for new funding sources. With today’s focus on federal spending, that could be a long, hard-fought battle.
FirstNet must plan as though this money is forthcoming, and the ACT did advance FirstNet the first $2 Billion from the U.S. Treasury as an interest-free loan, so there is money, if used wisely, to leave the starting line and move part way down the field. The auction results and working with partners are both crucial to the success of the NPSBN. While there are many technical challenges ahead for the network, there are even more financial issues to work out.
In this installment I intended to cover the cyber security requirements for this network in detail. However, this is such an important topic that I will offer a brief discussion of the requirements here and follow up with a much more detailed discussion in the next part of this series. The bottom line is that this network must be the most secure wireless network ever built. Security must be planned into every aspect of the network from the beginning and the plans for including security must be vetted not only by the cyber security experts within several federal organizations, many having three initials, but also by those who provide security products and software in the public sector.
There are many aspects to a secure network, and the Internet provides a way in. As we are all aware, the Internet is the conduit for all manner of attacks on networks: denial of service, breaking into databases, stealing credit card and banking information, breaking into websites, and much more. Many public companies have been the target of such attacks, and many government sites and databases have been compromised. There are those, both inside the United States and in other countries, who hack anything and everything they can reach via the Internet. It is a game of cat and mouse. The bad guys hack a site or enter a database, we find out how and plug the hole, and they find another way in. The bottom line is that security of the network will require ongoing attention and people who monitor the network for attempts and fend them off.
The NPSBN will be capable of accessing criminal records and other information that is protected by federal and state laws. This information must be protected no matter what and we will need several different layers of security for this network. The first will be securing access to the basic LTE network, which will be made more difficult because it will be shared with non-Public Safety users. Then we must assure that the databases being accessed are secure, that the applications being used are secure, and that the devices in the field are also secure. If a police car or other vehicle with a laptop in it that is connected to the network is stolen, how will the laptop and thus the network be secured?
The devices will have to be capable of being wiped remotely, which is common practice today, and there should be a way to instantly block a device from the network if it is missing or stolen. There are some who will try to hack this network either for fun or for serious disruption, and this will be made easier with so many commercial devices capable of using the NPSBN in the hands of secondary users.
I am sure that the FirstNet technical team is aware of all of this and more and is working with all of the proper agencies and commercial experts to stay one step ahead of the bad guys or teenagers who want to hack the network just to see if they can. One of the issues with adding multiple layers of security on the network is that there will be an overhead associated with each level of security that is added. How will all of this security be implemented in such a way as not to slow down law enforcement’s access to data when needed?
A careful analysis will be needed to determine how much of the available network capacity will be eaten up with these security measures. This is an area that deserves special attention from all of the experts in the field. In my opinion, the number of connection points to the Internet must be limited and the network must be able to be disconnected from the Internet within seconds to stop any attack coming via the Internet. This too will require careful planning. I am sure that most jurisdictions assume that their portion of the network will be connected to their existing Internet connection, but this would create problems for the network. Therefore, FirstNet must enforce some types of rules about who can connect to the Internet and how all of these connections can be dropped if and when necessary. It is better to have a functioning network that has no outside connectivity for a while than to leave the network open for additional attacks. There are lessons to be learned from the power companies and others that have multiple connections to the Internet and have no means of disconnecting their network during an attack.
The NPSBN network is wireless only for the “last mile.” The rest of the network will be made up of fiber and microwave connections. Each cell site will need to be connected with high-speed fiber and/or microwave circuits to the network. The various pieces and parts of the network must be connected to each other, and there must be as much redundancy as possible. I don’t believe that it is either practical or affordable to require that each cell site have two separate fiber or microwave connections to the network. However, some primary sites could certainly be designed in such a way. Further, since the network will be nationwide, the transport used across the country will vary from location to location. Looking at commercial networks we see a mixture of network-owned connections and those leased from a third party. In my county, AT&T is leasing fiber from Verizon because AT&T does not have its own fiber. The city has its own fiber so it might be a good resource for additional connectivity, but we are prone to earthquakes so it might make sense to make use of fiber, microwave, or both as the county does with its LMR systems.
When commercial operators order fiber for their cell sites they rate the capacity of the fiber needed based on a formula for site activity. If the site is capable of, for example, 34 Mbps per sector down to the devices, that should mean they need to install fiber capable of 3 times 34, or 102 megabits. Most sites I have seen don’t have fiber with the full potential of the site but they do have 60-80% of the potential maximum. This works for commercial sites but in major metro areas, at least, this may not work for the Public Safety network where an entire site’s capacity might be needed during a single incident. Not having enough backhaul introduces a potential choke point that could limit the potential capacity. Further, since it is my belief, shared by others, that the NPSBN will demonstrate a 180-degree difference in data handling when compared with commercial networks—there will be more data sent from the devices in the field than to them—capacity planning becomes even more critical.
If there are secondary users on the network, how will the backhaul break out the Public Safety and non-Public Safety data, or will it all simply ride on the same fiber? When the data is being moved to the EPC, will it be in Public Safety-only fiber or will that fiber be shared with commercial customers? Will the network operators that partner with FirstNet provide their own fiber connections when the NPSBN eNodeB is located on one of their cell sites? If so, where will it be split off and routed to the NPSBN EPC? Planning this type of network is a daunting task. Planning the network to be shared by Public Safety and secondary, non-Public Safety users adds several more levels of complexity. This is one reason that it is vital that Public Safety have visibility into all aspects of the network design and that those on the commercial side that are working on the network design seek out Public Safety input and vet their thinking in terms of the needs of the Public Safety community.
The NPSBN is the most ambitious nationwide network ever undertaken in the United States and it is different from the public commercial networks in many aspects. First and foremost, it must provide the capabilities needed by the Public Safety community, and it must be more reliable. It presents many technical and financial issues that need to be addressed, and it will be a work-in-progress for a number of years to come. For those who expect this network to be ready quickly, I will say that it cannot be rushed. There are, as you have seen, many moving parts, each of which is dependent on a number of others. Some who come from the world of IT believe that since LTE is a technology based on the Internet Protocol it is easier to build and manage. However, commercial network operators will all tell you the same thing: It is an extremely complex architecture. The spectrum at each site is reused in its entirety at all of the other sites. In some ways, those with experience in simulcast LMR systems will have some appreciation of what needs to happen in the planning stages. In a simulcast network, the areas of overlapping coverage must be managed and tweaked until they are right; if not, the result will be garbled voice.
LTE systems must be designed to minimize interference between cell sites. New sites cannot simply be inserted without careful planning and a lot of advanced work because the addition of a new site could cause interference to existing sites. Moving sites and bringing in portable and temporary sites needs to be managed, and all of the pieces have to fit together. It is also important to realize that the NPSBN and other LTE-based networks are a combination of smarts: in the devices, in the cell sites, and at the core of the network. All of these pieces must function together in order for the system to operate correctly and there are more points of failure for a system like this than in most typical LMR systems. Further, if there are failures, the devices simply won’t work. There is no fallback mode or work-around. If a site or multiple sites are down or overloaded, the result is the same—the system will not function in that area.
The Evolved Packet Core is the heart of the system. It must be redundant, and hardened, there must be more than one, and they all must share information back and forth on a real-time basis. If some major jurisdictions are permitted to house their own cores, and I believe they should be permitted to do so, these too must be integrated into the overall network. Backhaul must be robust and redundant wherever possible and emergency power must be provided to every aspect of the network. Security is a critical element of this network and as mentioned, having secondary users on the same network as Public Safety will introduce new challenges.
Still this can be done. The NPSBN can be successful and it can bring to Public Safety capabilities it never before had. We cannot be in a hurry; we must get it right the first time. One of my favorite expressions is, “Why is there always time to do something over but never time to do it right the first time?” I believe that this applies here to a large degree and everyone involved—FirstNet, PSCR, NTIA, the FCC, Public Safety, commercial network operators, and everyone else—must work together to make it happen and do it right the first time.
Andrew M. Seybold