Industrial Internet of Things (IIoT) applications, featured with data-centric innovations, are leveraging the observability, control, and analytics, as well as the safety of industrial operations. In IIoT deployments, wireless links are increasingly used in improving the operational connectivity for industrial data services, such as collecting massive process data, communicating with industrial robots, and tracking machines/parts/products on the factory floor and beyond. The wireless system design for IIoT applications is inherently a joint effort between operational technology (OT) engineers, information technology (IT) system architects, and wireless network planners. In this paper, we propose a new reference framework for the wireless system design in IIoT use cases. The framework presents a generic design process and identifies the key questions and tools of individual procedures. Specifically, we extract impact factors from distinct domains including industrial operations and environments, data service dynamics, and the IT infrastructure. We then map these factors into function clusters and discuss their respective impact on performance metrics and resource utilization strategies. Finally, discussions take place in four exemplary IIoT applications where we use the framework to identify the wireless network issues and deployment features in the continuous process monitoring, discrete system control, mobile applications, and spectrum harmonization, respectively. The goals of this work are twofold: 1) to assist OT engineers to better recognize wireless communication demands and challenges in their plants, 2) to help industrial IT specialists to come up with operative and efficient end-to-end wireless solutions to meet demanding needs in factory environments.
Index Terms—: Industrial Internet of Things (IIoT), industrial wireless networks, design reference framework
INDUSTRIAL practices heavily employ operational technology (OT) domain data in their production activities, such as the asset performance monitoring, work flow optimization, and plant safety management [91]. The volume and categories of industrial data exhibit a remarkable growth. For example, a modern computer numerical control (CNC) machine already produces data in the order of 30 Terabytes (TB) per year [1], and it is estimated that by 2020 a smart connected factory will generate 1 Petabyte (PB) of data per day [2]. To feed the huge data demand, the Internet of Things (IoT) techniques are introduced into vertical industrial domains. The manufacturing sector is anticipated to occupy 33% of the total IoT applications in 2025 [89]. As the industrial variant of IoT, the Industrial Internet of Things (IIoT) provides customized architectures and standardized interfaces in data acquisition, transmission, and analytics for industrial applications [3]. IIoT is credited to boosting the visibility of production processes and the transparency of control decisions. It is also actively playing in the ongoing innovations of cyber-physical systems (CPS), known under labels, such as Smart Manufacturing [4] and Industry 4.0 [5], [117]. 1
IIoT, as an open and scalable information technology (IT) platform, enables the exchange of machine-typed data between industrial devices and the on-premises/cloud computing facility in the local and wider-areas industrial operations [95], [118]. Wireless communication networks are playing an increasingly important role in such a machine-to-machine (M2M) communication paradigm. Compared to their wired peers, wireless networks have feature advantages, such as connection flexibility and cost efficiency, which facilitate IIoT operations, e.g., connecting massive industrial “things” in the field, conveying the system state within open and closed-loop control processes, and serving objects in motion such as mobile robots and parts/goods in logistics flows.
In the remainder of this section, we will first walk through the state-of-the-art wireless techniques for IIoT applications to briefly review current research and implementation progresses. Next, in Section I-B, we will discuss challenges and opportunities that motivate this work. Finally, we will identify this paper’s contributions in Section I-C and introduce how the following sections are organized.
Wireless use cases in current IIoT implementations can be generally classified by (1) the associated applications in industrial sectors and/or (2) the quality-of-service (QoS) levels in wireless links. For example, wireless use cases have been identified in a variety of applications including asset performance monitoring [86], [113], [118], real-time process control [16], [104], [105], inventory and logistics management [88], [112], and safety [28], [69]. The numerology for wireless networks specifies qualitative and quantitative measures on the desired performance, e.g., the data rate, transmission range, latency, and reliability. For example, we can differentiated the delay (in-)tolerant and loss (in-)tolerant services in industrial applications and design different wireless networks to adapt to their respective tolerance to the transport delay and data loss [90]. Fig. 1 illustrates a few of exemplary wireless use cases with the identified performance requirements.
Exemplary performance requirements of industrial wireless use cases. (Wireless use cases are exemplary illustrations of possible system settings, the values of which are not mandatory thresholds in the deployments; the higher performance bounds of metrics are illustrated when applicable; the data rate per link is estimated by adding a protocol overhead which is as much as 50% of the payload; and the data rate per serving area assumes that the frequency reuse factor is 1 and all links are saturate, i.e., at the peak value. Specifications of use case a, b, and c are referenced from the 3GPP TS 22.261 [8, Sec. 7.2.7]; use case d refers to the specifications in [47]; use case e showcases the Tennessee Eastman (TE) chemical process control problem [73] with wireless settings discussed in [75]; use case f and g are identified in the 3GPP TS 22.804 [27, Table 5.3.2.1], which are also used by 5G-ACIA in its report as typical 5G wireless use cases [29]; use case h presents an example of massive wireless sensor networks collecting uncompressed temperature, humidity, and pressure sampling data [27, Sec. 5.3.8].)
Wireless standard bodies have been involved in specifying the representative use cases in industrial environments [8], [25]–[27]. The European Telecommunications Standards Institute (ETSI) performed a series of site surveys and concluded that industrial applications mainly carry machine-typed data which refer to the data generated and consumed by machines [26]. Meanwhile, ETSI also identified a number of industrial use cases with specific radio communication requirements. Based on this report [26], the International Telecommunication Union Radiocommunication Sector (ITU-R) released the “International Mobile Telecommunication (IMT) 2020 and beyond” vision on the fifth generation (5G) cellular mobile systems. In 5G systems, ITU-R considers two major usage scenarios for M2M communications, i.e., massive machine-type communications (mMTC) and ultra-reliable and low latency communications (URLLC), respectively [120]. The former refers to the cost-efficient and robust connections to massive devices 2 without overloading the network in industrial use cases such as process sensing and control, remote machine diagnostic, and over-the-air update; the latter applies in the use cases under more stringent latency and reliability rules such as motion control applications in factory automation, safety-related updates in robotic operations, and industrial virtual/augmented reality (VR/AR) applications. Meanwhile, ITU-R also defined typical wireless industrial automation (WIA) applications and posted performance requirements for specific services [25]. Mirroring ITU-R’s 5G recommendations, the 3rd Generation Partnership Project (3GPP) identifies industrial wireless use cases in its cellular service scenarios [8]. Ever Since its first release for 5G specifications, i.e., the Release 15, 3GPP has been conducting the study of 5G Communications for Automation in Vertical domains (CAV) applications and planning the roadmap to industrial cellular deployments [27].
Industrial users are normally playing a much stronger and more active role in deciding wireless services in their plants compared to personal customers in the wireless market. The newly founded 5G Alliance for Connected Industries and Automation (5G-ACIA) has provided some inputs from industrial manufacturers in the form of white paper. In its inaugural white paper [29], 5G-ACIA released new requirements on OT-driven metrics (in process-related aspects) in addition to the network-driven performance indicators (in link-related aspects) that had been raised by wireless industries [8], [25], [27]. In the recent industrial wireless guidelines published by the National Institute of Standards and Technology (NIST), surveying procedures for identifying a factory’s radio activities and wireless needs are suggested along with the terminology used in characterizing the industrial wireless applications [32]. As robots are widely used in the plant, wireless links have been increasingly employed to connect the industrial robots to its controller or the remote supervisor [114]. The American National Standards Institute (ANSI) and the Robotic Industries Association (RIA) are collaborating on drafting a series of safety-related standards for industrial robots which include the general wireless control use cases [115] and ongoing discussions on mobile robots [116].
There are no one-size-fits-all wireless solutions for industrial use cases as the service requirements and operation environments may differ vastly from one another. Earlier industrial wireless networks were mainly developed for providing the connectivity in each single vertical manufacturing sector [99]. As a result, the solutions that function well under the specific service requirements and operating conditions they were designed for, may only yield limited value in different use cases. To replicate the wireless success in more emerging IIoT applications, wireless networks are expected to facilitate the wider and deeper digital contact with industrial systems and provide flexible interfaces and quick deployments while keeping data integrity. Current discussions exhibit a few distinct features in the design principles and operations.
The first keyword is machine-typed data. The coupling between IIoT and industrial systems is realized by the definition, exchange, and utilization of data occurring in various industrial areas and processes with respective functions in the applications [93]. As a result, IIoT data have unique requirements on their formats and transmissions. Specifically, in the upper layers, discussions on the OT communication traffic are looking for new ways of extracting the industrial data patterns from daily operations, especially for the QoS requirements. For example, M2M traffics require the deterministic packet delivery which needs the performance guarantees in latency and reliability over wireless links. Otherwise, it would result in error-prone control decisions which raise risks such as property damages and personnel injuries. Meanwhile, the IIoT data can also be labeled with one or more case-specific features such as massive access requests, high updating frequencies, and/or small payloads. Former models were found less effective in new IIoT cases where the analytical prediction calculated by these classic approaches often deviates from the real performance [121]. For example, the channel capacity is estimated in the communication theory based on the assumption that a packet can always increase its payload without bound [56]. In the IIoT links, as data usually have small payloads, typically of only a few bytes, such a classic assumption does not hold [121].
Second, IIoT exhibits unique characteristics in the radio environment. The diversity and variety of plant layouts and onsite activities have attracted multiple measurement efforts on characterizing industrial radio environments [33]–[36], [102]. Measurements have been taken in the sites of different types and sizes, such as automotive assembly lines, steam supply plants, small-sized machine shops, and collaborative robotic work-cells. Topographic differences have been captured and associated with the parameter settings in wireless channel models. The measurements observed a noticeable deviation in the settings, such as the delay spread profile and path lost exponent, from the models in non-industrial cases. Even in the same plant, wireless channel characteristics may also vary with the location-specific topographic pattern, e.g., walls and machines [33]. Moreover, a plant can accommodate multiple wireless systems whose transmissions may interfere with one another in the same or neighboring spectrum bands. It is impractical to solely rely on radio regulations to protect the spectrum usage considering the large number of legacy and unlicensed wireless applications and the delay. Therefore, it is important to recognize the coexistence issues from the use case and develop interference management schemes in different situations [23].
The last but not least, wireless IIoT techniques adopt an evolutionary roadmap in compliance with the IT innovations [95]. IIoT is looking for coherence in a factory’s IT system where wireless networks are an integral part. Wireless engineers need to leverage their design from pure discussions on air interfaces to the full-stack wireless solutions considering the end-to-end service provision. Currently the industrial IT infrastructure is moving forward to aggregating the massive data at the centralized computing facility for improving computational efficiency [94]. Meanwhile the control intelligence is being deployed closer to the field and utilizing proximate compute resources to realize low-latency responses and situational awareness [87]. As a result, wireless networks need to be capable of negotiating with the IT infrastructure on the end-to-end performance once industrial services are deployed in the cloud. Otherwise, a service level agreement (SLA) cannot be reached [85]. New system integration techniques, such as software-defined networking (SDN) and network function virtualization (NFV), are developed to adapt the IT resources and management strategies for end-to-end service provision.
Wireless networks are still evolving to address more IIoT features in their standardization efforts [89]. Candidate wireless communications protocols for serving IIoT use cases are found to be diverse and dynamic ranging from near-field communications (NFC), e.g., the product scanning with radio-frequency identification (RFID) tags, to long range wireless transmissions, e.g., smart meters in a wide area. These standards are enablers of IIoT services by innovating key wireless techniques. Huang et al. reviewed current standardization efforts on supporting wireless IIoT transmissions [84]. Generally wireless standards can be classified by their working channels and QoS levels in the wireless services, e.g., the data rate, coverage, and energy consumption. We will briefly discuss their respective features for IIoT practices.
There are a huge selection of wireless IIoT systems using the unlicensed spectrum bands, e.g., the industrial, scientific and medical (ISM) bands at sub-1 GHz, 2.4 GHz, and 5 GHz. The representative techniques include the low rate wide-area (LRWA) networks, BlueTooth and BlueTooth Low Energy (BLE) techniques, IEEE 802.15.4 and its industrial variants, and wireless local-area networks (WLAN).
The unlicensed LRWA IoT techniques, e.g., Long Range (LoRa) and Sigfox, provide the wireless connectivity mainly in the retrieval of data from field devices [141]. The radio part utilizes the sub-1 GHz bands and transmit over long distances (up to tens of kilometers). The link traffic is very light, i.e., there are only a few bytes in the payload and a few seconds of air-time per device per day. They usually connect the low-cost environmental sensors and meters to the data collector. The field nodes work outdoors in a wide area deployment and can survive for years on batteries.
BlueTooth added the support of mesh networks in its latest release, i.e, BlueTooth v5.0, to serve device-to-device (D2D) connections [122]. It can be used in wearable devices for workers and safety-related industrial applications [124]. Meanwhile, its low power version, known as the BlueTooth Low Energy (BLE), can work at only one tenth of the power consumed by the classic BlueTooth devices and last for months or years before the battery replacement. BLE also supports industrial wireless communications [125].
The IEEE 802.15.4 standard along with its industrial variants, e.g., WirelessHART [12], ISA100.11a [13], and Wireless Network for Industrial Automation Process Automation (WIA-PA) [14], support the low data rate (up to 250 kbps) and long durability wireless transmissions that carry process automation data. To meet up with the transmissions of mission-critical data, the medium access control (MAC) layer replaces the original carrier-sense multiple access (CSMA) design by the time-division multiple access (TDMA) scheme that allows the deterministic allocation of transmission slots for periodic industrial updates and increases the time resolution as short as 10 ms.
Recently the IEEE industrial electronics society (IES) has identified the new trend of expanding wireless adoptions from slow paced continuous processes to more time-sensitive discrete automation applications such as industrial robots and motion control cases [77]. The IEEE 802.15.4 or BlueTooth radios can only support the low rate data transmissions for real-time industrial control applications [92]. Such discussions have triggered the inventions of new wireless Ethernet techniques, such as wirelessHP [105]. Among them, IEEE 802.11 is a strong candidate to match the industrial Ethernet’s high throughput performance [126].
WLAN techniques, mainly from the IEEE 802.11 standard family, have received worldwide success in office and home wireless scenarios. However, the early versions, i.e., the IEEE 802.11b/g/n/ac specifications, use the CSMA scheme which does not fully support the IIoT data features in mission-critical applications, such as determinism and small payloads. The IEEE 802.11 working group (WG) has started drafting a series of new amendments to adapt WLAN to the IIoT use cases. IEEE 802.11ah was the first of such efforts [127]. Working in the unlicensed 900 MHz ISM band, it achieves a denser deployment, i.e., up to 8000 devices in a coverage range up to 1.5 km, with lower power consumption compared to legacy IEEE 802.11 networks [127, Table 1]. As another effort for supporting mission-critical data transmissions, IEEE 802.11ax adopts the orthogonal frequency-division multiple access (OFDMA) MAC layer which guarantees the explicit transmission scheduling for the low latency threshold. Mean-while, IEEE 802.11ax can also serve in more challenging use cases, such as robotic motion control and VR/AR, thanks to its diverse modulation and coding schemes (MCS) to cope with high dynamics in wireless links [128]. The industrial variants of WLAN are further encouraging the adoption of WLAN techniques in vertical industrial domains. For example, the Wireless Network for Industrial Automation–Factory Automation (WIA-FA) standard introduced the TDMA scheme into the IEEE 802.11 radio targeting the class of factory automation applications for the high throughput and low latency performance [15].
IIoT techniques in the licensed spectrum are mainly developed in cellular systems which can be further classified into the cellular IoT techniques and ongoing 5G efforts.
Cellular IoT refers to a subset of technical specifications in the 3GPP Release 13 and defines the IoT deployments in the 2G/3G/4G bands. 3 There are three options: narrowband (NB)IoT [22], enhanced MTC (eMTC), and Extended Coverage-GSM-IoT (EC-GSM-IoT). These techniques are designed mainly for the outdoor connections that operate at the low to moderate data rate (ranging from around 100 kbps to 1 Mbps), have long standby time (for years), allow massive radio access (with typical 100 to 10000 nodes per cell), and transmit over long distances (from hundreds of meters to tens of kilometers) [129]. In the following Release 14, 3GPP further strengthened these techniques with new features including positioning, serving mission-critical IoT data, e.g., 1 ms latency in vehicle-to-everything (V2X) communications, the mobility support for service continuity, and reduced system overhead [129].
NB-IoT belongs to the LRWA techniques and was designed to compete with its unlicensed peers, i.e., LoRa and Sigfox. Compared to them, NB-IoT has its own advantages, such as the higher data rate (up to 150 kbps), additional support for downlink (DL) transmissions, and interference management in licensed cellular bands. However, cellular IoT relies on the local carrier’s network and spectrum resources to provide the service. The cellular IIoT techniques will focus on high-valued applications for mission-critical applications and wider area deployments where they would face less challenges from the peer wireless techniques, such as low-power low-cost indoor wireless techniques including WLAN, Bluetooth, etc [83].
In 5G systems, industrial data services are considered in two main usage scenarios, i.e., mMTC and URLLC, which provide connections for the long range IoT coverage and mission-critical applications, respectively [83]. The 5G air interface, known as 5G new radio (NR), has implemented new techniques in its physical (PHY) layer and MAC layer to support machine-typed communications [98]. 5G NR defines more dedicated radio resources, such as the sub-millisecond time resolution in the transmission time interval (TTI) and OFDM symbol durations. Meanwhile, it also introduces extra transmission redundancy to improve the link reliability and employs quicker allocation schemes, e.g., smaller HARQ reply messages, to better support industrial applications in the URLLC scenario [8], [101], [110].
5G networks consider the performance optimization from the entire system architecture. For example, the end-to-end service delay consists of the air-time delay over cellular links, the transport delay in the 5G infrastructure, and the service processing time. Using network slicing in NFV, 5G can preallocate the end-to-end resources for both communications and computing to guarantee the service availability and reliability [53]. Mobile edge computing (MEC) techniques were also developed to further reduce the transfers between gateway and application servers by deploying cloud compute resources close to operating sites. Spectrum bands and base stations can also be assigned with private cellular connections so that industrial data are transmitted in the reserved channels and secured with access privileges [11].
Besides innovations in each standard’s protocol stack, there are also active discussions on many general technical topics on improving the industrial wireless performance. These topics mainly address problems and challenges in three aspects: system models, radio resource management (RRM) schemes, and protocol interfaces.
System models can be created and verified through approaches including theoretical inference, empirical measurements, and simulation/emulation tests. Since IIoT is a complex system, the models of data traffic pattens and wireless environments serve as the important reference in the design work. New system models were developed [73]. Based on these models, system verification methods using co-simulation platforms [75], [136], [138], [140], hardware-in-the-loop (HIL) experiments [139], and testbeds [137] become popular in learning the industrial environment and service characteristics.
Radio resource management (RRM) schemes provide reliable services in dynamic and diverse wireless environments [18]. In the RRM topics, wireless coexistence becomes critical in the deployment site of massive industrial instruments [109]. The interference management and load balancing are the main issues in the design of coexistence mechanisms [23], [24]. Technologies using cognitive radios (CR) are also used in the industrial networks to sense the ambient radio environment and estimate the interference level [107]. Besides reliable links, wireless networks are also expected to achieve energy efficiency, especially in the cases where IIoT devices are working on batteries. Accurate energy models are the key. Meanwhile, the co-design approach helps fight against the harsh industrial radio environments by considering both the industrial process state and the transmission energy in search of optimal network operations [97].
Besides the data plane performance in industrial data transmissions, general discussions are also underway in the control and management planes. Studies of designing and improving protocol (plane) interfaces are identified in both horizontal and vertical directions.
Horizontal interfaces refer to the ones between interconnected nodes to fulfill specific network functions, such as the clock synchronization between devices. Timing is critical regarding the real-time performance in industrial applications [106]. Time sensitive networks (TSN) protocols in local and metropolitan area networks (LAN and MAN) target the real-time performance for mission-critical information updates [19]. The IEEE 1588 precision timing protocol (PTP) and the IEEE 802.1AS protocol were initially developed for the device synchronization in the hardwired Ethernet. Recent studies have confirmed that these protocols can also serve in WLAN for the synchronization between distributed and heterogeneous IoT devices with mission-critical data [123], [126]. The other schemes developed in TSN, such as deterministic scheduling algorithms [20], [21] and MEC schemes [96], can further reduce the end-to-end service delay and ensure the real-time performance in industrial wireless transmissions.
Vertically functions in the control plane secure the data integrity and service consistency when the IoT information flow travels through the protocol stack. IIoT assigns a unique IP address to each network device as its identity in communications. The full-length address, such as an 128-bit IPv6 address, is a big overhead for many IoT devices that operate at the low power and transmit small-sized messages, just of a few bytes. Current industrial wireless practices usually adopt the shortened subnetwork addresses in their networks to save the header space. However, such proprietary addressing methods impair the interoperability of IIoT data in the wide area deployment and increase the processing delay to allow the upper layers to translate the address between different subnets. The Internet Engineering Task Force (IETF) released the encapsulation and header compression mechanisms, known as IPv6 over Low-Power Wireless Personal Area Networks (6LoWPAN) [131], which unify such operations in the IEEE 802.15.4 devices. IETF also released the similar approach for IPv6 over BLE devices [132]. Currently, there are further discussions on replacing the IP-based addressing by topic-based information distribution mechanisms, known as information centric networks (ICN) or named data networks (NDN) [133]. IIoT deployments and wireless networks are recently considered in such a new paradigm [134].
The wireless network design in emerging IIoT applications is a multi-disciplinary challenge. IIoT revisits strategies of data utilization in industrial CPS which (1) integrate the end-to-end application procedures into the “OT-IT-OT” data flows and (2) orchestrate vertical and networked industrial systems with case-specific configurations for the incoming orders [119]. Since key performance indicators (KPIs) and network bottlenecks are dispersed in production activities, factory environments, and on-premises/cloud IT platforms, precisely identifying and tackling them in the design involves close collaborations between OT system engineers, enterprise IT architects, and wireless network planners [31]. As discussed above, most of current design efforts are still focused on specific techniques used for separate use cases or design principles for general wireless communications, which have only partially addressed the complexity of such a problem.
Industrial wireless users, especially those in the small and medium-sized enterprises (SME), may lack of the required knowledge to comprehensively reviewed their wireless needs. Recently industrial wireless users are calling for new wireless design principles and guidelines to support their digital vision on CPS and Industrial 4.0 [32], [77], [105], [135]. Wireless industries have also echoed the importance of building such a common design language allowing different parties to effectively and efficiently exchange their knowledge and opinions [83].
Among the earliest attempts, NIST has been conducting a series of measurements and evaluation activities to standardize generic procedures of implementing industrial wireless networks [30], [32], [35]. As part of such efforts, we present a new design reference framework in this paper which reviews a variety of factors in a generic use case and identifies their impact on the design of efficient industrial wireless networks.
Fig. 2 illustrates the structure of the proposed reference framework. In a wireless use case, the design workflow consists of four stages that are performed sequentially including 1) survey & characterization of use cases, 2) measurement & quantification of system metrics, 3) modeling & verification of design problems, and 4) delivery & evaluation of solutions. Accordingly, the whole design problem can be divided into four concatenated subproblems which have their own tasks described by the names. Three distinct moves are identified between the consecutive stages which are impact mapping, requirement instantiation, and solution exploration, respectively. In each move, the former stage’s output is translated to the next one as the input. The details about the design framework and its components will be addressed in the following sections.
