Internet of Multimedia Things (IoMT): Techniques and Applications

Internet of Multimedia Things (IoMT): Techniques and Applications disseminates research efforts in the security and resilience of intelligent data-centric critical systems to support advanced research in this area. Sections cover the background of IoMT Architectures and Technologies, describe the problems that arise in IoMT Computing and protocols, and illustrate the application of IoMT on Industrial applications. The book will be beneficial for engineers, developers, solution designers, architects, system engineers and specialists from professional environments interested in the IoMT to seek appropriate solutions to their specific problems.

• Addresses recent developments, along with relevant prospects on opportunities and challenges in IoMT
• Presents the concept and vision of the IoMT, whose potentialities are discussed with speci?c use-cases
• Discusses the distinct architectural design and characteristics of IoMT as compared to the existing multimedia systems

• Language: English
• Publisher: Academic Press
• Release date: Jun 16, 2022
• ISBN: 9780323900829

Book preview

Chapter 1: A review on Internet of Multimedia Things (IoMT) routing protocols and quality of service

Dinesh Singh; Ashish Kumar Maurya; Rupesh Kumar Dewang; Niharika Keshari    Department of Computer Science & Engineering, Motilal Nehru National Institute of Technology Allahabad, Prayagraj, India

Abstract

An interconnected network of multimedia devices is known as the Internet of Multimedia Things (IoMT). The characteristic of IoT to produce scalar sensor data differentiates this from IoMT. The IoMT consists of tiny multimedia devices that require less energy consumption and real-time data delivery. IoMT requires many multimedia devices connected through a stable network, efficient routing protocol, and lightweight computation to achieve the objective. The suitable design of the routing protocol improves the performance of IoMT in many folds. The Internet Engineering Task Force (IETF) user working group has standardized the IPv6 routing protocol for low power and lossy networks (RPL). The RPL is ubiquitous for resource-constrained devices and perfectly suitable for IoMT. In IoMT, the significant issues with the routing protocols are a large number of heterogeneous devices, high bit error rate, frequent link breaks, assurance of the quality of service, and emission of CO2. In addition to the suitable traditional routing protocol of IoT, many green routing protocols, fault-tolerant routing protocols, context-aware, and energy harvesting-aware routing protocols are specially designed for IoMT. The efficient design of routing protocol and improved quality of service (QoS) widens the application spectrum of IoMT in intelligent environments. This chapter presents a comprehensive review of traditional and most recent routing protocols and QoS in IoMT. The design of these protocols successfully satisfies the current and future requirements of the IoMT network in terms of handling a large volume of data, heterogeneous data types, and energy issues.

Keywords

Internet of Multimedia Things; Routing protocols; Quality of service; IoMT; RPL; IoT; Multimedia applications; Survey

1.1 Introduction

During the last decade, Internet of Things (IoT) has grown very fast and connected numerous devices worldwide. IoT can provide anytime, anyplace connectivity for anyone and anything [1,2]. Presently, 23.8 billion units of devices are connected to the Internet, and this amount is expected to jump 41.2 billion units by 2025 [3]. The IoT devices have limited memory, size, energy, and computing capabilities [4]. Thus, these IoT connected devices highly depend on reliable communication network and efficient routing protocols [5]. Technical developments in IoT operating systems [6], IoT enabling platforms [7], 5G IoT [8,9], wireless sensor networks (WSNs) [10,11], software defined networks (SDNs) [12], mobile ad hoc networks (MANETs) [13–16], vehicular ad hoc networks (VANETs) [17,18], fog/edge computing [19,20], cloud computing [21], are facilitating IoT to comprehend to connect anything and anywhere [22].

The massive growth in multimedia on-demand traffics such as images, audios, and videos, has evolved the concept of Internet of Multimedia Things (IoMT) from IoT [22]. IoMT devices produce massive amount of multimedia data and are more constrained than IoT devices. They need more processing power, massive memory storage, adequate bandwidth, and consumes more power to support the underlying application efficiently [22,23]. The comparison between key data characteristics of IoT and IoMT [22] has shown in Table 1.1. The IoMT has diverse applications like smart grid, smart cities, industrial IoT, smart homes, health IoT, smart farming and agriculture, traffic monitoring, soil health monitoring, and satellite control systems. Most of these applications are interactive applications that involve reliable and timely delivery of data. Hence, it demands efficient routing protocols, and enforces stringent quality of service (QoS) parameters [23]. Due to transmission of unstructured, bulky and multimedia data over the network, IoMT needs revision of existing routing protocols of IoT which focus on energy aware computing, efficient feature extraction, and optimizing routing criteria to minimize delays [23]. The quality of service performance in IoMT is determined by various metrics such as bandwidth, packet loss ratio, delay, throughput, resource management, and energy conservation [23]. The quality of service in technical white paper [24] is defined by Microsoft as: Network QoS refers to the ability of the network to handle this traffic such that it meets the service needs of certain applications. The frequent packet loss and redundant packet delivery degrade the quality of constrained multimedia applications in IoMT network. The massive amount of data produced in IoMT network from many heterogeneous sources creates the processing overburden on the routing devices, resulting in packet loss.

Table 1.1

The multimedia traffic is growing very fast which increases newer challenges for computing, sharing, storing, and transmitting the multimedia data. Computing data in IoMT needs novel methods for fog/edge and cloud computing devices. Similarly, for storing multimedia data, different compression/decompression methods are required in IoMT [22]. For example, utilizing a standard IoT routing protocol called RPL (Routing protocol for low-power and lossy networks) [25] in IoMT deployment scenarios, requires more improvements in terms of fault tolerance, energy-awareness, delay-awareness, and load balancing [22]. The Green-RPL [26], Context-Aware and Load Balancing RPL (CLRPL) [27], and free bandwidth (FreeBW)-RPL [28] are some modified versions of RPL protocol for IoMT. In this chapter, we give a comprehensive review on routing protocols and quality of service in IoMT.

The remainder of the chapter is organized as follows: In section 1.2, we illustrate the working methodology of routing protocols used in the IoMT network. The QoS routing protocols are explained in section 1.3 of this chapter. The conclusion and future directions is presented in section 1.4.

1.2 Routing protocols in IoMT

Wireless Sensor Network (WSN) performs a crucial role in assisting IoMT in diverse application spectrum. The energy constraint of nodes in WSN somehow restricts it to fulfill the expectations of IoMT applications. The efficient routing protocols save the energy of WSN nodes and thus, allow to carry off a massive amount of IoMT data. Recently, cluster-based routing has attracted due to less communication overhead and reduced routing energy consumption. The cluster members of a cluster can save their routing energy expense because of the routing assistance of the cluster head. But, since everything has its pros and cons, cluster-based protocol has its disadvantage.

In case of battery discharge or malfunctioning of the CH, all the cluster members would fail to transmit their collected data to the destination, which would defeat the smooth functioning of the IoMT application. Thus, the fault-tolerant approaches are in use in cooperation with cluster-based routing protocol for IoMT applications. Here, we illustrate the working principle of a fault-tolerant-based routing protocol used for IoMT.

1.2.1 Fault tolerant routing protocol

The fault-tolerant routing protocol [29] uses cluster-based scenario for the processing of IoMT application data. It uses the cluster join method for handling faulty cluster heads (CHs). Whenever a fault is detected in a CH, its cluster members are the first ones to catch it as they fail to receive an acknowledgment from their CH upon receiving transmitted data. So, they broadcast a help message to their neighboring CHs for re-selecting their CH. Out of all the respondents of the help message, the closest one is selected as the best backup CH. But, even this method has some leftover loopholes described as follows:

•  If there are many faulty CHs, they will introduce a new problem of help message explosion.

•  Each member of the faulty CH cluster may select a different CH as its backup for data routing, and then there would be a problem in handling that cluster's data.

The possible solutions to the above problems are:

•  One of the solutions is the re-clustering of the entire network nodes. But, this is a laborious and costly process.

•  Another solution is to keep a CH entirely for backing up the failed CHs by not initially assigning a cluster to it.

•  The other solution includes identifying overlapping nodes (two nodes having similar coverage area) and putting one of such nodes into sleep mode, then wake it up only when the existing CH fails.

In view of these solutions to overcome the issues, a fault-tolerant routing protocol is illustrated in [29]. The protocol performs the following operations:

1.  The protocol initially detects the failure in CHs and record faulty and nonfaulty CHs.

2.  It analyzes the energy levels of nonfaulty CHs and attempts to construct the virtual CH. The data is routed through this virtual CH to the destination node.

3.  The maximum fault-tolerant capability of nonfaulty CHs is calculated to accommodate the members of faulty CHs.

4.  The protocol uses a flow-bipartite graph (FBG) to estimate the energy cost of IoMT data routing. A flow-based pairs of faulty and fault-free CHs are formed to create an FBG.

5.  Using the FBG, a flow transmission pattern is identified between the faulty and fault-free CHs. This indicates that which of the faulty CH is tolerated by which of the fault-free CHs.

The virtual CH and FBG construction methods used in this routing protocol are explained below:

Virtual CH

The steps involved for creating of a virtual CH and virtual super frame are:

1.  The destination node organizes the available energy of all the failure-free CHs, as shown in Fig. 1.1[29]. There are three components in that total available energy: the energy taken for receiving the IoMT data from all the failure-free members, the energy is taken to aggregate the IoMT data obtained from the cluster members, and the energy taken to route the aggregated data from the CH to the destination node.

Figure 1.1 Organization of the virtual CH with the available energy of all the fault-free CHs.

2.  The remaining energy of a failure-free CH after using the above three components is used for virtual CH construction. The failure-free CHs receive IoMT data from failure-affected cluster members, aggregate the data and then transmit them to the destination node, as shown in Fig. 1.2[29]. But, which failure-free CH is tolerated by which of the faulty node remains unknown. Thus, the concept of average transmission energy consumption is used to analyze energy used in fault tolerance.

Figure 1.2 Data aggregation of faulty CHs by fault-free CHs.

3.  A virtual superframe structure is constituted for some failure-free and some failure-affected members. The number of such virtual super frames transmitted by the virtual CH is defined as its transmission capability. Thus, the transmission capability is derived by dividing the sum of the energies of all failure-free CHs by sum of failure free and fault-tolerant energy consumption.

4.  Finally, the verification of fault-tolerant capability begins. After confirmation of one or more CH failures, the minimum data of faulty CHs required to be handled by the fault-free CHs is calculated. If that data is greater than the availability of data received in fault-free CHs, then the purpose of IoMT application is not solved.

Flow-Bipartite Graph

After creating virtual CH, flow-based pairs of faulty and fault-free CHs are formed to develop a flow-bipartite graph (FBG). An FBG, as shown in Fig. 1.3 [29], is a graph whose vertices are divided into the source and destination sets such that every edge connects a vertex in source to a vertex in destination and is associated with a transmission cost and an energy cost. Each destination vertex is attached with a capacity cap. To establish an FBG, the following steps must be followed –

1.  All faulty CHs act as source, and failure-free CHs as destination node.

2.  The amount of input flow from each source vertex is set to the demanded amount of data (i.e., the data to be gathered at failure-free CHs due to failed CHs).

3.  The capacity cap of the destination vertex is set to the available energy of the failure-free CHs.

4.  The distance between the farthest member of a faulty CH and a fault-free CH is calculated and compared with a node's communication range. If the former is larger, an edge can be created between those source and destination nodes.

5.  The transmission cost of an FBG is calculated. It is the sum of cost of transmitting data from failure affected members to the chosen fault-free CH and for transmitting from failure-free CH to the destination node.

Figure 1.3 An example illustration of flow-bipartite graph formed by faulty and fault-free CHs.

Now, the fault-tolerant energy cost of FBG is calculated.

1.  It is the sum of energy taken to assist the transmissions of failure-affected members to the destination node and the energy taken to transmit the IoMT data of the failure-free cluster members to the destination node.

2.  With this, fault-tolerant load distribution can be achieved by making two or more fault-free CHs tolerate a single faulty CH.

3.  The minimum cost flow (MCF) problem is also solved using the FBG approach such that the total minimum transmission cost can be obtained.

1.2.2 DDSV routing protocol

An optimizing Delay and Delivery Ratio for Multimedia Big Data Collection in Mobile Sensing Vehicles (DDSV) routing protocol [30] uses optimized routing criteria so that the delay involves in data collection and data delivery is minimum. It considers a road scenario that has multiple intersections on a fixed distance. The two types of moving vehicles, Bus and taxi, are considered on the road to receive the IoMT data generated from the source, i.e., traffic light. The vehicles deliver the received IoMT data to the other vehicles or the Data Collection (DCs) centers located at the intersection of the road segment based on the optimized routing criteria. The DDSV routing scenario is shown in Fig. 1.4 [30].

Figure 1.4 DDSV routing scenario.

The route for IoMT data transfer consists of intersections and road segments. The protocol takes data packet priority as well as vehicular priority for decision-making purposes. The data packet priorities are assigned on the basis of location or area from where the data is generated. The routing scenario is partitioned into many urban and suburban sections.

1.  At each intersection i of the road, the routing decisions are taken. The routing decision is based on the minimum of expected data delay ( ) on the possible routes.

2.  The expected data delay ( ) is computed as the function of two components: movement probability ( ) of vehicle ε at intersection i to the routing decision and data forwarding probability .

3.  There are multiple routes between intersection i and intersection j to transfer the IoMT data. The movement probability ( ) of vehicle ε at intersection i is computed as

(1.1)

where is the priority of vehicle ε and is the probability of the vehicle ε to travel in the suburban area. The is defined as the ratio of time spent by a vehicle in suburban area to the time it taken in moving as per the historical trajectory datasets.

4.  A vehicle finds the following possibilities, as shown in Fig. 1.5[30], at each intersection i to route the IoMT data.

Figure 1.5 DDSV outing decision at each intersection.

The movement probability ( ) of vehicle ε at intersection i in view of the above routing possibilities can be computed as

(1.2)

Where is the probability of the vehicle ε to move on the road segment between intersection i and intersection j and is the probability of the vehicle ε to deliver the data packet to another vehicle moving on the road segment between intersection i and intersection j.

5.  The data forwarding probability ( ) for a vehicle ε depends on the vehicle type, i.e., Bus or taxi.

a.  The Bus vehicle has predefined trajectories and the probability depends on the length of the road segment and average speed of the vehicle only. For Bus type vehicles, it is calculated as:

(1.3)

where is the length of one of the subset of the road segment between intersection i and intersection j and is the average speed of the Bus on the respective road segment.

b.  The taxi type vehicles have no fixed trajectories and thus, the probability considers the transmission range, vehicle speed as well as the wireless transmission delay for computation. It is calculated as:

(1.4)

Where r is the transmission range of vehicle's Wi-Fi channel, c is the wireless delivery delay, and is the density degree and vehicle's speed on the road segment between intersection i and intersection j, respectively.

6.  For routing purpose, it is assumed that the priority of the vehicle moving in the suburban area is high as compared to vehicles moving in urban area.

1.2.3 Optimal routing for multihop social-based D2D communications

An optimal routing algorithm for multihop communication between Device-to-Device (D2D) is given in [31]. The algorithm is well suited for efficient IoMT data communication and 5G networks. It considers the social behavior of the devices in communication to their neighbors. The algorithm computes the trust probability for D2D communications based on the rank model. Both, random and fixed locations of the base stations are considered for measuring the connection probability (CP) between any pair of devices. The measurement of CP is taken with and without, channel state information (CSI). The network model of the Social-aware multihop D2D routing algorithm is shown in Fig. 1.6 [31]. The transmission of information between D2D transmitter and D2D destination using the number of D2D relays is taking place. Here, the interference due to the cellular communication equipment (CUEs) and position of base stations (BSs) is considered in follow-up. The Nodes are working in half-duplex mode. A single antenna and D2D transmitter are assumed to be at the origin, and D2D destination is at a fixed distance away from the origin, whereas using the Poison Point Process (PPP), BS and CUEs are modeled.

Figure 1.6 Network model of optimal routing for multihop social-based D2D Communications.

With the help of real data traces from online social networks, trust connectivity of D2D based on rank-based trust model is calculated. The probability that is trusted by is given as

(1.5)

where β is the parameter from the rank-based model and is a normalizing factor.

A trusted connection will be established only when two nodes can trust and communicate with each other and thus, the trusted connectivity probability (T-CP) is defined as

(1.6)

where the connectivity probability between and is denoted as .

In case of random BS and fixed BS scenario, both CSI aware and not CSI aware situations are actively taken for the computation of the CP. In CSI aware situation, the channel state information between BSs and D2D transmitter is used in CP computation. However, none of the channel state information between BSs and D2D transmitter is used in CP computation. In random BS scenario, the CP between any D2D devices for given depends on the intensity of BS and CUE, path lose exponent, transmit power of each CUE, threshold . However, in fixed BS scenario, rest other parameters are similar to one that we use in random scenario except the two. Here, in place of intensity, the location and number of BSs is used in CP computation between any D2D devices for given .

The objective of the routing algorithm is to find the optimal path between the D2D transmitter and the receiver. The path selection is based on the maximum value of the T-CP at every intermediate link. The maximum T-CP between D2D transmitter and D2D destination in presence of multiple D2D relays can be obtained by routing algorithm which helps in selecting optimal path. The optimal path satisfies:

(1.7)

where denotes the set of all potential paths between the D2D transmitter and receiver. The routing algorithm finds the maximum T-CP between source and destination in a weighted graph with the help of standard Dijkstra's algorithm. Initially, at each D2D node, distance between itself and other D2D devices and BSs are calculated and stored in topology information, which contains the neighbor list. The adjacency T-CP matrix is obtained by using updated topology information. Therefore, the D2D transmitter is set up as permanent node and all other nodes as temporary nodes. Here, we calculate the T-CP for all possible neighbors links and find the link with maximum T-CP. The intermediate destination node in selected link with maximum T-CP is set as permanent node and now keeps on finding maximum T-CP unless all the nodes are marked as permanent node. Thus optimal path and maximum TCP of multihop D2D communication are obtained. The Computational complexity of routing algorithm is as same as that of Dijkstra algorithm, i.e., .

1.2.4 Green-RPL routing protocol

The Green-RPL [26] is an energy-efficient green routing protocol for IoMT. It is an enhanced version of the RPL (Routing Protocol for Low-Power and Lossy Networks) protocol [25]. RPL is a routing protocol for resource-constrained devices that uses Destination Oriented Directed Acyclic Graph (DODAG) to maintain network topology. This DAG comprises multihop routes from leaf nodes to the root node. RPL optimizes an objective function and chooses the best path by selecting the desired predecessor nodes starting from leaf nodes. The previous RPL protocol implementations are not feasible for IoMT and do not consider the multimedia data.

In contrast, the Green-RPL protocol considers the data generated from multimedia devices. The Green-RPL protocol reduces energy consumption and carbon footprint emissions together with QoS requirements of applications. To guarantee QoS for a particular multimedia application, it determines the delay bound for the application. For example, in VoIP applications, the delay limitation usually is 120 msec. To ensure energy efficiency, the protocol considers the features of all the intermediary links between the leaf nodes and the root node and estimates the energy consumption by the chosen immediate predecessor node to support traffic needs for one more immediate successor node. An optimization model is given for this protocol based on various requirements and