IoT-connected Group Deployment of Unmanned Vehicles with Sensing Units: iUAGV System

experiments on the collaborative between and successfully three-dimensional collaborative As of with


Introduction
There are many drone-related products on the market, including remote control toys and model airplanes in a broader sense. In a narrow sense, products are controlled by a ground base and carry necessary heavy tools when flying out to complete assigned tasks. Ground vehicles have also been well-developed. There are many types of tracked and wheeled vehicles, and even two-wheeled self-propelled vehicles featuring an inertial measurement unit (IMU) that helps balance the vehicles.
In addition, through an app's collaborative communication mechanism, users of different vehicles can communicate with each other to discuss tasks and make decisions, or make requests for three-dimensional support during tasks. In this way, the efficiency and flexibility of operations can be improved. Our goal is to integrate these functions into a multipurpose monitoring system, as shown in Fig. 1. In short, iMonitor box has implemented multiple monitoring functions and is attached to either by air or ground vehicle.
This system can be monitored and controlled in real time through a mobile phone app, (1) using WiFi as the transmission interface, and when WiFi base stations are connected to a cloud platform, real-time monitoring and moving control can be greatly improved. The multipurpose monitoring system called iMonitor has an installed microcamera, infrared thermograph, and gas sensors that can perform tasks such as face recognition, (2,3) thermal imaging, and gas detection, respectively.
In many cases, it is difficult to achieve the objectives of a task by relying on a single aircraft or ground vehicle. The objectives may be changed accordingly. If necessary, an air-ground cooperative operation system is needed to send commands to aircraft or ground vehicles. That is, ground vehicles request support, or directly group three-dimensional air-ground vehicles, increasing flexibility and the ability to make rapid changes when completing a task. Hence, a system is proposed in this study that provides integrated operation and communication for both air and ground vehicles, which we refer to as an unmanned aerial/ground vehicle (iUAGV) system, where the i refers to the attached iMonitor.

Related Work
Jing et al. conducted research on controlling a car via a remote control interface in an Android system, and live video streams were sent to a phone via WiFi. (4) Bravo et al. watched video streams on a phone connected to WiFi and to enhanced hardware in a remote control car equipped with a pulse-width modulation (PWM) speed adjustment function, an off-road function, and a monitoring function in harsh environments. (5) Alongside the advancement of artificial intelligence (AI), face recognition systems have also been improved significantly. Through deep learning, a dynamic access control system based on face recognition can help to recognize members of a laboratory. (6) Moreover, in Ref. 7, face recognition was performed using a combination of Python and OpenCV as an example of supervised learning. Branzila et al. collected temperature, humidity, and gas (CO, CH 4 , natural gas) measurements to determine the information immediately, then the data were saved in a cloud system. (8) Li et al. (9) studied automatic ground mapping and effective route planning using a system of cooperating unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs). In Ref. 10, a fixed-wing unmanned aircraft was examined as an important tool for low-cost aerial surveillance and mapping in remote areas. In Ref. 11, a novel air-ground integrated detection group was used for vehicle landing detection and potentially for the search and rescue of manned spacecraft. With the increasing demand for autonomous reconnaissance and surveillance tasks in urban environments, in this study, we introduce a method for tracking moving targets by adopting a collaborative route planning algorithm for UAVs and UGVs. (12) A stereo visual system with multiple UAVs based on the system in Ref. 13 was proposed to assist UGVs in global route planning even in environments where GPS is blocked.

Integration of IoT, cloud platform, and app
The current study on the deployment of a group of unmanned vehicles adopts the concept of the Internet of Things (IoT) (14) through the use of the multipurpose monitoring system called iMonitor that is attached to unmanned vehicles as a payload, as shown in Fig. 2. The system uses phpMyadmin (15) open-source software as a server site platform in a cloud system. (16) The images, videos, and gas information obtained by the multipurpose monitoring system on the app will be uploaded to the cloud system simultaneously, without occupying the built-in memory of the sensors and platforms that are inside the iMonitor box (a white box), as shown in Fig. 2.

Multipurpose monitoring box and unmanned vehicles
The iMonitor multipurpose monitoring system, as shown in Fig. 3, is treated as a payload and attached to the UGV and UAV. The modules of iMonitor are listed in Table 1, which allow it to offer several sensing services such as face recognition, infrared thermal imaging, top-view live streaming, and gas detection. The UGV (Fig. 4), which is a self-balancing car, (17,18) carries the payload for monitoring the existing situation in the surrounding area on the ground. The UAV (Fig. 5) is a ZD550 PID-controlled drone (19,20) that is capable of flying at low speeds in still air, and its attached payload is used for low-altitude space surveillance and monitoring the current situation in the air. The modules of the UGV and UAV are listed in Tables 2 and 3, respectively. The monitoring system of iMonitor is mainly controlled by a mobile app. Figure 6(a) displays the starting screen of the app. As shown in Fig. 6(b), there are two main buttons on the main Panten DS-CO2-20 carbon dioxide sensor module 9 MQ-4 methane sensor module 10 Grove multichannel gas sensor   page: "Streaming" to view live streaming or gas information and "Folder" to view videos and images retrieved from the in-cloud system.

Cooperative communication protocol between UAVs and UGVs
The UGV and UAV equipped with iMonitor can be operated through an app and a server. The app has a communication panel that allows the UGV and UAV to communicate with each other and facilitates collaborative operation (21) between them, as shown in Figs. 7(a) and 7(b). Tarot TL68A00 two-axis gimbal 8 SJCAM SJ4000 sports camera 9 ZD550 55 cm four-axis rack 10 TAROT 6S 4108-380KV brushless motor 11 14 inch carbon fiber propeller 12 Hobbywing Lotte 40A electric transformer   (a) Scenario 1: Heart-Beat wake-up notification The first scenario is a Heart-Beat wake-up call that allows an unmanned vehicle to access the communication interface. Each time an unmanned vehicle enters the communication range, a Heart-Beat wake-up call will be sent to notify other unmanned vehicles, as shown in Fig. 8. (b) Scenario 2: UGV calls UAV to assign tasks Scenario 2 refers to a UGV equipped with iMonitor at a certain location for environmental monitoring. However, it is observed that the location is large and blocked by a wall. To enter the location, the UAV needs to find the entrance and sends information back to the unmanned vehicle. After the UGV arrives, it is found that the entrance has collapsed and is closed. Thus, the UGV assigns the UAV to complete the task. The Callout function on the communication panel is used to make a Heart-Beat wake-up call. That is, the UGV calls the UAV at the vehicle's location for the assigned task, as shown in Fig. 9, in order to upload the GPS location of the UGV to the server and to conduct stable communication through flag setting.
(c) Scenario 3: UAV calls UGV to hand over task In Scenario 3, a UAV equipped with iMonitor heads to a factory for environmental monitoring. After the external monitoring is completed, the UGV is called at the indoor entrance since the UAV is unable to enter the room. At this moment, the Heart-Beat wakeup call is performed, and the calling process between the UAV and the UGV is started where the UAV sends the location of the UAV to the UGV as well as hands over the task to the UGV, as shown in Fig. 10. That is, the GPS location of the UAV is uploaded to the server to conduct stable communication through flag setting.
(d) Scenario 4: UGV assigns task to multiple UAVs (two UAVs are used in this scenario) Scenario 4 refers to a UGV equipped with iMonitor. However, it is found that the current location is empty and multiple areas are to be monitored. Therefore, it is necessary to employ two UAVs to complete the task faster. To complete the task, the user uses the Callout function    on the communication panel. After the Heart-Beat wake-up call, the UGV requests two UAVs to head to its location for support. As shown in Fig. 11, the communication process uploads the GPS location of the UGV to the server and enables stable communication to be conducted through flag setting.
(e) Scenario 5: UAV assigns task to multiple UGVs (two UGVs are used in this scenario) In Scenario 5, a UAV equipped with iMonitor heads to a certain place to monitor it and finds that there are two warehouses in the area. Therefore, it is necessary to call two UGVs to complete the task more efficiently. The user uses the Callout function to make a Heart-Beat wake-up call. The UAV calls two UGVs, UGV1 and UGV2, and requests them to head to its location for support. The communication process is shown in Fig. 12, where the GPS location of the UAV is uploaded to the server for communication through flag setting.

Face recognition and forehead temperature
When the system detects a face in a database successfully, it will be compared with the person who has registered, and his/her ID with the current forehead temperature will be shown. If the person has not registered before, it will display the word "Stranger" and his/her current forehead temperature. In the UGV, the camera sensor in iMonitor can accurately mark the person and display his/her ID with the forehead temperature when s/he is moving or standing, (22) as shown in Figs. 13(a) and 13(b).

Gas detection and top view of ground
In the UAV, four gas detectors implement air quality detection (23) by detecting 12 gases: PM2.5, CO 2 , LPG, CO, smoke, NH 3 , NO 2 , C 3 H 8 , C 4 H 10 , CH 4 , H 2 , and C 2 H 5 OH. Moreover, the large camera sensor built in the UAV can take live top-view images of a grass lawn. Both types of information are saved in an in-cloud database (24) and concurrently displayed on the app to inform the user, as shown in Figs. 14(a) and 14(b).

UAV calling UGV
When performing a monitoring task on a campus, a UAV may not be able to lower its flying height because there are many trees or buildings, and it may be too large to enter indoor areas, so it is necessary to request support and assign a task to a UGV through the app as shown in Fig. 15(a). The UGV then arrives at the location and completes the task [ Fig. 15(b)]. The execution procedure of a UAV calling a UGV is listed in Table 4. Through location sensing and time sensing, a trajectory tracking diagram can be generated during the execution of the task to verify the trajectory of the UAV, as shown in Fig. 16.  Table 4 UAV-calling-UGV execution procedure.
Step Operation 1 UAV's app sends support task messages to UGV 2 UGV's app receives task messages 3 Task accepted and UGV moves to designated location 4 UGV arrives at designated location, enters room, and performs support task

UGV calling UAV
As shown in Fig. 17(a), a UGV monitoring a campus may encounter a situation where the terrain is complicated or there are steps, making it difficult for the UGV to operate. Then the UGV can request support from a UAV and assign a task to the UAV through the app, as shown in Fig. 17(b). Through location sensing and time sensing, the UAV-calling-UGV procedure is executed as listed in Table 5. A trajectory tracking diagram can be generated during the execution of the task to verify the moving path of the UAV, as shown in Fig. 18.

UAV calling two UGVs
When monitoring a campus, a UAV may encounter large buildings or gymnasiums. As shown in Fig. 19(a), to complete tasks efficiently, two UGVs are requested for support through the app to help take people's forehead temperatures in the building. The two UGVs arrive at the designated location to complete the task, as shown in Fig. 19(b). The execution procedure for a UAV calling two UGVs is listed in Table 6. Through location sensing and time sensing, a trajectory tracking diagram of the UAV during the execution of the task can be obtained to verify its path, as shown in Fig. 20.

UGV calling two UAVs
While monitoring a campus, a UAV receives a support request from other buildings and sends requests for support to two UAVs, as illustrated in Fig. 21(a). Soon afterwards, the two Table 5 UGV-calling-UAV execution procedure.
Step Operation 1 UGV's app sends support task messages to UAV 2 UAV's app receives task messages 3 Task accepted and UAV moves to designated location 4 UAV arrives at designated location and performs support task  UAVs arrive at the designated location to perform the task, as shown in Fig. 21(b). The execution procedure for a UAV calling two UGVs is listed in Table 7. Through location sensing and time sensing, a trajectory tracking diagram of each UAV during the execution of the task can obtained to verify its path, as shown in Fig. 22. Table 6 Execution procedure for UAV calling two UGVs.
Step Operation 1 UAV's app sends support task messages to two UGVs 2 UGVs' app receives task messages 3 Task accepted and two UGVs move to designated location 4 Two UGVs arrive at designated location and enter room to perform support task

Conclusion
We have developed a novel cooperative communication protocol to efficiently implement the collaborative operation between a UAV and a UGV. Through position sensing and time sensing, the UAV and UGV can achieve autonomous control by sending commands to each other with timely seamless hand-off WiFi. In particular, the user can track the trajectory of UAV flight through position sensing and time sensing to determine whether the UAV has done a task. According to experiments on the collaborative operation between UAVs and UGVs, the proposed approach enables the group deployment of unmanned vehicles to successfully execute three-dimensional collaborative operations such as gas detection, top-view imaging of the ground, face recognition, and infrared thermal imaging. As a result, the cooperative communication protocol through position sensing and time sensing efficiently realizes the IoTconnected group deployment of unmanned vehicles with sensing units.
Step Operation 1 UGV's app sends support task messages to two UAVs 2 UAVs' app receives task messages 3 Task is accepted and two UAVs move to designated location 4 Two UAVs arrive at designated location and perform support task