Design and Implementation of a Mobile and Elevating Panoramic Acquisition Platform

Panoramic imaging is a new, rich media technology that provides an immersive user experience and has been widely used by large Internet companies. A panoramic acquisition platform is typically integrated with a vehicle carrier and installed by non-reversible means such as welding. The fixed height of the panoramic camera hinders panoramic photography on some roads because of the ever-changing surroundings. The aims of our study were to reduce the maintenance cost of the acquisition platform and obtain high-precision positioning information. Thus, we developed a detachable, elevating panoramic acquisition platform based on a high-precision positioning function. The developed platform was mainly composed of four parts: data acquisition, connection and fixation, support, and auxiliary control parts. The performance of the system was tested for different camera heights, and the installation and disassembly were found to be convenient, fast, and flexible. Moreover, the system overcame the height limitation imposed by overhead traffic facilities and adjusted the height of the panoramic camera in real time according to the heights of the targets. Additionally, through the coupling of multi-position sensors, the acquisition system obtained stable and high-precision shooting position information (at decimeter-level accuracy) as well as the panoramic platform attitude, speed, and other types of information required for diverse applications of panoramic images.


Introduction
Panoramic imaging, also known as three-dimensional (3D) panoramic imaging, (1) is an increasingly popular visual technology. The major difference between panoramic imaging and traditional streaming media, such as video, sound, and pictures, is that panoramic imaging is operable and interactive. Moreover, the data source of panoramic imaging is not simulated images generated by computers but real-life photos taken by single-lens reflex cameras or street-This version has been created for advance publication by formatting the accepted manuscript. Some editorial changes may be made to this version. and disassembly, high applicability, stability, and compactness. Furthermore, the vehicle carrier does not require modification, and non-motor vehicles can be used as carriers to adjust the height of the panoramic camera anytime and anywhere. Thus, the system can not only avoid colliding with overhead traffic facilities, but also adjust the height of the camera in real time to shoot targets of different heights for the best shooting effect. Moreover, the panoramic acquisition system can provide stable and high-precision positioning accuracy at the decimeter level by employing a shooting position acquired through multi-position sensor coupling. Additionally, the system can provide information such as the altitude and speed of the panoramic platform, and it can better support the subsequent application of panoramic images.

Related Work
Panoramic acquisition tools mainly use mobile mapping systems. Ground vehicle-mounted mobile mapping systems are fast and efficient map production systems that integrate various sensors, computers, and spatial positioning technologies that emerged in the 1990s. Several scientific research institutions and companies have successively developed vehicle-mounted 3D data acquisition systems that integrate GNSS, INS, a laser scanner, and a charge-coupled device (CCD) camera. A comparison of sensing instruments installed on panoramic acquisition vehicles in China and abroad is shown in Table 1.
With the rapid development of various related technologies, ground vehicle-mounted spatial data acquisition systems based on multi-sensor integration have developed rapidly and successfully entered practical use.
As early as 2007, Google developed a panoramic collection vehicle and launched their Street View Map Service, which provided street view maps in several countries and regions around the world. (6) When the street view test project was first launched, Google Street View personnel equipped several computers in a sport utility vehicle (SUV) and fixed cameras, laser scanners, and GPS equipment on the roof for the image acquisition tour. Today, 360° panoramic images are available in many parts of the world, and the street view camera has been made more intelligent through the latest machine learning and artificial intelligence technologies, helping Google to provide users with clearer and richer color rendering images as well as improved map information.
As an established surveying and mapping company, Topcon has also launched its own 3D mobile surveying system, IP-S2. The innovation of 3D data acquisition technology is a major driving force for the growth and expansion of the geographic information systems industry. The Topcon IP-S2 mobile surveying system is an advanced surveying and mapping system that avoids the bottleneck of multi-dimensional spatial information collection. Topcon IP-S2 adopts satellite positioning, inertial navigation measurement, and other fusion positioning methods; it uses integrated 360° camera and multiple laser scanning equipment, can quickly provide accurate multiple data streams, and can update data online, making it an economical system. (7) Currently, nearly 1000 IP-S2 mobile measurement systems have been successfully used in the fields of urban landscape roaming (such as 3D digital street view data collection), surveying road and pipeline facilities, and public safety.
Leador Corporation was established in 1999 to advance scientific research. The academic Deren Li became the chief scientist of the company and led the industrialization of mobile measurement technology. The panoramic laser mobile measurement system developed by the company can continuously collect high-resolution 360° panoramic images and output the result data through image mosaics and geographic reference software processing.
The high-precision map developed by AMAP has attracted recent attention, and several studies have also achieved significant progress, such as in data collection, production, and visual expression of high-precision maps in China and abroad. (8) The AMAP collection vehicle has an acquisition accuracy of approximately 10 cm and is equipped with two laser scanners and four CCD cameras, which complement each other as the system functions. The laser scanner mainly photographs road information such as edge lines and lane lines to scan 3D objects. The CCD camera photographs planar objects with road attributes, such as identification signs. When the lines and surfaces are superimposed, a 3D model can be formed. Additionally, AMAP's high- precision maps provide location information services for public areas by providing highprecision movement and high-quality, real-time traffic information. The KRPanoGUI 3D street view vehicle is a street view shooting system developed by Dongying Shengshi Panoramic Network Technology Co., Ltd. It can be installed on the top of any vehicle and controlled by a computer to completely automate the process from shooting to synthesizing panoramic images. The iScan integrated mobile 3D measurement system produced by Guangzhou Hi-Target encapsulates each sensor in a rigid platform, which can be conveniently installed on automobiles, ships, or other mobile carriers. (9) The SSW vehicle-mounted laser modeling and measurement system is a new generation of high-tech measurement equipment for fast data acquisition and processing jointly developed by Beijing Geo-Vision Information Technology Co., Ltd., and Capital Normal University. The system is composed of a control unit, data acquisition unit, and data processing software. Various vehicles can be used as carriers, and the system integrates a domestic 360° laser scanner, inertial measurement unit (IMU), GPS, CCD camera, turntable, odometer, digital measurable image (DMI), and other sensors. The system automatically produces a multi-element structured 3D model of a city (city holographic 3D rendering) and provides a geographic information database for smart city construction in this era of big data. It is a high-level, measurement-oriented mobile measurement system for holographic 3D modeling at home and abroad.

Design factors
The equipment that collects panoramic images is called a panoramic acquisition platform or panoramic collector. Generally, a panoramic acquisition platform is integrated and installed on a dedicated vehicle carrier. The platform and vehicle are fixed and connected by non-reversible traditional methods, such as welding and screwing of structural parts. The interior of the vehicle needs to be rearranged to provide the space for the installation of control cabinets, power supply devices, and other equipment. Thus, the typical design of a panoramic collector involves considerable internal and external transformation of the vehicle itself.
As reported in this paper, we designed a mobile and elevating panoramic acquisition platform that integrates various sensors, including panoramic cameras, a GNSS/INS combined INS, and a telescopic Mast LT (a column-type elevating platform) on the vehicle-mounted platform. The specific design requirements of the mobile and elevating panoramic acquisition platform are as follows: 1. The light detachable elevating panoramic collector, instruments and equipment, and fixed connecting devices should be of appropriate sizes and taken as a whole to ensure that the system can be easily installed on and disassembled from the top of ordinary commercial cars and non-motor vehicles. It should also be possible for the installation and disassembly to be entirely handled by two persons. 2. The light detachable elevating panoramic collector should suitably and remotely adjust the height of the panoramic camera according to the conditions of ground objects around the road (building heights, surrounding vehicles, etc.). This ensures that the best shooting effect can be obtained for the captured panoramic images. Additionally, when the vehicle encounters low overhead traffic facilities during driving, the panoramic camera should be lowered so that the vehicle can pass smoothly underneath without impact. 3. When the light detachable elevating panoramic collector is not running, it should be possible to use the acquisition vehicle as a normal vehicle without the modification of devices. 4. When the light detachable elevating panoramic collector is fixed on the vehicle carrier, the center of the panoramic camera and GNSS/INS integrated INS should be on the longitudinal central axis of the vehicle and located near the middle of the main horizontal base. All instruments and equipment on the panoramic acquisition platform should be calibrated only once and remain relatively fixed.

Design scheme
The panoramic acquisition platform is mainly divided into four components according to its functions: data acquisition, bracket, fixed connection, and auxiliary control parts, (10) as shown in Fig. 1. The design scheme of each part is detailed below.

Data acquisition part
The data acquisition part is the core part of the panoramic acquisition platform and collects panoramic image data, spatial position coordinate data taken by the panoramic camera, speed and attitude data, time information, etc. This platform uses the Ladybug 3 panoramic camera of Point Grey Research. (11) The panoramic camera is equipped with six high-quality CCD image sensors: five of the sensors are arranged in a horizontal loop, and the other one is placed at the top. This arrangement ensures the synchronous acquisition of six images and automatic operations such as image acquisition, processing, splicing, and correction. Multiple video images are integrated into full-resolution digital spherical video and full-motion video in real time as well as synchronized with other sensor devices through general-purpose input/output (GPIO) trigger lines. (12) The panoramic acquisition platform adopts an SDI-600GI dual-antenna optical fiber integrated INS to obtain spatial position data. The GNSS and INS are integrated through a combination of GNSS boards, three-axis fiber optic gyroscopes and accelerometers, and tight coupling technology. By means of mutual "compensation" between the GNSS and INS, the problem of long-term INS drift or restricted GNSS satellites is solved. This problem involves the inability to locate normally; hence, the proposed design ensures that when the GNSS signal is weak or out of lock, the system can be continuously and accurately positioned for a long time. (13) Additionally, the GNSS dual-antenna design improves the positioning (orientation) accuracy and shortens the initialization time of the positioning system. The two GNSS positioning antennas are located at opposite ends of the GNSS/INS integrated inertial navigation equipment, and the connection lines of the three components are parallel to the driving direction of the vehicle. The distance between the front antenna and the GNSS/INS integrated inertial navigation equipment in the driving direction is approximately twice the distance between the GNSS/INS equipment and the rear antenna, as shown in Fig. 2.
The GNSS/INS integrated inertial navigation equipment outputs pulse signals to the panoramic camera while acquiring its own position. Thus, the system controls the acquisition frequency of the acquired panoramic images and synchronizes the time between the acquired spatial position coordinate data and panoramic image data.

Bracket part
The bracket part is mainly used to fix various instruments and equipment of the panoramic acquisition platform, such as the GNSS/INS integrated inertial navigation equipment and panoramic camera. The bracket includes the horizontal base of the main board of the platform, the GNSS master-slave antenna-fixing strip board, and the panoramic camera telescopic Mast LT, as shown in Fig. 3. The main horizontal base of the platform, which is the main carrier of the instruments and equipment, is made of a rectangular aluminum alloy plate with a thickness of 2 cm. The horizontal base can bear a heavy weight without deformation, and the corners of the main plate are smoothed to prevent injuries during disassembly or handling of the platform. However, the GNSS master-slave antenna-fixing strip plate for carrying a GNSS antenna is made of aluminum alloy with a thickness of 1 cm because of the light load it bears.
To set the center of the panoramic camera, the coordinate origin of the GNSS/INS inertial navigation master and the connection line of the phase center of the master and slave antennas were set consistent with the axle direction. The relative positional coordinates of each instrument and equipment were conveniently calibrated. A card slot was excavated at the longitudinal middle position above the platform's main body plate. The GNSS master-slave antenna-fixing strip plate had the same width and thickness as the card slot; thus, it was embedded in the card slot and rigidly connected through four evenly distributed rows and eight evenly distributed screws. The two ends of the strip plate were semi-circular. Because the base of the GNSS master-slave antenna was a circular magnet, two round flat discs with a thickness of 5 mm and radius of 5 cm were fixed on the two ends of the strip plate by screws, and the GNSS masterslave antenna was fixed in such a way that the magnet is attracted on the round flat discus. However, the telescopic Mast LT and the GNSS/INS were located on opposite sides of the center of the main plate of the rectangular platform. At the bottom end of the elevating column was a motor for providing power, and at the top end was a connecting device to fix the panoramic camera. A bamboo-joint electric elevating column was adopted to considerably improve the flexibility of panoramic data acquisition in two ways. First, the camera height can be adjusted in real time according to the changes in the acquisition environment to achieve better image shooting effects. Second, when encountering low overhead traffic facilities, the camera can be adjusted to an appropriate height to ensure the smooth passage of the panoramic platform.

Connection and fixing part
To connect the panoramic collection platform and vehicle carrier, the platform should be parallel to the ground, convenient for installation and disassembly, flexible, mobile, and suitable for various types of ordinary commercial cars. A connecting and fixing device composed of four spherical tripod heads, two universal joints, and six corresponding vacuum suckers was designed. The system was designed to firmly fix the entire panoramic acquisition platform on the vehicle carrier (usually the roof) and provide a stable working environment for the panoramic acquisition instruments and equipment. The four vacuum suckers were combined with the spherical pan-tilt head. The upper end was connected to the main horizontal base of the mobile platform through the spherical pan-tilt head. This head adjusts the angles in the horizontal and vertical directions to ensure that the main horizontal base is indeed horizontal. The lower end firmly fixes the main horizontal base of the platform on the vehicle carrier through the vacuum suckers. The other two vacuum suckers were connected to the universal joint at the bottom of both ends of the GNSS master-slave antenna-fixing strip plate, one after the other, relative to the direction of the vehicle. To fix the GNSS master-slave antenna-fixing strip plate to the vehicle carrier, the position and height of the universal joint were made adjustable in any horizontal direction such that the strip plate was on the same horizontal plane as the main horizontal base. Figure 4 illustrates this arrangement.
Considering that the roof of the vehicle carrier is generally not horizontal, the height and contact angle of the six suction cups designed for the connection part were freely adjustable. Each vacuum sucker was provided with two sections of connecting rods, which included adjustment knobs to adjust the length of the rods. If the angles between the vacuum suckers and contact surface were different, the bending angles of the two sections of the connecting rods were also different, and the connection between the connecting rods and panoramic platform was provided with hemispherical sliders. The sliders could rotate 360° horizontally and 180° vertically, making it possible to adjust the platform to a horizontal position. Additionally, the length and thickness of the suction cup connecting rods were the same, and the four suction cup connecting rods of the main horizontal base of the platform were shorter and thicker. In this way, the platform was closer to the roof of the vehicle carrier and could support heavier weights, ensuring the relative stability of the platform. Since the GNSS master-slave antenna-fixing long strip is lighter, to ensure that the top of the vehicle carrier can be turned through a larger angle when the length is short, the two suction cup connecting rods of the GNSS master-slave antennafixing strip used to achieve this were fixed.

Auxiliary control part
The auxiliary control part is mainly responsible for the power supply and control of the panoramic collection platform. In this part, the power supply device provides power for the panoramic camera, GNS/INS combined inertial navigation equipment, telescopic Mast LT, and control storage equipment (such as a portable computer) used in the acquisition process. The power supply device comprises a storage battery pack, vehicle-mounted power supply connection port, and power supply input and output controller. During data acquisition, these devices are stored separately in the vehicle carrier and are not located on the roof platform.
Because several instruments require a power supply, the battery pack is often insufficient to keep the instruments and equipment running for a long time as data collection is gradually prolonged. Therefore, two power supply modes (a battery pack and vehicle-mounted power supply) were adopted in this system to supply power to the acquisition platform separately. The normal operation of the vehicle itself is typically required to supply sufficient power to the platform. However, when the vehicle stalls, all instruments cease functioning, which may also affect the instruments and collected data. Thus, a power input and output controller was used to switch between the two power supply modes, thereby continuously providing power according to the specification and ensuring the normal operation of all instruments and equipment on the panoramic acquisition platform.
The acquisition cooperative controller ensures seamless synchronization between the GNSS/INS integrated inertial navigation equipment and panoramic camera. That is, while the GNSS/INS integrated inertial navigation equipment obtains the position coordinates of the panoramic acquisition platform, the panoramic camera is triggered to acquire panoramic images. (14) This design realizes synchronization by connecting GPIO trigger lines and adopts a fixed time interval acquisition method. Thus, it acquires information such as spatial position, speed, posture, and time at regular intervals (generally 1 s) and triggers the panoramic camera to capture images to acquire image information.
Finally, the remote controller of the elevating column can accurately adjust the telescopic Mast LT wirelessly within its range, which is convenient for the height adjustment of the panoramic camera.

Cooperative power supply between vehicle power supply and battery
When the collection vehicle is started, the on-board power supply can charge the storage battery in real time. The battery can supply power to the panoramic camera, GNSS/INS integrated INS, telescopic Mast LT, and system controller. The scheme not only continuously provides sufficient electricity for each instrument and equipment to ensure the normal operation of the platform system, but also does not interfere with the collection of data, instruments, and equipment when the vehicle-mounted power supply is suddenly cut off. The design of the scheme is shown in Fig. 5.

Pulse-triggered synchronous data acquisition
The panoramic acquisition platform uses a panoramic camera to acquire panoramic image data. The system uses a GNSS/INS integrated inertial navigation equipment to acquire coordinate position data, attitude data, speed, and time information, which require a time synchronization controller to use the GNSS time for sending pulse signals to the panoramic camera through a GPIO trigger line for synchronous data acquisition. (15) The time synchronization is divided into soft time synchronization and hard time synchronization. Soft time synchronization is realized through the time system of each instrument or software. Each instrument or equipment has its own time system. When storing data from an instrument or equipment, the corresponding time is recorded simultaneously according to the time system of the instrument or equipment. Similarly, when processing data, data synchronization is realized according to the time recorded when each instrument collects data. Owing to the different time system accuracy of each instrument, this synchronization method sometimes encounters dislocation. Thus, synchronization with low time accuracy requirements can be considered. The hard synchronization of time uses GPIO trigger lines connected externally to each instrument to calibrate the time system of an accurate instrument (16) (usually GNSS time), and the data collected by each instrument and equipment are synchronized by sending pulse signals as a trigger. A schematic of the process of using pulse trigger GNSS time synchronization control to collect data is shown in Fig. 6.

Platform implementation
The overall design of the acquisition platform is shown in Fig. 7. A sample implementation of the light detachable elevating panoramic acquisition platform is shown in Fig. 8.
In summary, the system has a simple, compact structure, a firm support, and good portability. Additionally, the platform can be conveniently and quickly installed and disassembled through the vacuum suckers. (17) Not only can the platform be installed on the top of various types of ordinary automobiles and non-motor vehicles, it can also be used as an external sensor installation platform for on-board panoramic collection and measurement systems. The traditional and developed platforms are compared in Table 2.

System Implementation and Discussion
The panoramic camera used in the proposed system was composed of six high-quality Sony CCD image sensors, and the relative positions between the image sensors were fixed. The optical element of each camera was a micro lens with 3.3 mm focal length. The cameras were located at the top of the vehicle carrier and screwed at the bottom to the top of the telescopic Mast LT, which was capable of adjusting the height in real time through a structural member. The height of the car was approximately 1.5-1.7 m, the height of the elevating column was 0.5 m, the stroke was 0.5 m, the vertical distance between the platform and roof was 0.2 m, and the lowest height of the panoramic camera above the ground depended on the height of the vehicle carrier itself, which was approximately 2.2-2.4 m (and up to approximately 2.7-2.9 m). (18) The panoramic camera took pictures of scenes (the ground) on both sides of the road from different heights. Scenes of the same height had different positions and deformation in the panoramic photographs. When collecting panoramic images, the setting of the panoramic camera height depended on whether there were buses on the road, the heights of the surrounding buildings and trees, and the requirements of the photography.
For the best effect, a panoramic camera should ideally be at the same height as the object to be photographed. Thus, the height of the panoramic camera should be based on the height of the photographed object rather than the height of the camera itself when collecting panoramic image data. However, the panoramic camera of a traditional panoramic acquisition vehicle has a fixed height, making it impossible to adjust the height of the panoramic camera in real time according to the height of the scenery around the road for the best shooting effect. A schematic of the panoramic camera with different heights is shown in Fig. 9.
Therefore, in this study, we improved the design to remedy the defects of the traditional panoramic acquisition vehicle. The proposed design successfully realized automatic adjustments of the height of the panoramic camera for suitable shooting. (19) The system implementation results are as follows. (1) When the street view was taken of buildings on both sides of the road and when a plaque or outdoor advertisement was installed on a high-rise building or a bus passed by the collection vehicle, the panoramic camera was raised in a timely manner to its maximum height. This swift height adjustment minimized the deformation of high-rise buildings in the panoramic images taken, and information such as plaques and outdoor advertisements on the upper parts of buildings were more visible, thus obtaining panoramic photographs with better effects and improving the user experience. (2) When the streetscape taken contained low-rise and small shops or storefronts on both sides of the road, the height of the panoramic camera was reduced appropriately, and the panoramic photograph effect was viewed by various methods through the Ladybug CapPro software of the panoramic software. (20) (3) When the shooting scene comprised narrow roads (one-way streets, hutongs, etc.), the surrounding buildings were low; hence, the panoramic camera was lowered appropriately and switched to the top-view interface. This ensured an adequate view of the surroundings of the collection vehicle and provided good driving vision for the driver. A panoramic camera with an adjustable height determines the visible range on both sides of the vehicle and the surrounding ground objects suitable for shooting. Table 3 and Fig. 10 present the features of images taken by the real-time elevating panoramic camera and images taken at different heights of the camera, respectively. (21)

Conclusions
Panoramic maps and their applications have attracted considerable attention in the development of geographic information systems. To address the shortcomings of existing multisource road data panoramic acquisition vehicles, we developed a light, detachable elevating panoramic acquisition system. The system was optimized to rapidly acquire high-quality panoramic image data on both sides of the road as well as high-precision, real-time geographic position and attitude data of the vehicle carrier. Additionally, the platform integrates sensor equipment such as a panoramic camera, GNSS/INS integrated inertial navigation equipment, and telescopic Mast LT on an aluminum alloy main horizontal base. The entire main horizontal base system can be firmly fixed to the top of a vehicle carrier through vacuum suction cups, spherical pan-tilt heads, and universal joints. Compared with a traditional mobile road data acquisition vehicle, the proposed system has the following advantages.
(1) The novel use of vacuum suckers enhances the portability and flexibility of the panoramic acquisition platform and facilitates installation and disassembly. (2) The vehicle carrier requires no internal or external modifications, thereby maintaining the integrity of the vehicle and its normal functions. Thus, when the data collection is completed, the vehicle can resume its normal functions. (3) The panoramic acquisition platform and vehicle carrier are designed to be easily detachable.
If the vehicle fails during the data acquisition process, the panoramic acquisition platform can be quickly removed and installed on another vehicle carrier without affecting the subsequent data acquisition. (4) The panoramic camera is fixed with screws through structural members and bamboo-joint elevating columns; thus, the camera height can be remotely adjusted in real time during data acquisition. This avoids problems when passing under overhead road facilities and the distortion of the shooting effect of panoramic images. (5) To reduce the weight of the platform and facilitate the position calibration between the GNSS master-slave antenna that receives satellite signals and other instruments, an aluminum strip plate was designed and embedded into the main horizontal base of the platform. In a future work, we will further optimize the overall performance of the platform. For example, the synchronization mode will be improved and an odometer or wheel encoder will be added. The data processing flow will also be optimized by developing processing software with greater operational automation, efficiency, and speed.