Asymmetric Bimanual Control of Dual-arm Serial Manipulator for Robot-assisted Minimally Invasive Surgeries

1Department of Electronics, Information and Bioengineering, Politecnico di Milano, Via Giuseppe Colombo, 40, Milan 20133, Italy 2State Key Laboratory of High Performance Complex Manufacturing, Central South University, Changsha 410083, China 3State Key Laboratory of Intelligent Control and Decision of Complex Systems, Beijing Institute of Technology, Beijing 100081, China 4BioMEx Center & KTH Mechanics, KTH Royal Institute of Technology, SE-100 44 Stockholm, Sweden 5Department of Informatics, Technical University of Munich, Munich 85748, Germany


Introduction
Laparoscopy requires a small incision in the abdominal wall to allow the insertion of a trocar and surgical tools. Robotic implementation of such minimally invasive surgery (MIS) techniques is expected to improve the control and precision of the surgical tools used in interventions while reducing trauma to patients. (1)(2)(3) However, specialized surgical robots are expensive, which limits their acquisition and use in hospitals.
Industrial robots with redundancy have meanwhile been successfully adopted and further developed for precise automation processes in the last few decades. Their lower cost than specialized surgical robots has increased interest in their application in the medical field, particularly in MIS. Their redundancy, such as more DoFs than required for the executed task, provides benefits over nonredundant robots, including the facilitation of human-like behavior, (2,4) enhanced manipulability, (3) and obstacle avoidance. (5,6) In robot-assisted MIS, the motion of the surgical tool is constrained to a point, referred to as the remote center of motion (RCM), where the surgical tool crosses the abdominal wall, allowing only translational movements around its axis. Whereas a mechanical implementation is generally considered safer but requires bulky, expensive structures and calibration procedures, a programmable RCM whose movement is restricted by the control algorithm is cheaper and more flexible and is therefore the preferable option. (7,8) In our previous work, we achieved RCM control with a null space controller for a redundant serial robot. (9,10) In actual surgery, the RCM moves continuously owing to the respiration of the patient. To ensure accuracy, the controller has to compensate robustly for these movements to prevent bending of the surgical tool by the resulting forces. (11,12) Such deformation of the tool would not only directly distort its trajectory but also interfere with its readjustment since robotic systems generally obtain the position of the end effector (EE) from the positions of the manipulator's joints.
In this work, we simulate a moving RCM using clinical respiratory data. (13) Neural networks, particularly radial basis function neural networks (RBFNNs), have proven to be a powerful tool for robustly implementing model predictive control mechanisms, outperforming traditional controllers even for linear applications where proportional-integral-derivative (PID) controllers have been established as a benchmark. (14) With a Gaussian activation function, an RBFNN (11) can smoothly approximate any continuous function. Thus, it is suitable as an adaptive controller for compensating the uncertain disturbances caused by the respiratory motion, and therefore can be employed to improve the control of a surgical tool.
The aim of our research is to simulate a programmable RCM in a scenario close to the clinical scenario by implementing bimanual control (15,16) with two asymmetric KUKA LWR4+ robotic manipulators for MIS and utilizing an RBFNN to compensate for the disturbance from the respiratory movement. An evaluation is performed on surgically relevant trajectories for intraoperative suturing and knot tying.
The contributions of this paper are as follows. An RBFNN method is used to compensate for the disturbance from the respiratory movement under a constrained RCM. The experiment is verified on surgically relevant trajectories for intraoperative suturing and knot tying.
The paper is organized as follows. Section 2 describes the fundamentals and control methodology. In Sect. 3, the performance of the proposed control scheme is validated in a simulation with clinical data. Conclusions are drawn in Sect. 4.

Methodology
In this section, we present the clinical workspace and the essential mathematical concepts used.

Clinical aspects
During the MIS of the torso, the breathing mechanism generates shifts in the abdominal wall, as shown in Fig. 1, and modifies the RCM position, causing disturbances in the system, which in turn lead to possible harm at the trocar. Furthermore, EE errors can result in insufficient accuracy of the suturing and knot-tying procedures. (12,17,18)

Null space control
The defined workspace and the RCM, EE, and wrist are depicted in Fig. 1. Our implementation is based on the programmable RCM introduced by Khatib (19) and controlled with two Jacobian matrices and an appropriate null matrix. (7,20) The desired Cartesian velocity for the subsequent time step of the EE can be described as where EE x  denotes the desired Cartesian velocity, x EE denotes the current EE position, and x des is the next time of the EE position. T is the duration of one-time step. Additionally, the desired velocity of the robot wrist wrist x  can be defined as subjected to where x wrist represents the current position of the wrist, v tool denotes the vector describing the surgical tool position, x RCM is the position of the RCM, and L is the length of the surgical tool.
To control the robotic manipulator, the desired joint velocities EE q  and wrist speed wrist q  at the EE can be presented as where J EE and J wrist are the corresponding 3 × 7 Jacobian matrices, respectively. With the two matrices controlling the Cartesian positions of the EE and wrist, the rotational DoFs of the tool assume a configuration that makes the tool pass through the RCM.
The desired joint position des q  can be defined as subjected to 7 wrist wrist where I 7 denotes the seven-dimensional identity matrix, and J wrist denotes the pseudoinverse. (5) The desired overall joint velocity can be given by All nonredundant components are removed from wrist q  to maintain the precise trajectory of the EE.

RBFNN
An RBFNN (21,22) is a type of feedforward neural network with an input layer, a hidden layer, and an output layer. The neurons of the hidden layer have Gaussian radial basis activation functions with adaptable center and width; hence, there are no weights of the hidden layer. The weights of the output layer are updated continuously with learning rate α. In the output layer, the weighted radial basis functions are summed to obtain one smooth continuous function, allowing it to adapt, and thus robustly compensate for unknown dynamics, nonlinearities, and disturbances. (23,24) The activation function h i of the ith neuron in the hidden layer is given by where x EE,v is the current Cartesian position of the EE, c is the vector containing the centers, and b is the vector of the widths of the Gaussian functions in the hidden layer.
The update of the weights Δw ij of the output layer is given by where e j is the error between the desired and actual positions of the EE, α = 0.4 is the learning rate, and β = 0.02 is the weighting of the previous update δw ij . The RBFNN control is implemented in the manner depicted in Fig. 2. An external disturbance simulating the respiratory movement is introduced, where clinical data is used. The RBFNN ensures that the EE of the surgical tool does not deviate significantly from its trajectory.

Experimental Validation
To evaluate the performance of the proposed control approach, simulations were conducted with the dual-arm robot shown in Fig. 3. The robot arms were set to autonomously follow surgical trajectories with five different shapes, a straight line, an angle, a sine curve, a semicircle, and a helix, to demonstrate and evaluate its versatility. (25,26) These shapes are directly related to the usual trajectory performed in MIS suturing and knot-tying techniques, (27)(28)(29) as shown in Fig. 4. The length or diameter of the trajectory, depending on the shape, was set to 5 mm. (30) The respiratory movements were based on clinical data recorded by Shafiq and Veluvolu. (13) The registered movements of two markers in the abdominal region were picked as moving RCMs. Because of the low sampling frequency, spline interpolation was used to obtain the missing values. (31)(32)(33) The control and robustness of the proposed system were evaluated considering the Euclidean norm tracking error of the EE, which measures the Euclidean distance between the optimal RCM position x RCM at the incision and the surgical tool, where v tool is the vector describing the surgical tool position. From the EE error, depicted for the angle trajectory in Fig. 5(a), it was concluded that the RBFNN considerably improves the accuracy of the following trajectory, and at the same time, the RCM constraint by a small amount. The boxplots in Fig. 6 confirmed that the neural network control reduced the EE error for the other trajectories, decreasing the median of the error and shortening the whiskers. For more complex and nonlinear trajectories such as the helix, the errors of the implementations both without and with the RBFNN are larger than those for the simpler ones. However, the RCM error for the helix also decreased, suggesting that the proposed controller will enhance accuracy by compensating disturbances.

Conclusions
This article mainly centers on the asymmetric bimanual control of a dual-arm serial robot with two RCM constraints for MIS. To evaluate the developed control scheme, some simulations are carried out, indicating that the proposed method can achieve advantageous performance for the accurate control mechanism of dual-arm serial robots. Even for a moving RCM, the RBFNN can robustly keep the EE error sufficiently small to meet the precision requirements of medical surgery with 1 mm at the EE. However, further improvements are required. Since our simulations did not consider physical effects such as inertia or force feedback, (34) the next step should be to validate the performance of our implemented algorithm on actual KUKA LWR4+ robots and a phantom with force feedback. Furthermore, humanrobot physical interaction in the surgical environment will be implemented to assist the motion of the surgeon. (35) Yunus