Simulation of Intelligent Robot in Boxing Exercises Based on Kinect Sensor

: In the continuous development process, robot technology has been widely used in aerospace, medical, education and service industries, and the relationship between robots and humans is getting closer.The improvement of robot intelligence is a process of continuously learning outside knowledge.Since the introduction of human-computer interaction technology in 1959, it has provided more and more technical support for human research robots.This paper first takes Taijiquan as an example, and carries out mechanical analysis and deconstruction of its standard Taiji pusher action, and obtains corresponding action parameterization information;Then, based on the human-computer interaction technology, the human body depth image and the skeleton structure are acquired through the Kinect sensor, and the human joint position data is recorded.The coordinate conversion is sent to the Nao robot, which realizes the imitation learning of the robot's push action on the Taiji.


Introduction
Robotics has been widely used in aerospace, medical, education, and service industries in the continuous development process.In this process, the robot is accessible to ordinary people.The robot is the product of computer automatic control, new materials, new energy, artificial intelligence and other high-tech crossfusion.At present, the research on robot technology has made great progress, and some artificial intelligence robots have a considerable degree of independent thinking and learning ability.However, for the industrialization of robots, there are still many technical problems that need to be solved, such as how to improve the learning ability of robots and how to better realize the interaction between humans and robots.The convenience and effectiveness of human-computer interaction has a very large space for improvement in robot applications [1]. A good human-computer interaction method will make the robot have high practical value.Human-computer interaction has always been the focus of robotics research.A successful interaction often relies on more factors, such as whether the interaction can be accepted by others;Interaction in a restricted environment can not work well.Therefore, in order to improve the recognition of human-computer interaction by the society, the primary problem to be solved is the adaptation of the robot to the characteristics of the human body.The movements and postures of some humanoid robots are important manifestations of the robot itself, and these also have a profound impact on the user.Therefore, controlling the robot's limb movement is a more challenging task, and it is also a relatively important topic in robot research.

Related work
In order to study the human-computer interaction problem of the supple mechanical arm in the inclusive environment, this topic selects the Taiji pusher with the supple interaction feature to study the representative of human-computer interaction [2].

Taiji pusher action analysis
Taiji Pusher is also called pusher and beater. It is a freehand exercise against Tai Chi.It is a confrontational part of the upper limbs and the trunk as the attacking part. It uses a technique such as "shed, push, squeeze, press, pick, butterfly, elbow, and lean" to achieve the advantage of borrowing power and causing the other body to lose balance. As a branch of traditional Chinese martial arts, Taiji Pusher has Chinese traditional characteristics.Taiji Pusher has been widely loved for its unique techniques and its fitness, entertainment and enjoyment. It has become one of the most widely spread projects in Chinese martial artsP [3].
At the beginning of the pacing push, the two sides stand opposite each other, with their knees slightly bent, the right foot in the right front of the body, and the left foot in the left rear. The standing position of the two feet is basically fixed during the pushing process.The two hands are in contact with each other. This is the "right hand" in the Taiji term. The right hand trajectory of both sides of the grading and flat pushing process basically constitutes a "circle" on a horizontal plane.(This circle mainly highlights the repeatability and regularity of the movement, not a circular shape in the strict sense).In the hands of the vertical circle, the right hand trajectory of the two sides roughly constitutes a "circle" on the vertical plane of the middle position between the two bodies.
Combined with the main and passive characteristics of both sides of the pusher process, the single-step pusher movement can be divided into four stages. The characteristics of each stage are shown in the Figure 1: Division of the stage of the pusher At the beginning of the push, stage 1 is the attacker's offensive stage, also called the active stage. The pusher uses the right palm and pushes forward and pushes it toward the chest of the pusher.In this stage, the pusher's arm actively pushes the other side to be the "press" in the Taiji terminology, and the pusher needs to maintain the resistance so as to avoid being pushed against his chest. This follows the opponent's movement but maintains a certain resistance [4]. The "shed" in the saying goes.Stage 2 is the conversion phase. At the beginning, the pusher is about to be pushed to his chest, and the pusher uses the waist to turn to the back of the body.At the same time, the palm of the pusher is gradually guided from the chest of the pusher, and the pusher is pushed forward by the pusher and pushed to the right side of the chest.The guiding force adopted by the pusher in this process is the rolling back called in the Taiji terminology. The pusher's hand conforms to the "hold", and the push is the pressing of the Taiji term.Phase 3 is the offensive/active phase of the pusher, which is the passive phase of the pusher.At this time, the pusher actively pushes the hand of the pusher forward to the front of the chest of the pusher.In this stage, the pusher uses the "press" force, while the pusher uses the "shed" to avoid being pushed to his chest.Phase 4 is the conversion phase of the promoter.At the beginning of this phase, the pusher is about to be pushed to the chest of the pusher [5]. At this time, the pusher will guide the hand of the pusher to his right side with the palm of his hand. In the stage 4, the pusher adopts the "will" and is pushed by the pusher. Squeeze "strength."At the end of this phase, both parties enter the next push cycle.

Biomechanical measurement of Taiji pusher motion
In order to deeply study the supple interaction process of Taiji Push Hands, it is necessary to study the characteristics of human body's movement, muscle strength and joints during the process of Taiji pushing.This requires a biomechanical measurement of the human body's Taiji pusher.The step-by-step flat pusher is the most basic and most important form of Taiji.In this paper, the movement of the singularity of the singularity of the singularity of the singularity of the singularity of the singularity and the mechanism of interaction between people and human being Sports biomechanical measurement techniques are divided into: kinematic parameter measurement techniques such as measurement time, displacement and derived kinematic parameters;Kinetic parameter measurement techniques such as measurement time, mass, and derived kinetic parameters;Biological measurement techniques, such as human morphology measurement methods, human body weight measurement methods, and myoelectric measurement methods.
In this paper, the biomechanical measurement of Taiji Push Hands is divided into two parts.
(1) Kinematics data The first part is the kinematic data of the human body during the measurement of the Taiji pusher.In order to measure the motion information of various parts of the whole body during the process of pushing the hand of the testee, it is necessary to use a motion capture system.Through research, the motion capture system is divided into three types: optical, inertial and mechanical.After the investigation, the Canadian BIOSYN company's Step 1 Step 2 Step 4 Step 3 Pusher B inertial motion capture system was used to measure the motion information of the human body during the Taiji pusher.
(2) Surface electromyography At present, most of the sports biomechanics studies surface EMG signals to study muscle activity during human exercise. In the Taiji pacing and round single pusher movement, the two sides reached the right hand and pushed the hand. Therefore, this paper uses surface electromyography equipment to measure the surface myoelectric signal of the right forearm of the human body during the pusher process. Through research, the surface electromyography equipment uses the MYO armband launched by the Canadian company ALMIC LABS [6]. The measurement test schematic is shown in the Figure 2:

Measurement data and analysis
The data measured by the FAB motion capture system includes the kinematic data of the whole body and the pressure data of the soles of the feet.As mentioned above, the Taiji pacing and flat single pusher movement is one cycle after another, and the measurement results are four cycles in which the data is relatively stable and the time is relatively advanced.This selection is to avoid the muscle fatigue of the test subject for a long time to ensure the reliability and accuracy of the data.The data of the joint motion data of the four cycles of the FAB measured by the FAB, that is, the joint motion angle data and the foot sole pressure measured by the foot insoles, were processed and edited, and the motion curve of the upper limb shoulder is shown in Figure 3. Shown [7].All solid lines in the figure are the joint pitch angle, and the uplift is positive;The dotted line is the angle of the joint roll, turning left to positive.

Figure 3 Curve of shoulder movement
The total time of 4 cycles of exercise is 14582.5 milliseconds, which is 13.08 seconds, and the average push cycle time is 3.27 seconds. In one cycle, the stages of 1, 2, 3, and 4 of the step-by-step flat pusher are: 0.5 seconds, 0.68 seconds, 0.99 seconds, and 1.1 seconds. The curve of the sole is shown in Figure 4.The left and right foot pressures are expressed as a percentage of the total body weight of the subject.   From the motion capture data and the plantar pressure curve, it can be seen that the three-dimensional kinematic angle and the plantar pressure of each joint show good periodicity. It can be seen from Fig. 6 that in the first three stages of the Taiji pusher pacing and flat pusher, the right foot pressure is greater than the left foot pressure. In summary, the upper right limb and the right lower limb of the whole exercise process have a larger amplitude [8].The movement of the right lower limb and the lower back (pelvis) causes the upper limbs of the human body to move, causing heavy sputum and changes to be the main factors causing changes in plantar pressure.

Methods
Robots usually have multiple joints. Different joint variables cause the robot to assume different states. The state of the robot's end depends on the value of each joint variable from the link chain.How to solve the robot end state according to each joint variable is the concern of the robot positive kinematics.

Robot position and attitude description
Analyzing the position and attitude of the robot in space can facilitate the establishment of a mathematical model of the robot to represent the state of the robot.Each link of the robot is considered a rigid body in a Cartesian coordinate system.To analyze the pose information of a rigid body in the reference coordinate system, you need to establish the coordinate system of the rigid body.The homogeneous matrix is used to describe the pose relationship between the rigid body coordinate system and the reference coordinate system [9]. The homogeneous matrix T is expressed as follows: (1) Where p represents the positional information of the origin of the rigid body coordinate system relative to the reference coordinate system.And n, o, a respectively represent the coordinates of the unit vector on the three axes of the rigid body coordinate system in the reference coordinate system [10].Together they form the rotation matrix R of the rigid body relative to the reference coordinate system: The rotation matrix R describes the attitude information of the rigid body in the reference coordinate system.The position vector p and the rotation matrix R together form a homogeneous pose matrix T of the rigid body with respect to the reference coordinate system.

Robot positive kinematics solution
In order to establish a positive kinematics model of the robot, it is necessary to use the pose relationship of the adjacent links of the robot.First analyze the case where the common reference coordinate system number is 0, the joint number is i, and the next joint number adjacent to the joint is .The pose matrix of the joint of the i joint in the reference coordinate system and the pose matrix of the joint in the coordinate system of the i joint are known [11].Then, the homogeneous pose matrix of joint i+1 in the common reference coordinate system can be obtained by the following formula: Therefore, we can call as the homogeneous transformation matrix of the joint coordinate system of the i-th joint to the coordinate system of the i+1th joint.Similarly, the pose matrix T can be understood as a homogeneous transformation matrix from the joint coordinate system to the rigid body coordinate system.For a joint chain of a robot, the reference coordinate system number is 0, and each rigid body number is in sequence.The pose transformation matrix of the adjacent two parts is denoted as [12].Therefore, the homogeneous pose matrix of the robot end relative to the reference coordinate system can be calculated by the following formula: This is the equation of robot kinematics.The idea of solving the positive motion of the robot is to obtain each pose transformation matrix according to each joint variable, and then use the robot kinematics equation to calculate the pose matrix of the robot end in the reference coordinates.The key to the forward kinematics of robots is the establishment of a joint coordinate system.In 1955, Denavit and Hatenberg proposed a matrix method for establishing a coordinate system for each member of the joint chain, referred to as the D-H method.It has been widely used in robot kinematics modeling and has become the standard method for establishing robot linkage coordinate system, as shown in Figure 5 [13].

Figure 5 DH method linkage coordinates
Therefore, the four parameters of describe the transformation process of adjacent link coordinate systems, which are called D-H parameters. The homogeneous transformation matrix of the adjacent link coordinate system is called the D-H transformation matrix, which can be calculated by these four parameters: For a rotating joint, the joint angle determines i, and the remaining three parameters are constant, so the homogeneous transformation matrix is determined by the joint angle.Generally, according to the joint link model of the robot, the coordinate system is established by the D-H method, and four parameters are determined, thereby obtaining a homogeneous transformation matrix of each group of adjacent joints [14].Then the robot kinematics equation can be used to obtain the robot's end pose matrix.

Nao Robot Motion Control System
Nao is a humanoid robot developed by Aldebaran of France. He is 58 cm tall and has a very good design. He is widely used in robotics labs around the world.As shown in Figure 6, the Nao robot has 25 degrees of freedom and is flexible.The Nao robot maintains excellent balance and precision during walking and movement.In addition, he is equipped with a variety of sensors including ultrasound, pressure, infrared, camera, touch and so on, which can be used not only for scientific research, but also an excellent entertainment and service robot [15].

Figure 6 NAO robot structure
The Nao robot has excellent motion control performance, and the ALMotion module is responsible for the motion control module of the robot.The ALMotion module primarily controls the joint strength, joint position, travel, and Cartesian coordinate space control of the robot.ALMotion works at 50hz. Whenever a user calls the API to request a motion, the corresponding "motion task" is created to respond to the user's request.The coordinate space of the Nao robot is defined as: the X axis is facing the front of Nao, the Y axis is from right to left, and the Z axis is the vertical positive direction.
DCM (Devices Communication Manager) is a software module of the Naoqi system.It controls the data communication (main board, sensors, actuators, etc.) of all robotic devices, and connects the robot's superstructure (CPU layer) and the underlying structure (device layer).When any module wants to send a command to the executor, the module first sends the updated command to DCM using the timed command command.Because other modules cannot directly change the DCM data stored in ALMemory, they must first send a request to DCM.The DCM then modifies the actuator data stored in ALMemory [16]. Two 2D cameras to recognize shapes, objects and even people.
The Nao robot operating system defines timed command={target value, time}.Each time a timed command is sent to a child device, DCM stores all commands sent.These commands are then processed periodically.The difference algorithm is used when DCM processes commands.So each time a motion control command is sent to the joint.It is first sent to the DCM, and then the control of the joint is obtained by the difference algorithm.The joint strength value of the Nao robot is defined as 0 to 1.When the value is 0.0, the joint is in the free state, and when the value is 1.0, the joint moves to the specified position with the maximum torque.Joint strength control is shown in Table 1. Proxy.stiffiness interpolation(p-name,p-stiffnesslists,p-timelist); The joint control of the robot can control only one joint or multiple joints at the same time.Each joint of the Nao robot has a specific name and a corresponding range of motion angles.Joint control is shown in Table 2. Proxy.angle.interpolation(name,anglelist,time,isAbsolute);

Robotic Taiji Pushing Action Imitation Based on Somatosensory Human-Computer Interaction
This section builds a robotic attitude simulation system using Microsoft's somatosensory device Kinect and the biped humanoid robot developed by Aldebaran Robotics of France.Analyze the characteristics of Kinect bone information and NAO robot joint model, and use the analytical method to transform the human body posture to realize the imitation of the Taiji pusher action by the NAO robot.

Extracting human skeleton posture data based on Kinect
Kinect bone tracking technology can distinguish the various parts of the human skeleton, and at the same time can process the coordinate information of each joint of the human body by processing the depth data.The robot motion simulation first needs to acquire the human body posture and acquire the three-dimensional information of each joint for coordinate conversion.
Bone is a hard organ that makes up vertebrate. Its function is to exercise, support and protect the body. The body has 206 bones.Skeletal tracking is the core technology of Kinect, as shown in Figure 7.A small black dot represents a joint name of the human body, and the human skeleton can be represented by 20 joint structures.Kinect can represent the human skeleton structure by capturing 20 joints, and the position of each joint point is represented by the (x, y, z) coordinate values [17].The Kinect sensor is at the origin of the coordinate system and the entire coordinate space conforms to the right-handed spiral rule.The y coordinate axis is parallel to the direction of the Kinect placement, the x coordinate axis is perpendicular to the orientation of the Kinect, and the z coordinate axis is parallel to the direction of the Kinect sensor.Therefore, when human bones appear in the field of Kinect's field of view, the Kinect sensor can calculate the coordinate space of each joint.

Action Simulation Control Process
This paper builds a motion simulation system to complete the NAO robot to imitate human motion in real time.Firstly, the human skeleton information is extracted by the Kinect sensory sensor, and the human body posture represented by the joint position is converted into the robot posture represented by the joint angle by the analytic method, thereby obtaining the posture sequence.The pose sequence is then processed to eliminate disturbances and limit motion speed.In order to make the robot maintain its stability while imitating the movement, it is necessary to determine the support of the feet and adjust the direction and height of the soles.The stability of the robot is determined according to the COM projection method, and the unstable attitude is corrected to control the joint angle of the NAO robot.The general control flow of the system is shown in Figure 8.

Imitation experiment of Taiji pusher action
The computer is connected to Kinect via USB, and uses the depth information it collects to detect the instructor and extract the skeleton information composed of the joint position of the teaching staff.Then the computer processes the human body posture according to the joint model of NAO, and obtains the control amount of each joint of the robot.The robot's motion is monitored and controlled by Wi-Fi, so that the NAO robot can imitate the human body posture online.The flow chart is shown in Figure 9. Consider the above proposed pose transformation algorithm on a system consisting of NAO robot, Kinect, and PC.From the perspective of analytic geometry, the algorithm transforms the target pose of the NAO robot according to the human body posture.Due to the various differences between the robot and the human body, the robot's whole body mimicking movement will inevitably involve the maintenance of the stability of the robot.The above algorithm is verified to avoid robot stability problems.The experiment is divided into two parts: the upper body posture imitation and the leg posture imitation.
.The pose in the figure relates to the rotation of all the controlled joints of the robot hand. For each human posture in the figure, the robot can correctly imitate.The experimental results show that the attitude transformation algorithm in this paper can accurately calculate the angle of all controlled joints.And control the NAO robot to obtain a robot posture similar to the human body, that is, the robot can accurately imitate the human body's posture, thus verifying the effectiveness of the human body posture simulation algorithm.

Results and Discussion
Under the great attention and vigorous promotion at home and abroad, robot technology is developing rapidly.Human body mimicry uses natural human-computer interaction to allow robots to quickly learn human action behaviors, which is conducive to robotic operations and better serve humans.This paper first takes Taijiquan as an example, and analyzes and deconstructs its standard Taiji pusher action, and obtains corresponding action parameterization information.Then, using the Microsoft Somatosensory Sensor Kinect, the humanoid robot NAO and the PC, which can simultaneously acquire color images and depth information, a humanoid robot attitude motion simulation system is constructed.Then based on the Kinect human skeleton information and the characteristics of the NAO robot joint model, a transformation method based on the analytical method of Kinect human posture to NAO robot attitude is designed and implemented. Under the condition of not considering the stability of humanoid robot, the imitation experiment of human body training posture is carried out. The experimental results show that the attitude transformation algorithm of this paper can make a similar posture to the human body.

Availability of data and material
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests
We have no competing interests   Legend: Figure 3 show Curve of shoulder movement Step 1 Step 2 Step 4 Step 3  Legend: Figure 4 show Curve of plantar pressure   7 touch sensors located on the head, hands and feet, sonars and an inertial unit to perceive his environment and locate himself in space. 4 directional microphones and speakers to interact with humans.
Two 2D cameras to recognize shapes, objects and even people.