Real-time Tracking of Human Motions and Adaptive Robot Path Planning for Assembly Cooperation

H. Berggren and F. MelvÄs. Master thesis, Chalmers University of Technology, June 2018.


Today, industries are working towards Industries 4.0 and smart factories, where artificial intelligence controls the robots and the production processes are adaptive to changes. This is typically used in industries where a high product variation is common. Furthermore, some robots in smart factories have to collaborate with human workers. A digital twin of the manufacturing process can then be created and used to check whether the robots can indeed operate safely in the presence of the human workers.

The aim of this project to enable human-robot collaboration in smart factories by developing a decision framework on top of some existing software packages. Firstly, we track the movement of a human worker with IMU sensors attached to the worker and a stereo camera mounted on the ceiling. Secondly, we update the poses of the digital twin of the human worker in order to replicate the human worker’s movements in the virtual environment. Thirdly, our decision framework continuously predicts the next movements of both the human worker and the robot. Whenever the human worker is in danger of collision with the robot, an existing software package uses the digital twins of the human worker and the robot to re-plan a collision-free path for the robot. Finally, the robot receives the new collision-free path from the software package and executes it in order to avoid collision with the human worker. Consequently, we can guarantee that the human worker can collaborate with the robot without being worried about being injured by the robot.


Photo credits: Nic McPhee