Reinforcement Learning Control of a Forestry Crane Manipulator
J. Andersson, K. Bodin, D. Lindmark, M. Servin, and E. Wallin, Reinforcement Learning Control of a Forestry Crane Manipulator. arXiv:2103.02315. Submitted manuscript (2021). pdf
Forestry machines are heavy vehicles performing complex manipulation tasks in unstructured production forest environments. Together with the complex dynamics of the on-board hydraulically actuated cranes, the rough forest terrains have posed a particular challenge in forestry automation. In this study, the feasibility of applying reinforcement learning control to forestry crane manipulators is investigated in a simulated environment. Our results show that it is possible to learn successful actuator-space control policies for energy efficient log grasping by invoking a simple curriculum in a deep reinforcement learning setup. Given the pose of the selected logs, our best control policy reaches a grasping success rate of 97%. Including an energy-optimization goal in the reward function, the energy consumption is significantly reduced compared to control policies learned without incentive for energy optimization, while the increase in cycle time is marginal. The energy-optimization effects can be observed in the overall smoother motion and acceleration profiles during crane manipulation.

Supplementary video

This work has in part been supported by Mistra Digital Forest (Grant DIA 2017/14 #6). Extractor AB has kindly provided 3D models for the Xt28 forwarder.

Algoryx Simulation AB