Article

# Bridging Reinforcement Learning and Iterative Learning Control: Autonomous Motion Learning for Unknown, Nonlinear Dynamics

doi: 10.3389/frobt.2022.793512.

eCollection 2022.

Affiliations

Item in Clipboard

Michael Meindl et al.

Front Robot AI.

.

## Abstract

This work addresses the problem of reference tracking in autonomously learning robots with unknown, nonlinear dynamics. Existing solutions require model information or extensive parameter tuning, and have rarely been validated in real-world experiments. We propose a learning control scheme that learns to approximate the unknown dynamics by a Gaussian Process (GP), which is used to optimize and apply a feedforward control input on each trial. Unlike existing approaches, the proposed method neither requires knowledge of the system states and their dynamics nor knowledge of an effective feedback control structure. All algorithm parameters are chosen automatically, i.e. the learning method works plug and play. The proposed method is validated in extensive simulations and real-world experiments. In contrast to most existing work, we study learning dynamics for more than one motion task as well as the robustness of performance across a large range of learning parameters. The method’s plug and play applicability is demonstrated by experiments with a balancing robot, in which the proposed method rapidly learns to track the desired output. Due to its model-agnostic and plug and play properties, the proposed method is expected to have high potential for application to a large class of reference tracking problems in systems with unknown, nonlinear dynamics.

Keywords:

Gaussian processes (GP); autonomous systems; iterative learning control; nonlinear systems; reinforcement learning; robot learning.

### Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

### References

1. Ahn H.-S., Chen Y., Moore K. L. (2007). Iterative Learning Control: Brief Survey and Categorization. IEEE Trans. Syst. Man. Cybern. C 37, 1099–1121. 10.1109/tsmcc.2007.905759

DOI

1. Ai Q., Ke D., Zuo J., Meng W., Liu Q., Zhang Z., et al. (2020). High-Order Model-free Adaptive Iterative Learning Control of Pneumatic Artificial Muscle with Enhanced Convergence. IEEE Trans. Ind. Electron. 67, 9548–9559. 10.1109/TIE.2019.2952810

DOI

1. Amann N., Owens D. H., Rogers E. (1996). Iterative Learning Control for Discrete-Time Systems with Exponential Rate of Convergence. IEE Proc. Control Theory Appl. 143, 217–224. 10.1049/ip-cta:19960244

DOI

1. Apgar T., Clary P., Green K., Fern A., Hurst J. (2018). “Fast Online Trajectory Optimization for the Bipedal Robot Cassie,” in Robotics: Science And Systems XIV (Robotics: Science and Systems Foundation). 10.15607/rss.2018.xiv.054

DOI

1. Arimoto S., Kawamura S., Miyazaki F. (1984). Bettering Operation of Robots by Learning. J. Robot. Syst. 1, 123–140. 10.1002/rob.4620010203

DOI