Reinforcement learning : Biped locomotion using a CPG contoller

Keyword

Biped walking, reinforcement learning, central pattern generators(CPGs)

abstract

Rhythmic locomotion such as walking, flying and swimming, are fundamental to animals' locomotion. Neurobiological studies have revealed that these rhythmic locomotion are controlled by neural circuits called central pattern generators(CPGs). Motivated by such animals' control mechanisms, motor control mechanisms using a CPG controller have been studied. As an autonomous learning framework for a CPG controller, we proposed a new reinforcement learning (RL) method called the CPG-actor-critic method, and applied our RL method to an automatic control problem by a biped robot simulator. The CPG controller that allows a biped robot walk stably, was obtained by our method.

Images

Animals' control mechanisms for rhythmic movements

Control scheme using a CPG controller

Movies

Animation of biped locomotion

References

Nakamura, Y., Sato, M., & Ishii, S. Reinforcement Learning for Rhythmic Movements Using a Neural Oscillator network. Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J87-D-II(3), pp.893--902, (2004) in Japanese.
Mori, T., Nakamura, Y., Sato, M.,& Ishii, S. Reinforcement Learning for CPG-driven Biped Robot Nineteenth National Conference on Artificial Intelligence (AAAI2004), pp.623-630.