ResidualAssist

ResidualAssist: Residual Reinforcement Learning in Simulation for Leg-Decoupled Exoskeleton Control in Walking, Running, and Asymmetric Gait

Paper PDF

Learning progression

Initial random motion in the musculoskeletal simulation
0 steps
Random motion
Early training with repeated falls
2k steps
Repeated falls
Intermediate unnatural walking motion
5k steps
Unnatural gait
Natural walking motion after training
10k steps
Natural gait
Muscle-actuated walking visualization
10k steps
Natural gait
Muscle-actuated walking in simulation Participant walking on a treadmill while wearing the hip exoskeleton
A policy trained in musculoskeletal simulation is deployed on a physical hip exoskeleton without retuning across the tested conditions.

Abstract

ResidualAssist combines a simple velocity-based torque prior with a reinforcement learning policy that learns state-dependent residual corrections.

Learning exoskeleton assistive torque directly through reinforcement learning remains challenging because assistance is highly phase-sensitive, whereas noisy biomechanical rewards provide indirect guidance for torque timing and amplitude. Rather than learning an entire torque profile from scratch, the proposed framework begins with an interpretable nominal assistance pattern and learns how to adjust it.

A parameter-shared, leg-decoupled policy generates bilateral torque from each leg’s local state. The study evaluates this structure in a unilateral-load-induced asymmetric gait condition, after simulation training and deployment on the physical device.

01

Prior-guided residual learning

A velocity-proportional prior supplies nominal assistance while the learned policy corrects timing and amplitude.

02

Leg-decoupled control

The same actor processes each leg separately using only that leg’s local observation.

03

Simulation-to-hardware evaluation

The controller is tested in walking, running, and a unilateral-load condition with six participants.

Method

The training procedure separates the acquisition of natural, muscle-actuated locomotion from exoskeleton assistance learning.

Stage 1

Muscle-actuated locomotion learning

The musculoskeletal human controller is trained through imitation learning to reproduce the reference gait before exoskeleton assistance is introduced. The resulting controller provides the starting point for the second training stage.

Simulation frequency
600 Hz
Control frequency
100 Hz
Musculoskeletal model
284 musculotendon units
Stage 1 human musculoskeletal simulation and imitation learning pipeline
Stage 1 training pipeline. The actor policy and muscle network map human state to muscle activation under imitation learning.
Stage 2

Residual exoskeleton assistance learning

The pretrained human controller continues to update under the adaptation objective while the exoskeleton policy learns residual torque corrections around a velocity-based prior.

Stage 2 human adaptation and residual exoskeleton torque learning pipeline
Stage 2 training pipeline. The residual policy and velocity-based prior produce bilateral assistive torque while the simulated human controller continues to adapt.
01

Residual torque formulation

The prior maps filtered hip angular velocity to nominal assistive torque. A recurrent policy outputs a residual command, and the combined torque is clipped and low-pass filtered before it reaches the simulated or physical exoskeleton.

02

Leg-decoupled control

Both legs share actor parameters, while each torque command depends only on the corresponding leg’s local angle, velocity, and previous torque.

Learning progression

The first stage develops stable muscle-actuated locomotion before the exoskeleton policy is introduced.

Initial random motion in the musculoskeletal simulation
0 steps
Random motion
Early training with repeated falls
2k steps
Repeated falls
Intermediate unnatural walking motion
5k steps
Unnatural gait
Natural walking motion after training
10k steps
Natural gait
Muscle-actuated walking visualization
10k steps
Natural gait

Results

The reported reductions below refer to the simulation condition with active assistance and human adaptation, compared with the no-exoskeleton baseline.

13.2%lower RMS biological hip moment in simulation
10.2%lower RMS hip muscle-group activation in simulation
R ≥ 0.92measured versus simulated torque at 1.25 m/s
≈90%positive power ratio on both legs with leg-decoupled control under the tested asymmetric condition
Hip angle, biological moment, and assistive torque across the gait cycle
Assistive torque timing. The learned residual shifts the torque peaks relative to the baseline biological hip moment.
Simulated hip moment and muscle activation results across assistance conditions
Simulation comparison. Human adaptation further reduces the simulated biological hip moment and muscle activation proxy.

Hardware deployment

The device uses a quasi-direct-drive hip exoskeleton, wearable IMUs, a Raspberry Pi 5, and a Teensy 4.1 control stack.

Hip exoskeleton hardware and controller deployment stack
Deployment system. Mechanical platform, electronics, sensing, and the tested locomotion conditions.
Simulated and measured torque profile comparison
Sim-to-real consistency. Raw measured torque profiles reach R ≥ 0.92 across both legs at 1.25 m/s.
Assistive torque and power across walking and running speeds
Cross-speed evaluation. One policy is deployed without retuning from 0.75 m/s walking to 2.5 m/s running.
Torque and power profiles under unilateral-load-induced asymmetric gait

Unilateral-load evaluation

Under the tested 2 kg right-ankle load, the leg-decoupled controller produced more repeatable torque and power profiles across gait cycles than the leg-coupled controller.

Positive power ratio remained approximately 90% on both legs with leg-decoupled control, compared with 85% and 76% on the unloaded and loaded legs with leg-coupled control.

Scope. This controlled loading protocol does not represent pathological gait. The reported advantage is limited to the tested unilateral-load condition in healthy participants.

Paper

The complete eight-page manuscript is embedded below and can also be opened as a standalone PDF.

Open or download the PDF

Your browser cannot display embedded PDFs. Open the paper directly.

Citation

@article{residualassist2026,
  title = {ResidualAssist: Residual Reinforcement Learning in Simulation for
           Leg-Decoupled Exoskeleton Control in Walking, Running, and Asymmetric Gait},
  year  = {2026}
}