Skip to content

RL for rocket engine control

A literature review, a domain primer and a runnable CPU codebase for deep reinforcement learning control of liquid-propellant rocket engines. It is anchored on DLR's LUMEN Control Challenge: two turbine valves of a 25 kN LOX/methane expander-bleed engine must make chamber pressure and mixture ratio follow a reference, with the engine's limits respected.

2 × 2turbine valves in; chamber pressure and mixture ratio out (benchmark design)
1.3 %mean tracking error of DLR's SAC controller on the real engine (thesis Table 6.2)
0.5 s vs 20 ssettling after a TOV step against a TFV step: one fast valve, one slow (Table 4.7)
2.61 → 0.97 %chamber-pressure error, tuned PI against SAC at 20 Hz without preview, as in the challenge, on this site's surrogate engine (baselines)

Built on

The work this project stands on

This is an independent study. Its foundation is the research of DLR's Institute of Space Propulsion in Lampoldshausen:

  • The benchmark. The LUMEN Control Challenge, designed by Vincent Bareiß, Kai Dresia, Günther Waxenegger-Wilfing and colleagues (AI4Aerospace 2025; RL4AA'25).
  • The method and most of the numbers. Kai Dresia's doctoral thesis, Rocket Engine Control with Deep Reinforcement Learning (RWTH Aachen, 2025; DLR-FB-2025-16). It covers the LUMEN model, SAC against MPC, and the first RL controller to run the engine in a hot-fire test.
  • The engine. LUMEN, designed, built and fired by DLR (Deeken et al. 2021; Traudt et al. 2022, 2024). Also its validated simulation model (Kurudzija et al. 2025) and the safety work around its RL hot-fire tests (Dauer et al. 2025).

Every statement on this site about LUMEN, its model or DLR's controllers cites one of these works, usually with a page number. The surrogate engine, its baselines and the Engine Lab are this project's own; they are not DLR's simulator, and their numbers say nothing quantitative about the challenge.

Cite the foundation (BibTeX) 
@phdthesis{dresia2025thesis,
  author = {Dresia, Kai},
  title  = {Rocket Engine Control with Deep Reinforcement Learning},
  school = {RWTH Aachen University},
  year   = {2025},
  note   = {DLR-Forschungsbericht DLR-FB-2025-16},
  doi    = {10.18154/RWTH-2025-05076},
  url    = {https://elib.dlr.de/219040/}
}
@inproceedings{bareiss2025benchmark,
  author    = {Barei{\ss}, Vincent and Dresia, Kai and Waxenegger-Wilfing, G{\"u}nther},
  title     = {Development of a Benchmark for Deep Reinforcement Learning Based Control of Liquid Propellant Rocket Engines},
  booktitle = {5th AI4Aerospace Workshop (DLR--ONERA)},
  address   = {Toulouse},
  year      = {2025},
  pages     = {55--56},
  url       = {https://w3.onera.fr/ailab/sites/default/files/2025-06/abstractsAI4A5thworkshop_external.pdf}
}
@misc{dresia2025rl4aa,
  author       = {Dresia, Kai and Waxenegger-Wilfing, G{\"u}nther and Dauer, Jonas and Hirlaender, Simon and Barei{\ss}, Vincent},
  title        = {A Benchmark for Deep Reinforcement Learning-Based Control of Liquid-Propellant Rocket Engines},
  howpublished = {RL4AA'25 workshop, DESY Hamburg},
  year         = {2025},
  url          = {https://indico.kit.edu/event/4216/contributions/19241}
}

Status on 2026-10-08: the challenge is expected around the end of October 2026

DLR presented the LUMEN Control Challenge at the RL Bootcamp 2026 on 17 September, promising "Public access to generalized LUMEN simulator model" and "Automatic evaluation of your control performance". The organisers now expect to share a repository around the end of October 2026 (personal communication). They have confirmed:

  • control at 20 Hz;
  • episodes of 10–100 s;
  • no preview of future set points in the observation;
  • fine-tuning data from simulation with changed model parameters, not from hot fire.

Until the release, this site runs a surrogate: a reduced LUMEN-like engine calibrated to the gains, settling times and overshoot DLR publishes for its model (model card). The primer's widgets, the Engine Lab and the baselines all run on it. The earlier search for a public release is logged in §7.1.

Five things to know before reading further

  1. The task is a coupled 2×2 tracking problem. Two turbine valves (TFV, TOV) set chamber pressure, i.e. thrust, and mixture ratio, i.e. combustion temperature. Both valves move both outputs (The engine as a control system). Seven benchmark test cases add unknown references, parametric model error, slow drift and faults (§1).
  2. One valve is fast and one is slow. The oxidiser side settles in half a second. The fuel side settles over 5–24 s, because the fuel is also the coolant whose heat drives both turbines (Fast chamber, slow heat).
  3. RL has already flown on this engine, briefly. DLR's SAC controller tracked four variables at a mean error of 1.3 % over more than 16 s of hot fire, zero-shot from simulation. In simulation the same controller reaches 0.4 % (thesis Table 6.2). The first hardware deployment failed on a valve-model error (Valves are the actuators).
  4. In simulation, RL beat MPC on speed and constraints; MPC won on fuel. On a shared 5-valve task SAC reached 0.3 % / 0.1 % tracking error against MPC's 1.4 % / 0.6 %, and settled in 0.7 s against 3.0 s. MPC used 0.38 % less propellant (thesis p. 93–97).

  5. On this site's surrogate, no controller wins every test case. At 20 Hz without preview, as in the challenge, SAC tracks chamber pressure to 1.0 % and PPO to 1.2 %, against 2.6 % for a tuned PI, with almost no time over a limit. Preview, which the challenge excludes, halved that error for the earlier 10 Hz agents. But on the challenge's test cases in miniature the picture splits (test cases):

    • the networks win nominal tracking;
    • when the engine changes, drifts or wears a bearing, they lose the mixture ratio and a PI does better;
    • when a valve sticks, everyone loses the mixture ratio.

    These are surrogate numbers, not LUMEN's. They show where to look: robustness and faults, which is what the challenge's test cases 3–7 probe. The Engine Lab runs the whole matrix live in your browser.

The sources, curated

The papers this site leans on most, with the one result or sentence that makes each matter. Every one is open to read. Full details and verification status are in §7 References.

The challenge

  • The benchmark design

    Bareiß, Dresia, Waxenegger-Wilfing · AI4Aerospace workshop, Toulouse, 2025

    Development of a Benchmark for Deep Reinforcement Learning Based Control of Liquid Propellant Rocket Engines

    "the resulting LUMEN control task is a 2×2 MIMO problem"

    Seven test cases, from nominal tracking to faults without a detection signal, each a Gym environment.

    Extended abstract, pp. 55–56 elib record

  • The announcement

    Dresia, Waxenegger-Wilfing, Dauer, Hirlaender, Bareiß · RL4AA'25, DESY Hamburg

    A Benchmark for Deep Reinforcement Learning-Based Control of Liquid-Propellant Rocket Engines

    "will be made freely accessible to the RL community"

    Simulation software calibrated on experiments, a fine-tuning dataset, and sensor and system faults.

    Abstract Indico

  • The pitch

    DLR Institute of Space Propulsion · RL Bootcamp 2026, Salzburg, 17 Sep 2026

    LUMEN Control Challenge slide and talk

    "Public access to generalized LUMEN simulator model"

    PPO and SAC demos in simulation, and about 15 s of closed loop on the real engine.

    Livestream at 7:18:00

The research it rests on

  • The doctoral thesis

    Kai Dresia · RWTH Aachen, 2025 · DLR-FB-2025-16

    Rocket Engine Control with Deep Reinforcement Learning

    1.3 % mean tracking error on the real engine, zero-shot from simulation, against 0.4 % in simulation (Table 6.2). Also SAC against MPC, and the LUMEN model's step responses.

    PDF RWTH record

  • Safety and the first hot-fire tests

    Dauer, Dresia, Deeken, Waxenegger-Wilfing · EUCASS 2025

    How to Guarantee Safety for Neural Network based Rocket Engine Controllers

    "likely attributable to an inaccurate model of the valve dynamics"

    Why the first RL hot-fire test oscillated, and an out-of-capability score to catch it.

    Paper

  • RL for engine transients

    Waxenegger-Wilfing, Dresia, Deeken, Oschwald · IEEE TAES 57(5), 2021

    A Reinforcement Learning Approach for Transient Control of Liquid Rocket Engines

    An early deep-RL engine controller from the same group: TD3 on a gas-generator engine model wrapped as a Gym environment, compared with PID and open loop.

    arXiv:2006.11108 doi

The engine and its model

  • The demonstrator

    Deeken, Waxenegger-Wilfing, Oschwald, Schlechtriem · Space Propulsion 2020+1

    LUMEN Demonstrator – Project Overview

    The project's goals, the operating envelope and nominal flows, the turbopumps, the laser igniter, and a nozzle extension designed for no flow separation at 60 bar.

    Paper

  • The hot-fire campaign

    Traudt et al. · IAC 2024, Milan

    LUMEN: A Versatile Test Bed for Rocket Engine Components: Hot-Fire Test Results

    Nominal 60 bar at mixture ratio 3.4, throttled from 58 to 133 %, with four valves in closed loop at 20 Hz. Its photos are the model for the Lab's test stand.

    Paper

  • The model, validated on hot fire

    Kurudzija, Dresia, Bareiß et al. · EUCASS 2025

    Validation of the LUMEN EcosimPro Model with Hot-Fire Test Data

    Chamber pressure predicted to 2.4 % and mixture ratio to 3.5 % mean absolute error over a test run (Table 2).

    Paper Preprint on the transient model (2026)

How to read this site

The left navigation tells one story, in this order. Each page's previous and next buttons follow it.

  • 1 · The control problem

    The task as an MDP, the seven test cases, and what "solved" should mean.

    The MDP

  • 2 · The physics

    Nine short chapters from thrust and mixture ratio to the reward, each with a widget. They build up to the Engine Lab and end with real hot fire and an equation sheet.

    The primer · The Engine Lab

  • 3 · Timeline 2018–2026

    Who did what, in date order: the engine line and the adjacent fields.

    The field

  • 4 · Designs and results

    Published controllers side by side: SAC, TD3, MPC, PI and curriculum PPO.

    The literature

  • 4a · Baselines on the surrogate

    This repo's dry run at 20 Hz: PI (tuned or by bandwidth), PPO and SAC on the 2×2 task and the seven test cases with faults, and the networks live in the Lab.

    The dry run

  • 5 · Limitations

    What fails and by how much: sim-to-real gaps, valve models, sensor noise, compute.

    The caveats

  • 6 · Open questions

    Ranked research openings with effort estimates, starting with anticipation without preview.

    What to do next

  • 7 · References

    Every source, how far it was verified, and the search log.

    The sources

  • Personal · For Leander

    The author's own on-ramp: what transfers from an RL background, what is new, what the organisers have confirmed, and ten working days.

    The on-ramp

Reproduce

git clone https://github.com/leandergrech/rl-rocket-engine-control && cd rl-rocket-engine-control
python3.12 -m venv .venv && . .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -e ".[dev,rl]"
bash scripts/reproduce.sh

reproduce.sh runs the tests, including the check that the browser model matches the Python one, and builds the docs strictly. It also re-evaluates every stored policy against data/results/summary.json; the environment is deterministic, so they must match. bash scripts/reproduce.sh --full also recalibrates the surrogate and retrains every baseline (about 3 hours on a laptop CPU). To check whether the challenge has gone public, run python scripts/check_challenge.py. STATUS.md tracks the build state.

References

Every source used on this site, with how far it was verified (full text, abstract only, unverified or personal communication):

7. References Timeline 2018–2026 Glossary Cite the foundation

Key documents: Dresia 2025 (thesis) · Bareiß et al. 2025 (benchmark design) · Dresia et al. 2025 (RL4AA'25) · Dauer et al. 2025 · Kurudzija et al. 2025 · Traudt et al. 2024 · Deeken et al. 2021 · Waxenegger-Wilfing et al. 2021