MPC and Reinforcement Learning Compared in Renewable Hydrogen Chain

Summary

A preprint from USC compares four control strategies for operating the renewable hydrogen supply chain under realistic electrolyzer, storage, and grid constraints: rule-based, MPC, and two RL variants (forecasting and non-forecasting).

Why it matters

As hydrogen infrastructure moves beyond the pilot stage, the choice of operational control directly determines whether the system is economically viable or a subsidy sink. MPC's dominance, and the fact that forecasting-based RL underperforms its non-forecasting counterpart, offer a concrete data point for investors on where to focus: on forecasting pipelines feeding model-based optimizers, or on end-to-end learning agents?

Highlights

• MPC reduces grid intake and efficiently utilizes storage by leveraging short-term forecasts, achieving the best economic score among the four methods.
• RL-NF demonstrates robust and competitive performance without requiring any future information, showing that learning-based control can hold up even in information-poor environments.
• RL-F consistently fails to outperform RL-NF, suggesting that noisy forecasts can weaken learned policies in complex energy systems.

Critical take

This study is a single-author preprint published on a non-peer-reviewed server. Consequently, the claim of an 'integrated and physically realistic framework' and the head-to-head comparison results are still awaiting independent validation. Furthermore, since MPC is architecturally designed to use forecasts, its superiority is somewhat tautological. A more interesting question is whether RL can approach MPC's performance when given forecasts of the same quality. This question is only partially answered by the RL-F results, which may reflect implementation choices rather than a fundamental paradigm limitation.