Reinforcement Learning-Based Dual-Policy Adversarial Model for Dynamic Difficulty Adjustment in a Turn-Based Roguelike Game

Dynamic Difficulty Adjustment (DDA) systems traditionally optimize for challenge, but challenge alone does not guarantee engagement. A player may win at an ideal 50% rate yet still feel bored if games lack tension, close calls, or dramatic reversals. This work presents a reinforcement learning-based adversary for an Into the Breach inspired turn-based game that optimizes for engagement besides difficulty. The system employs a dual-objective architecture, which combines an engagement-based policy seeking to maximize player retention, paired with a challenge policy that maximizes raw difficulty. A contextual bandit meta-controller learns to blend these policies based on player state, easing pressure when players struggle and increasing it when they dominate. Engagement is operationalized through validated metrics including flow zone rate, close call frequency, and grit score. The proposed framework involves a hybrid architecture with two components. Namely, a director level controller manages macro-level parameters such as spawn configurations, unit aggressions and target priorities, while a tactical controller manages micro-level decisions on a per unit basis spanning movements and attacks. The pipeline explores algorithm comparison across PPO, A2C, DQN, QRDQN and TRPO alongside memory architecture such as MLP, LSTM and GTrXL, with the final system deployed as a FastAPI server for real-time game integration. The architecture is designed for future human player studies to confirm that the engagement proxies correlate with subjective player experience.

Committee: Simanta Mitra (major professor)