Why Deterministic Rollouts Underestimate Uncertainty in Time Series Forecasting
How naive forecasting discards uncertainty and what to do instead.
1The one-step prediction problem
The article builds on Part I's foundation of one-step-ahead prediction using Gaussian negative log-likelihood (NLL) loss. This trains models to output both conditional mean (μ) and variance (σ²) for the next time step, capturing irreducible randomness. While perfect for single-step prediction, this approach breaks down when extended to multi-step forecasting, as it fails to account for accumulated uncertainty across steps.
2The forecast horizon challenge
Real-world applications require forecasting over extended horizons (10-1000 steps), not just single steps. The naive approach of recursively feeding mean predictions discards learned uncertainty at each step. This creates a false premise where each subsequent prediction assumes perfect knowledge of previous steps, leading to systematic underestimation of total uncertainty as errors compound over time.
3Joint distribution vs separate predictions
Proper forecasting requires modeling the joint distribution over future sequences, not just marginal distributions at each step. The chain rule decomposes this joint distribution into products of one-step conditionals, but each step's input becomes uncertain after the first prediction. Treating predicted means as certain inputs ignores this fundamental dependency between steps.
4Deterministic rollout flaws
The deterministic rollout approach feeds only the mean prediction (μ) into subsequent steps, effectively treating it as observed truth. This causes the model to answer the wrong question at each step - calculating uncertainty assuming perfect previous inputs. The resulting σ values only capture local noise, missing the propagated uncertainty from previous steps' distributions.
5Error accumulation analogy
Like a navigator assuming perfect compass readings, deterministic rollout ignores how small errors accumulate. After 50 steps, the model might show narrow confidence bands (2m uncertainty) when reality could be 200m off. Each step's reported uncertainty is technically correct for its false premise, but the premise itself becomes increasingly dishonest over time.
6Stochastic rollout solution
The correct approach uses Monte Carlo sampling - drawing random samples from each step's predicted distribution to feed forward. This maintains the full uncertainty through the prediction chain. Running multiple trajectories (M=500 in the example) approximates the true joint distribution, with each path reflecting possible error propagation.
7Implementation comparison
The stochastic rollout requires minimal code changes from deterministic - just replacing mean appending with random sampling. However, it processes all trajectories in parallel batches for efficiency. The key difference is preserving the full distribution at each step rather than collapsing it to a point estimate, allowing proper uncertainty propagation.
8Law of total variance
The mathematical foundation shows total variance decomposes into local noise (aleatoric uncertainty) and propagated uncertainty (epistemic). Deterministic rollout only captures the first term. Stochastic rollout accounts for both, as different sampled inputs lead to different prediction distributions, properly widening the overall forecast uncertainty.
9Diagnostic verification
The companion notebook implements both approaches on the same model and signal, with coverage and probability integral transform (PIT) diagnostics. These tests verify stochastic rollout produces properly calibrated uncertainty - prediction intervals that actually contain the true values at their claimed confidence levels, unlike deterministic's overconfident bands.
10Practical implications
Underestimated uncertainty has real consequences - false alarms might be missed or thresholds incorrectly deemed safe. Stochastic rollout provides honest uncertainty estimates essential for risk assessment and decision making. The method applies to any probabilistic time series model, requiring only that it outputs proper distributions.
11Computational tradeoffs
While stochastic rollout requires more computation (multiple trajectories), the batch processing keeps it efficient. The alternative - analytical uncertainty propagation - would require complex covariance tracking. Monte Carlo sampling provides a practical, general solution that works with any model architecture capable of probabilistic outputs.
12Fundamental forecasting principle
The core insight is that multi-step forecasting requires treating predictions as distributions at every step. Any approach that collapses intermediate uncertainty produces misleading results. This principle extends beyond Gaussian models to any probabilistic forecasting scenario where errors can accumulate through sequential predictions.