Executive Summary and Impact
Most infrastructure forecasting failures are system-definition problems rather than machine-learning failures. This article establishes the architectural foundation of mapping physical system boundaries, unobserved behind-the-meter generation, and multi-temporal resolution before training predictive models.
Core Thesis
Most forecasting problems are presented as data problems. In infrastructure, they are almost always system-definition problems first.
Act I: The Mirage of the Simple Dataset
1. The Moment the Problem Looks Easy
There is a familiar moment at the beginning of a forecasting project:
Key Insight
We have years of electricity data. We have consumer bills, network load, solar generation, weather data. Let's build a model.
It sounds like a modelling problem. It usually isn't.
The first question is much more basic: What exactly are we trying to forecast?
For an electricity network, "load" can mean several different things depending on where it is measured, which consumers are included, what generation is behind the meter, and what geographic boundary the data represents.
A model cannot resolve those ambiguities for you. You have to resolve them before the model sees the data.
Act II: Metering vs Physical System Boundaries
2. A Meter Does Not Measure "Demand"
A meter measures something very specific at a very specific point in a physical system.
These measurements are related, but they are not interchangeable.
- Consumer billing data: Tells you what was billed over a billing period.
- Solar-generation dataset: Tells you how much generation occurred at another point in the system.
- Substation measurement: Captures the net flow through that measurement point.
Treating all of these as interchangeable measurements of "demand" would have created a very sophisticated answer to the wrong question.
3. The Boundary Comes Before the Dataset
Suppose the objective is to forecast electricity demand for a particular geographic and administrative system.
It is tempting to join consumer bills, solar generation, and substation telemetry because they share dates and appear to describe the same electricity system.
But a shared timestamp does not imply a shared system boundary.
The substation may serve a larger area. The solar plant may offset grid demand rather than appear as consumer consumption. The consumer dataset may represent billed energy rather than instantaneous physical demand.
The forecasting target has to be defined in terms of the system first, and the datasets selected second.
Act III: Spatial and Temporal Granularity
4. Why Category-Wise Forecasting Matters
A single "Total Load" series is convenient, but it can hide very different underlying behaviours.
Residential consumption does not respond to the same factors as industrial consumption. Agricultural demand is strongly influenced by weather and seasonal activity.
Instead of forecasting one number, we forecast each consumer category separately, then aggregate the category forecasts into the system-level forecast.
5. Time Resolution Is Part of the System Definition
A monthly forecast and an hourly forecast are not the same forecasting problem.
Grid operations eventually care about something much more granular: What does the load look like at 2 PM on a particular day?
Forecasting the right quantity at the wrong resolution can still produce the wrong operational answer.
Act IV: Mental Model First
6. The First Model Is Not the Machine-Learning Model
Before writing the forecasting code, you need another model. Not a statistical model, but a mental model of the physical system.
You need to know:
- What the system boundary is
- Where each measurement sits within it
- What each measurement actually represents
- Which consumers or loads are included
- How generation and consumption interact
- What time resolution is meaningful for the decision you are trying to make
Only then does the machine-learning problem become well-defined.
Operator Takeaway
The first model you build in a complex infrastructure problem should be the model of the system itself.