The Controller Is Not the Platform

Every solar-plus-storage site you deploy comes with a controller. It speaks Modbus to the inverters, battery racks, meters and generators, runs local control logic, and usually ships with a perfectly serviceable local dashboard. It does its job well.

What it does not do is run your business.

The controller knows about one site. Your business is a growing fleet of them. The moment you are operating more than a handful of installations, a set of questions appears that no site-level controller can answer:

• How did all our installations perform last month?
• Which sites are underperforming against their forecast, and since when?
• Is site fourteen actually offline, or just its network link?
• What did we dispatch last Tuesday afternoon, and did it work?
• Can we bill a customer accurately from data we control?

This article is about the cloud layer that answers those questions — what it needs to do, how it should be structured, and the decisions that are expensive to change later.

Four Jobs the Cloud Layer Has to Do

Strip away the product features and an EMS or BESS cloud platform has four responsibilities. Everything else is built on top of them.

1. Ingest without losing anything

Site controllers publish telemetry continuously — often every few seconds. That stream has to reach durable storage you own, without gaps, through deployments, database maintenance and worker restarts. If a routine deployment punches a hole in your energy record, the platform is not fit for billing or performance guarantees.

2. Store data the business actually owns

Operational history sitting inside third-party hardware in a vendor format is not an asset. It cannot be queried across sites, cannot be joined to commercial data, and disappears if you change hardware supplier. The cloud layer converts device output into a record the company controls.

3. Know whether each site is alive

A site that stops reporting is either offline, or its link is down, or its controller has hung. You need to detect that within a defined window and record it as an event — because "when did site nine go dark" becomes a contractual question the moment you sign an availability commitment.

4. Dispatch, with proof

The commercial case for storage rests on charging and discharging at the right moments. Doing that centrally means sending commands to hardware and being able to demonstrate afterwards exactly what was sent, what the device accepted, and what it reported back.

The Reference Architecture

A structure that satisfies all four, and that we have built and run in production:

1. A managed IoT broker as the device front door — per-device identity with revocable credentials, TLS termination, and routing by message type.
2. A durable event stream as a buffer between the broker and everything downstream, with a retention window measured in days.
3. A dedicated ingestion worker — headless, no HTTP surface — that consumes the stream and writes to the database.
4. A relational time-series store holding both the raw device frame and typed, indexed columns.
5. A separate read API serving web and mobile clients, which takes no part in ingestion.
6. A command service alongside the API, handling dispatch, replies, timeouts and audit.

Model Sites and Devices Correctly, Once

This is the modelling decision that is painful to revisit later.

The thing that connects to your broker is the site controller — one authenticated identity per site. But the things you want to chart, bill and alert on are the units behind it: individual inverters, battery racks, meters, generators. The vendor payload identifies those with its own internal position or slot identifier.

So every stored row needs two identifiers: the authenticated controller identity, and the in-payload device identifier. Together with a timestamp and an index, a per-device time series becomes a trivial filtered query — and one controller reporting three units produces three independent, queryable series rather than one blended mess.

Above that sits the commercial hierarchy your business actually operates on:

Tenant → Site → Controller → Device. A tenant is the customer or operating entity. A site is a physical installation. A controller is the gateway at that site. Devices are the units behind it. Get this in place early, even if you currently have one tenant, because retrofitting multi-tenant isolation onto data that was never stamped with a tenant is genuinely difficult — and until ingestion can resolve a device identity to a tenant, you do not have real data isolation, only the appearance of it.

Store Raw and Typed — Both, Always

Write every frame twice in a single pass.

Raw, exactly as received, in a JSON column with stream metadata. This is your contract with the hardware vendor. When you discover in month nine that a scaling factor was misinterpreted, the typed tables can be rebuilt from raw. Without it, the only remedy is data you no longer have.

Typed, parsed into indexed numeric columns: per-phase voltages and currents, DC bus values, run status, power limits, state of charge, and per-device power flows across PV, battery, grid, load and generator. Dashboards and reports need real columns; JSON scans across millions of rows do not hold up.

One refinement that matters with third-party firmware: when an unrecognised message type arrives, store it raw and log a warning rather than rejecting it. Vendors add message types in firmware updates without telling you. That should be a log line, not an outage.

Separate Ingestion From the API

Ingestion load scales with device count — every controller in the field reporting every few seconds, entirely indifferent to whether anyone is awake. API load scales with user traffic — bursty, concentrated in working hours.

Two different curves, two different scaling policies. Run them in one process and a busy morning of dashboard usage competes with the field for the same resources, while a bug in either can take down both.

Run them as separate deployable services sharing one database, cleanly partitioned by schema — the ingestion worker owning its tables, the API owning its own. One source of truth, two independent failure domains.

Liveness: Do Not Trust the Device Clock

Site liveness looks simple — track when each controller last reported, alert if that exceeds a threshold — and there is one trap in it.

Do not use the timestamp inside the device payload. Field controllers routinely run with badly wrong clocks; we have seen an eight-hour offset on live hardware, which was enough to make valid commands appear expired on arrival. Judge liveness on broker-side sequence numbers and enqueue times, which are stamped by infrastructure you control.

Record offline and recovery as events, not just a status flag. A flag tells you the current state. Events let you answer "how many hours of downtime did site nine have last quarter" — which is the question that appears as soon as availability is contractual.

Control: Earn It Before You Enable It

Remote dispatch is where the commercial value concentrates, and also where the risk does. Two principles keep both in hand.

Every command reaches a provable terminal state. Acknowledgement and execution result are separate signals; three independent timeouts — acknowledgement, result, expiry — guarantee resolution even when the device says nothing. Every transition is written to an append-only log alongside the exact payload dispatched. When someone asks what happened, you produce a record rather than a recollection.

Physical actuation ships disabled. Build the full lifecycle, deploy it, verify it end to end against real hardware in the field — using mock commands only. A mock command traverses the entire path and returns a genuine result while touching nothing physical. That gets the highest-risk part of the system proven under production conditions at zero equipment risk. Real commands are then enabled one at a time, each after the manufacturer has supplied its codes, parameter ranges, safety limits and readback rules, and each with its own verification cycle.

Enabling every physical command at once, on the strength of a simulator test, is how an integration bug becomes an incident report.

Plan for a Vendor You Do Not Control

Unless you manufacture your own controllers, the payload contract, the firmware release schedule and the behaviour of the hardware belong to someone else. Design for that:

• Store unknown message types instead of rejecting them, so a firmware update degrades into a log entry.
• Dead-letter unparseable frames with the error text and keep processing, so one malformed payload cannot stall the fleet.
• Keep the raw archive, so a mapping assumption that turns out to be wrong is recoverable rather than terminal.
• Build a device simulator speaking the raw protocol rather than a cloud SDK, so you can develop and regression-test without field hardware, site access or vendor availability.
• Write the integration contract down — envelope, message types, identifiers, required fields, units, timestamp semantics, and the expectation of NTP synchronisation. Ambiguity becomes an incident.

Security: Know Which Secret Actually Matters

Not all credentials carry equal risk, and treating them uniformly means either over-protecting the harmless ones or under-protecting the dangerous ones.

The practical consequence: give every application the narrowest capability that works. An ingestion worker and a read API need listen-only stream credentials — they read, they cannot inject. The genuinely high-value secrets are device keys and the command-send service credential, and those deserve managed-vault storage, rotation procedures and a documented response plan.

One thing to be clear-eyed about: with shared-key device authentication there is no content-level signing of telemetry, so a valid device key is trusted injection. Key protection is the security boundary. For production fleets, per-device certificates — revocable individually — are the stronger position.

Scaling: What Actually Changes

A well-structured platform grows by configuration rather than redesign:

That last row is the payoff for keeping raw frames. A second controller vendor means a new parser and a new typed mapping, not a new platform.

What to Build First

If you are starting from a fleet of controllers and no cloud layer, the sequence that works is deliberately unglamorous:

1. Write the integration contract with your hardware vendor and build a simulator against it, before any cloud infrastructure exists.
2. Ship telemetry ingestion and verify it with real numbers — row counts, dead-letter counts, a confirmed replay from stream retention.
3. Add liveness monitoring, based on broker-side timing, with offline and recovery events.
4. Build the command lifecycle with physical actuation disabled, and verify it end to end on live hardware using mock commands.
5. Enable physical commands individually, once the manufacturer has delivered a signed-off contract, each with its own verification cycle.

Each phase ends with evidence rather than a status update. Resist the temptation to start with the dashboard — the dashboard is the easy part, and it is worth very little on top of data you cannot trust.

Closing Thought

The controller at each site solves local control, and it solves it well. What it cannot give you is a fleet you can see, a record you own, a liveness signal you trust, or a dispatch action you can prove.

That is the cloud layer’s job. Build it around a durable buffer, idempotent writes, raw plus typed storage, separated scaling axes, and control that is gated until it has earned the right to actuate — and you end up with a platform that grows with the business instead of one you rebuild at every step.

TTPL builds cloud platforms for energy and industrial operators — telemetry ingestion, fleet monitoring, time-series storage and audited device control on Azure and AWS. If you are moving from site-level controllers to a fleet platform, let us review the architecture with you.