The IoT pilot went perfectly. Fifty sensors, a Node service receiving readings over HTTP, writing each one to Postgres, a dashboard on top. Built in a fortnight, demoed well, everyone happy.
Then the rollout was approved for five thousand devices, and every assumption in that architecture turned out to be wrong. Not slightly wrong — wrong in kind. Each device holding an HTTP connection meant tens of thousands of concurrent connections. One row per reading meant a billion rows a month. And a firmware bug meant we needed to update five thousand devices in the field, which nobody had designed for, because with fifty devices you just reflash them by hand.
The gap between an IoT pilot and an IoT system is bigger than in almost any other kind of software. Here is what that gap contains.
Design Around the Constraints That Do Not Move
Three facts shape every decision in this space, and no amount of cloud architecture makes them go away.
Devices are offline a lot. Not occasionally — routinely. Signal drops, power cycles, a gateway reboots. A device that cannot buffer readings locally and send them later is a device that loses data every week.
Bandwidth and power are real budgets. A battery sensor sending a verbose JSON payload every thirty seconds over a cellular connection has both a data bill and a battery life measured in weeks instead of years. Payload size is a product decision.
You cannot physically reach the fleet. Devices end up in ceilings, on poles, inside machinery, in other countries. Anything that requires touching them is effectively impossible past a certain scale, and that includes fixing whatever you got wrong.
Design as if all three are true, even during the pilot, because they will be true later and the architecture is hard to change once devices are deployed.
The Layers That Actually Exist
Whatever the vendor diagram says, a working IoT system has four parts.
The device, running firmware, buffering locally, and holding credentials it must protect.
The transport, almost always MQTT over TLS. This is where the pilot's HTTP choice broke. MQTT keeps one lightweight persistent connection per device with tiny message overhead, supports quality-of-service levels for delivery guarantees, and has a last-will message so the broker can announce a device that dropped off. For constrained devices it is simply the right protocol, and HTTP is not.
Ingestion, which must decouple immediately. Messages land on a broker, and the broker feeds a queue or stream. Nothing that processes a reading should be in the path of receiving it — otherwise a slow database write becomes dropped telemetry.
Storage and processing, split by purpose. Recent data in something fast for dashboards and alerts. Historical data downsampled and archived. Which brings me to the mistake that produced a billion rows.
Do Not Store Every Reading Forever
A temperature sensor reporting every ten seconds produces about 260,000 readings a year. Multiply by five thousand devices. Nobody will ever query the individual reading from a Tuesday afternoon eight months ago, but you will pay to store and index it forever.
What works instead:
Report on change, not on a timer. If the temperature has not moved more than a threshold, do not send. This one change cut traffic by around 80% on a monitoring project and had no effect on what the dashboards showed.
Use a time-series database. Timescale, InfluxDB, or a managed equivalent. Purpose-built compression and automatic downsampling, rather than a general-purpose table you keep adding indexes to.
Downsample on a schedule. Full resolution for a week, per-minute averages for a month, hourly for a year, daily after that. Nearly all analytical questions are answered fine by the aggregate.
Batch at the edge. Send twelve readings in one message every two minutes rather than one message every ten seconds. Fewer round trips, less radio time, longer battery life.
Device Management Is the Part Nobody Budgets For
This is the difference between a demo and a product, and it is the least glamorous work in the project.
Provisioning. How does a device get its unique identity and credentials at manufacture or first boot? A shared credential across the fleet means one extracted device compromises everything, and extracting credentials from hardware you can hold is not difficult.
Over-the-air updates. Assume you will ship a bug. The update mechanism needs signed images so a device only installs firmware you authored, an A/B partition scheme so a failed update rolls back instead of bricking the unit, and staged rollout so a bad release reaches fifty devices rather than five thousand. Build this before the first deployment. Retrofitting an update mechanism onto devices you cannot reach is not possible.
Fleet visibility. Which devices are online, on which firmware version, with what battery and signal. Without this you cannot answer "did the update work?" or notice that a hundred devices went quiet last Thursday.
Decommissioning. Revoking a device's credentials when it is retired, sold or stolen.
Security, Where the Stakes Are Physical
IoT security has a worse reputation than most areas of software, and it is deserved. The failures repeat: default passwords, unencrypted transport, unauthenticated firmware updates, exposed debug interfaces, no patching mechanism.
What matters most, roughly in order:
Unique credentials per device. Certificates rather than shared keys, ideally held in a secure element the firmware cannot read out. This is the single highest-value control.
TLS everywhere, with the device verifying the server. One-way encryption where the device accepts any certificate is an invitation to a man in the middle.
Signed firmware. An update path without signature verification is a remote code execution path into every device you have shipped.
Least privilege in the cloud. A device should be able to publish to its own topic and nothing else. A compromised sensor should not be able to read other devices' data or send commands.
Treat device input as hostile. A device in someone's hands is a device that can be modified. Validate everything that arrives, and rate limit per device — one malfunctioning or malicious unit should not be able to flood your ingestion.
And accept that devices in the field will run old firmware for years. Whatever you deploy today is your compatibility floor for a long time.
AIoT: Useful, With One Real Rule
Combining machine learning with device data is where a lot of the current value is, and it is genuinely more mature than it was a few years ago. Predictive maintenance from vibration signatures. Anomaly detection on energy consumption. Vision quality control on a production line.
The rule that decides the architecture: put inference at the edge when the decision must be fast, private, or work offline; put it in the cloud when it needs context the device does not have.
A camera deciding whether a part is defective should decide locally — the latency budget is small, the bandwidth cost of streaming video is large, and it must keep working when the connection drops. A model predicting when a machine will fail, using patterns across an entire fleet plus maintenance history, belongs in the cloud.
The practical hybrid: small models on device for immediate reactions, aggregated data to the cloud for the models that need the whole picture, and updated models pushed back down through the same signed update channel you built for firmware.
What I Would Do Differently
Knowing what I know from that fifty-to-five-thousand jump: choose MQTT on day one even when HTTP would work for the pilot. Build the update mechanism before deploying anything. Decide the data retention policy before writing the first row. Give every device its own identity from the start.
None of that slows the pilot down much. All of it is close to impossible to add once there are devices in the field that you cannot reach — which is, in the end, the defining constraint of the whole discipline.
