flowchart LR
subgraph ESP32
W["WiFiClientSecure<br>(TLS)"] --> P["PubSubClient<br>(MQTT)"]
P --> J["ArduinoJson<br>(parsing the delta)"]
J --> I["Infrared sending<br>(IRremoteESP8266)"]
end
P <-- "port 8883" --> A[("AWS IoT Core")]
7 On to a Real ESP32 (Explanation)
These are the key points for doing what the virtual device has done so far on an ESP32 (Arduino framework). The contents of this chapter are what was confirmed on real hardware in the original project (ESP32-DevKitC + infrared receiver/transmitter, PlatformIO). In this hands-on, the ESP32 code has not been built or tested on real hardware.
7.1 Architecture
| Role | Library |
|---|---|
| TLS (certificate-based authentication) | WiFiClientSecure (included in the ESP32 Arduino core) |
| MQTT | PubSubClient |
| JSON | ArduinoJson |
7.2 Up to connecting
tlsClient.setCACert(AWS_CERT_CA); // contents of AmazonRootCA1.pem
tlsClient.setCertificate(AWS_CERT_CRT); // device.pem.crt
tlsClient.setPrivateKey(AWS_CERT_PRIVATE); // private.pem.key
mqtt.setServer(AWS_IOT_ENDPOINT, 8883);
mqtt.setBufferSize(2048); // see below
mqtt.connect(THING_NAME); // client ID = Thing name (match the policy)Put the certificate, private key and endpoint together in secrets.h and add it to .gitignore. Only a secrets.h.example with empty contents goes in the repository.
7.2.1 Stumbling block 1: TLS will not connect if the clock says 1970
TLS checks the validity period of the server certificate. The ESP32 has no clock battery, and right after boot its time is 1970, so the certificate is judged “not yet valid” and the connection fails. After connecting to Wi-Fi, set the time with NTP before connecting to MQTT.
configTime(0, 0, "pool.ntp.org", "time.nist.gov");
// wait until time(nullptr) is a realistic value (e.g. 2023 or later) before calling connect7.2.2 Stumbling block 2: PubSubClient’s receive buffer is 256 bytes
PubSubClient’s receive buffer is 256 bytes by default. get/accepted contains all of desired, reported and metadata (the update time of each field), so it quickly exceeds 256 bytes and is silently discarded without being received. Enlarge it, e.g. setBufferSize(2048), and enlarge ArduinoJson’s parsing area (StaticJsonDocument<2048>, etc.) to match. In the original project, even the initial 384 bytes was not enough, and parsing failed.
7.2.3 Stumbling block 3: BLE and TLS cannot run at the same time
The original project also had a feature for operating it from a smartphone over BLE. On an ESP32 without PSRAM (the WROOM series),
- the BLE stack constantly uses about 50–70 KB of heap
- PlatformIO’s default mbedTLS allocates its send/receive buffers as fixed-size 16 KB × 2 (it is a prebuilt library, so they cannot be made smaller)
so when Wi-Fi, BLE and TLS ran simultaneously, memory ran out and X509 - Certificate verification failed and SSL - Memory allocation failed kept appearing. As a countermeasure, BLE mode and cloud mode are made mutually exclusive; when switching to cloud mode, BLE is stopped with BLEDevice::deinit() to free memory before connecting with TLS.
7.3 Processing the delta
It has the same structure as the virtual device’s process_delta() (simplified from the original project’s code).
void processDelta(JsonObject delta) {
if (delta.isNull() || delta.size() == 0) return;
if (delta.containsKey("sendSlot")) {
String raw = delta["sendSlot"].as<String>(); // "5-1791382552299"
uint8_t slot = raw.substring(0, raw.indexOf('-')).toInt();
if (raw != lastSendSlot) { // duplicate check (Chapter 6)
irController.sendSlot(slot);
lastSendSlot = raw;
}
// write the received string back as a string (don't forget the double quotes)
String body = "{\"state\":{\"reported\":{\"sendSlot\":\"" + raw + "\"}}}";
mqtt.publish(TOPIC_UPDATE, body.c_str());
}
bool hasAc = delta.containsKey("power") || delta.containsKey("temp") /* ... */;
if (hasAc || delta.containsKey("cmdId")) {
// overwrite only the fields that arrived onto the current state (Chapter 6)
AcState s = irController.currentAcState();
if (delta.containsKey("power")) s.power = delta["power"].as<bool>();
if (delta.containsKey("temp")) s.temp = delta["temp"].as<int>();
// ...
irController.applyAc(s);
// write back the state and cmdId in "a single publish" (Chapter 5)
publishReported(s, delta["cmdId"].as<String>());
}
}Pass doc["state"] when it arrives on /update/delta, and doc["state"]["delta"] when it arrives on /get/accepted, to the same processDelta().
Writing "\"cmdId\":" + cmdId produces a number, while "\"cmdId\":\"" + cmdId + "\"" produces a string. If the type differs from desired, you get the infinite loop of Chapter 5. If you build it with ArduinoJson, the type of the received value is preserved as is.
7.4 Cloud commands do not go through the BLE processing path
The original project’s firmware was designed to put BLE commands into an event queue and process them with a state machine (the same idea as the nRF52840 event-driven firmware hands-on). At first, Shadow deltas were also put into the same event queue, but because BLE is stopped in cloud mode,
- the state machine had returned to its idle state on “BLE disconnected” and did not process the send command
- at the end of processing it tried to report the result over BLE, accessing the stopped BLE stack
and these problems occurred. Commands from the cloud are now separated so that they call the infrared-sending part directly. When the same function is used from multiple paths (BLE, cloud), check the assumptions of each path (which stacks are running).
- Before TLS, set the time with NTP, and enlarge PubSubClient’s receive buffer
- On an ESP32 without PSRAM, using BLE and TLS at the same time can run out of memory
- Processing the delta is the same as in the virtual device: type-consistent write-back, a single publish, a duplicate check