6  Pitfall 3: Partial Updates, Missed Messages and Double Execution

Less spectacular than infinite loops, there are three bugs that are hard to notice.

6.1 Partial updates: changing only the temperature turns the power off

When the air conditioner is “ON, cooling, 24℃” and you say “Alexa, set the air conditioner to 26 degrees”, the Lambda writes {"temp": 26, "cmdId": ...} to desired. The delta contains only the fields that changed (measured).

desired updated: {"temp": 26}
delta: {"temp": 26}

If the device assumes the delta is “the complete air-conditioner settings” and writes

power = delta.get("power", False)    # not included → False
temp  = delta.get("temp", 26)
send_aircon(power, temp)             # ends up sending a power-OFF signal

then the power turns off even though only the temperature was changed. This is because the missing fields were filled with default values.

The correct approach is to start from the current state and overwrite only the fields that arrived (virtual_device.py).

for k in AC_KEYS:          # overwrite only the fields that arrived; keep current values for the rest
    if k in delta:
        ac[k] = delta[k]
TipWhere to keep “the current state”

So that the current state is known even after the device reboots, the original project stores it in the ESP32’s flash (NVS). Infrared appliances do not report their state back, so the state the device remembers is the only clue.

6.2 Missed messages: a delta sent while disconnected does not arrive

A delta delivered while the device is offline due to a Wi-Fi drop or reboot does not arrive even after it reconnects (measured).

# with the device stopped
python device/send.py slot 1
python device/send.py ac --power on --temp 23
# start the device
python device/virtual_device.py
[   0.1s] Connected (Success)
[   0.1s]   <<< reported update: {"power": false, "mode": "cool", "temp": 26, "fan": "auto"}
[   0.2s] delta received (get): {"sendSlot": "1-1791382699754", "power": true, "temp": 23, "cmdId": "1791382700002"}
[   0.2s]   >>> IR send #1: slot 1 "TV volume+"
[   0.2s]   >>> IR send #2: air conditioner {'power': True, 'mode': 'cool', 'temp': 23, 'fan': 'auto'}

Right after connecting, the virtual device sends get and receives the delta at that moment all at once, so it was able to execute both operations made while it was disconnected. If it only subscribes without doing a get, these two are lost.

NoteWith an MQTT persistent session, it does arrive

If you specify clean session = false (a persistent session) when connecting and subscribe with QoS 1, AWS IoT retains deltas sent while disconnected and delivers them when you reconnect with the same client ID (measured). However, there is an upper limit on how long they are retained (1 hour by default), and PubSubClient, commonly used on the ESP32, defaults to a clean session. A get right after connecting reliably fetches the current delta regardless of session settings, so it is safest to use both.

6.2.1 Missed pushes, and periodic get

In the original project, pushed deltas sometimes did not reach the ESP32 even while it was connected (with the combination of WiFiClientSecure and PubSubClient, reception was missed when TLS data arrived split into pieces). So polling that sends get every 5 seconds always runs as a backup for push (--poll in virtual_device.py).

6.3 Double execution: the same delta arrives twice

If you design the device to receive deltas via both push and get, the same delta can arrive twice. Look at the continuation of the reconnection experiment above.

[   0.2s]   <<< reported update: {"power": true, "mode": "cool", "temp": 23, "fan": "auto", "cmdId": "1791382700002"}
[   0.3s] delta received (push): {"power": true, "temp": 23, "cmdId": "1791382700002"}
[   0.3s]   (cmdId 1791382700002 already processed, so not sending)

The same air-conditioner command processed via get also arrives via push 0.1 seconds later. The cause is the state report right after connecting (at 0.1 s). At that point it still differed from desired, so that report triggered delivery of the delta (rule 5 in Chapter 1).

Without a duplicate check, the same signal is sent to the air conditioner twice. For an air conditioner the harm is small, but a toggle-type light turns on and immediately off.

6.3.1 Countermeasure: remember processed commands (make it idempotent)

Each command carries a value that differs every time ("5-<time>", cmdId), so if you remember processed values, you can avoid executing the second one.

if raw in done_ids:
    log("already processed, so not sending")
else:
    send_ir(...)
    done_ids.add(raw)
report({"sendSlot": raw})     # always write back (to clear the delta)

The property that the result is the same as doing it once no matter how many times the same operation is received is called idempotency. The basic principle is to assume that commands over a network arrive “at least once (possibly two or more times)”, and to make the receiving side idempotent.

TipHow long to keep the record of processed commands

The virtual device remembers every processed value, but memory is limited on a microcontroller. Remembering just two things, “the last processed sendSlot value” and “the last processed cmdId”, is enough to prevent the double execution seen here.

6.4 Check it in the simulator

Set the “device implementation” to No duplicate check and try the following.

  1. Press “Disconnect device”
  2. Press “AC ON 24℃” (it does not arrive because the device is disconnected)
  3. Press “Connect device” → it arrives via both the get right after connecting and the push triggered by the state report, so it is sent twice

If you switch back to the Correct implementation and do the same thing, the second one is treated as “already processed” and not sent.

ImportantSummary of this chapter
  • A delta contains only the fields that changed. Overwrite the current state; do not fill in defaults
  • A delta sent while disconnected does not arrive. Do a get right after connecting (persistent sessions can also be used)
  • The same delta may arrive twice. Remember processed values to make the device idempotent