1  What Is a Device Shadow?

1.1 Why a “shadow” is needed

When operating a device from the cloud, the most naive approach is to “send a message directly to the device”. But IoT devices for home appliances have the following circumstances.

  • They are not always connected: Wi-Fi drops, the power is unplugged, the device is rebooting
  • They cannot always reply immediately: sending infrared takes time, the device is busy
  • You may be asked about the current state: you want to answer “what temperature is the air conditioner set to now?” from the cloud alone

So AWS IoT Core keeps a JSON document called a Shadow in the cloud for each device. Cloud-side programs interact with the Shadow rather than with the device, and the device synchronizes with the Shadow whenever it is connected.

sequenceDiagram
  participant C as Cloud (Lambda, etc.)
  participant S as Shadow
  participant D as Device
  C->>S: desired: {power: true}
  S-->>D: delta: {power: true}
  Note over D: Send infrared to the air conditioner
  D->>S: reported: {power: true}
  Note over S: desired and reported match → no delta

1.2 What is inside a Shadow

A Shadow is a JSON document like the following (the form shown by send.py get).

{
  "version": 12,
  "state": {
    "desired":  { "power": true,  "temp": 24 },
    "reported": { "power": false, "temp": 24, "mode": "cool" },
    "delta":    { "power": true }
  }
}

desired (the desired state)

Written by the cloud: “please make it like this”. Updated by Alexa’s Lambda, or by send.py in this hands-on

reported (the reported state)

Written by the device: “this is how it is now”. The device updates it when it has finished processing

delta (the discrepancy)

Only those fields of desired that differ from reported. AWS computes it automatically and delivers it to the device

version

A number that increases by 1 every time the Shadow is updated. As a meter that counts the number of updates, it can be used to detect accidents (Chapter 5)

1.3 Rules for delta (confirmed by experiment)

When a delta is delivered, and with what contents, is the core of this hands-on. Here we first summarize the rules confirmed by experiments on AWS IoT Core.

  1. An update is a “merge”, not an “overwrite”. Only the fields you send change; the others remain as they are
  2. Sending null as a value deletes that field. "desired": null deletes the whole of desired
  3. delta = the fields of desired whose “value and type” differ from reported. Fields absent from reported are also included in delta. "1700" (a string) and 1700 (a number) are different values
  4. version increases by 1 for every accepted update (even if no value changes)
  5. After an update is accepted, if delta is not empty, the entire delta is delivered to the device. This happens not only for desired updates but also for reported updates, and even for updates to unrelated fields
  6. A delta delivered while the device is disconnected does not reach it (except with MQTT persistent sessions; Chapter 6). A get retrieves the delta at that point in time

Rule 5 is easy to overlook just from reading the documentation, and it is the cause of the infinite loop in Chapter 5.

1.4 Shadow topics

The device interacts with the Shadow over MQTT using the following topics (<thing> is the Thing name).

Topic Direction Use
$aws/things/<thing>/shadow/update device → AWS Update reported (or desired)
$aws/things/<thing>/shadow/update/delta AWS → device delta is delivered here. The device subscribes
$aws/things/<thing>/shadow/update/accepted AWS → device The update was accepted
$aws/things/<thing>/shadow/update/rejected AWS → device The update was rejected (format error, version conflict, etc.)
$aws/things/<thing>/shadow/update/documents AWS → device The whole Shadow before and after the update
$aws/things/<thing>/shadow/get device → AWS Request the current Shadow (send an empty message)
$aws/things/<thing>/shadow/get/accepted AWS → device The current Shadow (including delta). The device subscribes

Cloud-side programs (Lambda, etc.) usually use the AWS SDK’s UpdateThingShadow / GetThingShadow (HTTPS) rather than MQTT. The Shadow behaves the same whichever way it is updated.

1.5 Try it in the simulator

Run a Shadow and a virtual device in the browser. Press the buttons and watch how desired, reported, delta and version change.

NoteTry this
  1. Press the “Light button”: it proceeds desired → delta → device sends → reported, and the delta disappears. version increases by 2
  2. Press “AC ON 24℃” and then “Temp only 27℃”: the second delta contains only temp and cmdId, not power (rules 1 and 3)
  3. Press “Disconnect device”, then the light button, then “Connect device”: the delta from while it was disconnected does not arrive, but the device fetches it with a get right after connecting (rule 6)
ImportantSummary of this chapter
  • A Shadow is a JSON document combining the cloud’s desired and the device’s reported. The discrepancy is the delta
  • Updates merge, null deletes, and version increases with every update
  • While a delta remains, the entire delta is delivered after any update