3  Running the Virtual Device

We treat a Python program on the PC as the “infrared remote control” and operate it via the Shadow.

3.1 Using three terminals

In code/, open three terminals with the virtual environment activated (source .venv/bin/activate).

Terminal What to run Role
A python device/virtual_device.py The device. Receives deltas and “sends infrared”
B python device/send.py watch Monitor. Shows the Shadow’s version and delta every 2 seconds
C python device/send.py ... Operator. Writes desired in place of Alexa

3.2 Starting the device

Terminal A:

python device/virtual_device.py
[   0.0s] Connecting to xxxx-ats.iot.ap-northeast-1.amazonaws.com as handson-aircon-01 (bug=none, poll=5.0s)
[   0.1s] Connected (Success)
[   0.1s]   <<< reported update: {"power": false, "mode": "cool", "temp": 26, "fan": "auto"}

When it connects, it first writes its current state to reported, then fetches any delta that arrived while it was disconnected, using get (Chapter 6).

Terminal B:

python device/send.py watch
Ctrl+C to quit. If version keeps increasing, run `send.py stop` immediately
21:03:10  version=1  delta=null

3.3 Pressing the light button

Terminal C:

python device/send.py slot 5
desired updated: {"sendSlot": "5-1791382552299"} → version 2

It worked if terminal A (the device) shows the following.

[   5.2s] delta received (push): {"sendSlot": "5-1791382552299"}
[   5.2s]   >>> IR send #1: slot 5 "Light toggle"
[   5.2s]   <<< reported update: {"sendSlot": "5-1791382552299"}
  1. send.py writes sendSlot to desired (version 2)
  2. Because it differs from reported, AWS pushes a delta to the device
  3. The device sends infrared and writes the received value back to reported as is (version 3)
  4. desired and reported now match, so the delta disappears

In terminal B you can see version increase from 1 → 3 and delta return to null.

NoteWhy "5-1791382552299"?

The time of sending (in milliseconds) is appended after the slot number. Why we do not use a plain 5 is the subject of the experiment in the next chapter.

3.4 Operating the air conditioner

python device/send.py ac --power on --temp 24
[  30.1s] delta received (push): {"power": true, "temp": 24, "cmdId": "1791382569430"}
[  30.1s]   >>> IR send #2: air conditioner {'power': True, 'mode': 'cool', 'temp': 24, 'fan': 'auto'}
[  30.1s]   <<< reported update: {"power": true, "mode": "cool", "temp": 24, "fan": "auto", "cmdId": "1791382569430"}

The delta contains only the fields that were sent (power and temp) and cmdId. The device overwrites its current state with only the fields that arrived, and writes all fields back to reported in a single update.

You can view the whole current Shadow with get.

python device/send.py get

3.5 Reading the device code

The core is process_delta(). It looks at the received delta and handles two kinds of fields:

  • sendSlot (button): command type. We want it sent exactly once per press
  • power, mode, temp, fan (air conditioner): state type. “Please put it in this state”
device/virtual_device.py
"""Virtual device: a Python program that pretends to be an infrared remote control (learned buttons + air conditioner)

Subscribes to the AWS IoT Core Device Shadow over MQTT; when it receives a delta (what it is asked to do),
it pretends to have "sent infrared" and writes the result back to reported.

    python device/virtual_device.py                 # correct implementation
    python device/virtual_device.py --bug null      # reproduce the accident (Chapter 5)

--bug reproduces bugs that actually happened (the duplicate check is also disabled, behaving exactly as it did then):
    none       correct implementation (default)
    nowrite    processes the delta but does not write back to reported
    null       sets reported to null (= deletes the field)
    number     writes cmdId back as a number (desired is a string)
    split      writes back the air-conditioner state and cmdId in two separate updates
    nodedup    writes back correctly, but does not check for already-processed commands
"""
import argparse
import json
import os
import ssl
import threading
import time

import paho.mqtt.client as mqtt

parser = argparse.ArgumentParser()
parser.add_argument("--config", default="device/config.json")
parser.add_argument("--bug", choices=["none", "nowrite", "null", "number", "split", "nodedup"], default="none")
parser.add_argument("--poll", type=float, default=5.0, help="interval (s) for re-fetching via shadow/get; 0 disables it")
parser.add_argument("--max-exec", type=int, default=10, help="safety device: stop when the number of IR sends exceeds this")
args = parser.parse_args()

cfg = json.load(open(args.config))
THING = cfg["thing_name"]
BASE = f"$aws/things/{THING}/shadow"
DEDUP = args.bug == "none"   # in bug-reproduction modes, no duplicate check, just as it was then

# Learned buttons (slot number → name)
SLOTS = {0: "TV power", 1: "TV volume+", 5: "Light toggle"}
# Current state of the air conditioner (the values the device actually knows)
ac = {"power": False, "mode": "cool", "temp": 26, "fan": "auto"}
AC_KEYS = list(ac)

exec_count = 0
done_ids = set()        # already-processed commands (sendSlot values and cmdIds)
lock = threading.Lock()
t0 = time.time()


def log(msg):
    print(f"[{time.time() - t0:6.1f}s] {msg}", flush=True)


def send_ir(text):
    """Pretend to send infrared. With a safety device."""
    global exec_count
    exec_count += 1
    log(f"  >>> IR send #{exec_count}: {text}")
    if exec_count >= args.max_exec:
        log(f"!!! Safety device: sends reached {args.max_exec}, stopping.")
        log("!!! Run `python device/send.py stop` in another terminal to clear desired.")
        client.disconnect()
        os._exit(1)   # stop the whole process, even from the MQTT receive thread


def report(state):
    """Update reported (1 publish = 1 Shadow update)"""
    client.publish(BASE + "/update", json.dumps({"state": {"reported": state}}), qos=1)
    log(f"  <<< reported update: {json.dumps(state, ensure_ascii=False)}")


def process_delta(delta, source):
    if not delta:
        return
    log(f"delta received ({source}): {json.dumps(delta, ensure_ascii=False)}")

    # --- Sending a learned button (command type: arrives as a value that changes every time, "<slot>-<time>") ---
    if "sendSlot" in delta:
        raw = delta["sendSlot"]
        slot = int(str(raw).split("-")[0])
        if DEDUP and raw in done_ids:
            log(f"  ({raw} already processed, so not sending)")
        else:
            send_ir(f"slot {slot} \"{SLOTS.get(slot, 'unregistered')}\"")
            done_ids.add(raw)
        if args.bug == "nowrite":
            pass                                # no write-back → permanently differs from desired
        elif args.bug == "null":
            report({"sendSlot": None})          # null means "delete the field"
        else:
            report({"sendSlot": raw})           # write back the received value as is

    # --- Air conditioner (state type: only changed fields arrive; cmdId changes every time) ---
    if any(k in delta for k in AC_KEYS) or "cmdId" in delta:
        cmd_id = delta.get("cmdId")
        for k in AC_KEYS:                       # overwrite only the fields that arrived; keep current values for the rest
            if k in delta:
                ac[k] = delta[k]
        if DEDUP and cmd_id is not None and cmd_id in done_ids:
            log(f"  (cmdId {cmd_id} already processed, so not sending)")
        else:
            send_ir(f"air conditioner {ac}")
            if cmd_id is not None:
                done_ids.add(cmd_id)
        if args.bug == "split":
            report(dict(ac))                    # 1st: state only
            if cmd_id is not None:
                report({"cmdId": cmd_id})       # 2nd: cmdId
        elif args.bug == "number" and cmd_id is not None:
            report({**ac, "cmdId": int(cmd_id)})  # different type (number)
        elif args.bug == "nowrite":
            pass
        else:
            out = dict(ac)
            if cmd_id is not None:
                out["cmdId"] = cmd_id           # same value and type as received
            report(out)                         # combine into a single publish


def on_message(c, userdata, msg):
    body = json.loads(msg.payload or b"{}")
    with lock:
        if msg.topic.endswith("/update/delta"):
            process_delta(body.get("state", {}), "push")
        elif msg.topic.endswith("/get/accepted"):
            process_delta(body.get("state", {}).get("delta", {}), "get")
        elif msg.topic.endswith("/rejected"):
            log(f"rejected: {body}")


def on_connect(c, userdata, flags, reason_code, properties):
    log(f"Connected ({reason_code})")
    c.subscribe([(BASE + "/update/delta", 1), (BASE + "/get/accepted", 1),
                 (BASE + "/update/rejected", 1), (BASE + "/get/rejected", 1)])
    report(dict(ac))                            # report the current state
    c.publish(BASE + "/get", "", qos=1)         # fetch any delta that arrived while disconnected


client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2, client_id=THING)
client.tls_set(ca_certs=cfg["ca"], certfile=cfg["cert"], keyfile=cfg["key"],
               tls_version=ssl.PROTOCOL_TLS_CLIENT)
client.on_connect = on_connect
client.on_message = on_message
log(f"Connecting to {cfg['endpoint']} as {THING} (bug={args.bug}, poll={args.poll}s)")
client.connect(cfg["endpoint"], 8883, keepalive=60)
client.loop_start()
try:
    while True:
        time.sleep(args.poll if args.poll > 0 else 3600)
        if args.poll > 0 and client.is_connected():
            client.publish(BASE + "/get", "", qos=1)  # polling to recover missed pushes
except KeyboardInterrupt:
    log("Exiting")
    client.disconnect()

Key points:

  • In on_connect, it subscribes to delta and get/accepted, reports the current state, and then sends get
  • on_message processes the pushed delta (/update/delta) and the delta inside the get response (state.delta of /get/accepted) with the same process_delta()
  • It sends get every 5 seconds so that it can pick up pushes it missed (Chapter 6)
  • It remembers already-processed commands (done_ids) and does not execute the same command twice (Chapter 6)
  • If it sends more than --max-exec times (default 10), a safety device stops it on its own (Chapter 5)

3.6 Stopping

Stop terminals A and B with Ctrl+C. The same three terminals are used in the next chapter too.

ImportantSummary of this chapter
  • The device receives a delta → executes it → writes the received values back to reported
  • When it connects, it reports its current state and picks up anything missed with get
  • Operate while watching version and delta with send.py watch