BenchPod pytest framework

Install

pip install embeddedci                 # Python 3.10+

# optional extras (combine them, e.g. "embeddedci[cloud,pytest]"):
pip install "embeddedci[pytest]"       # installs pytest alongside the plugin
pip install "embeddedci[cloud]"        # the embeddedci:<name> cloud destination
pip install "embeddedci[discovery]"    # find a pod on the LAN via mDNS
pip install "embeddedci[analysis]"     # numpy for Capture.fft / dominant_frequency

Needs Python 3.10 or newer. The core package covers local network and USB pods; the pytest plugin registers itself on install. [cloud] adds the WebSocket client the embeddedci: destination needs, [discovery] adds mDNS for discover, [analysis] adds numpy for the FFT helpers, and [pytest] simply installs pytest too.

OpenOCD (required for flashing)

Flashing shells out to OpenOCD, which must be on your PATH (or passed as openocd_bin=). The pod runs a CMSIS-DAP probe in its firmware, and the SDK drives it through OpenOCD's cmsis-dap adapter with its TCP backend (cmsis_dap_tcp), which ships whole DAP transfers instead of per-bit toggles. That backend only exists in OpenOCD builds newer than 0.12.0. The stock packages (apt install openocd, brew install open-ocd) are 0.12.0 and lack it. The SDK checks before it flashes and raises a FlashError telling you to update.

# Homebrew's stable open-ocd is 0.12.0, so build it from HEAD
brew install --HEAD open-ocd

# the xPack OpenOCD 0.12.0-7 snapshot carries the TCP backend
V=0.12.0-7
curl -fsSL "https://github.com/xpack-dev-tools/openocd-xpack/releases/download/v$V/xpack-openocd-$V-linux-x64.tar.gz" \
  | sudo tar xz -C /opt
export PATH="/opt/xpack-openocd-$V/bin:$PATH"

# PowerShell: the xPack OpenOCD 0.12.0-7 snapshot carries the TCP backend
$V = "0.12.0-7"
Invoke-WebRequest "https://github.com/xpack-dev-tools/openocd-xpack/releases/download/v$V/xpack-openocd-$V-win32-x64.zip" -OutFile openocd.zip
Expand-Archive openocd.zip -DestinationPath "$env:LOCALAPPDATA\xpack"
$bin = "$env:LOCALAPPDATA\xpack\xpack-openocd-$V\bin"
[Environment]::SetEnvironmentVariable("Path", [Environment]::GetEnvironmentVariable("Path", "User") + ";$bin", "User")
# open a new terminal so it picks up the PATH

Verify your OpenOCD has the CMSIS-DAP TCP backend:

openocd -c "adapter driver cmsis-dap" -c "cmsis-dap backend tcp" -c "cmsis-dap tcp port 4441" -c shutdown
# must exit cleanly: an error on the 'backend tcp' line means the build is too old

Flash a build uploaded to embeddedci.com

A build uploaded with the upload-artifact action or the build_report fixture (see Cloud builds) is flashed by naming it instead of passing file: build is the build id the upload printed (or "latest"), repo and ref pick the newest build of a branch or tag, and artifact names a file when the build holds several (else its .elf, then .hex, then .bin).

# the newest build of main, uploaded by CI:
benchpod.flash(build="latest", repo="acme/firmware", ref="main", target_power=INTERNAL)

# or the exact build a workflow uploaded (its build_id output):
result = benchpod.flash(build=os.environ["BUILD_ID"], check=False)
assert result.ok, result.stdout
print(result.artifact["name"], result.artifact.get("sha"))

On a cloud connection (embeddedci:<device>) embeddedci.com runs OpenOCD through the pod, so the runner needs no OpenOCD and a GitHub Actions OIDC session is enough; such a flash always verifies and resets. On a LAN or USB connection the file is downloaded and flashed by the local OpenOCD, which needs an API key or benchpod login. server_side= picks one explicitly, and FlashResult.artifact says which build was flashed.

Using the SDK directly

Outside of pytest, drive a pod with the BenchPod context manager:

from embeddedci.benchpod import BenchPod, INTERNAL, PIN1, PIN2

# la_voltage selects the LA I/O-bank voltage: the board's I/O voltage, 1.8 for a 1V8 board.
with BenchPod("192.168.1.213", la_voltage=3.3) as bp:   # or "/dev/ttyACM0", "usb", "embeddedci:<name>"
    bp.ping()                                           # raises if the pod doesn't answer
    result = bp.flash(
        file="firmware.elf", target="target/stm32f1x.cfg",
        swclk=PIN1, swdio=PIN2,
        nreset=True,              # a flag: the target's reset is wired to the pod's reset pin (J1 pin 22)
        target_power=INTERNAL,    # power the target from the internal 5 V eFuse first
    )
    assert result.ok              # a failed flash raises FlashError; pass check=False to inspect it instead
    assert not bp.target_status().efuse(INTERNAL).fault   # eFuse not tripped
    print(bp.power_status().rail(INTERNAL).current, "A")  # INA monitor on that rail
    bp.power_off(INTERNAL)

Named constants avoid magic numbers: INTERNAL / EXTERNAL for the target-power eFuse, and PIN1 … PIN14 for logic-analyzer channels, all importable from embeddedci.benchpod (plain ints still work; an out-of-range value raises ValueError). Units are volts, seconds and hertz throughout, pod state comes back as typed results, and a pod, transport or server failure raises a BenchPodError subclass (see Errors).

Using it in pytest

Installing the package registers a pytest plugin. Point it at a pod and use the fixtures:

pytest --benchpod-connection=192.168.1.213
# or: export BENCHPOD_CONNECTION=usb

# add the firmware image for tests that flash a target:
#   --benchpod-firmware=build/app.elf

# conftest.py
import pytest

@pytest.fixture(scope="session")
def benchpod_la_voltage():
    return 3.3  # the board's I/O voltage — change to 1.8 for a 1V8 board

import pytest

# The pod exposes 14 generic LA channels (pins.pin_1 .. pins.pin_14); it has no
# dedicated SWD/UART pins. Which DUT signal is on which channel is the pod's
# wiring profile (the web app's Wiring tab, or --benchpod-wiring=wiring.json),
# which is benchpod.wiring — so flash() needs no pin numbers at all.
@pytest.mark.hardware
def test_firmware_flashes(benchpod, firmware):
    wiring = benchpod.wiring
    assert benchpod.flash(
        file=firmware, target="target/stm32f4x.cfg",
        swclk=wiring.swd_swclk, swdio=wiring.swd_swdio,   # the defaults, spelled out
        target_power=wiring.efuse,
    ).ok


# benchpod_target powers the target on for the test and off at teardown:
@pytest.mark.hardware
def test_rail_is_healthy(benchpod_target, pins):
    assert not benchpod_target.target_status().efuse(pins.efuse).fault


# skipped unless the connected pod advertises the capability:
@pytest.mark.benchpod_capability("dac_deep_replay")
def test_long_replay(benchpod_dac):
    ...

Fixtures

Markers: @pytest.mark.hardware labels a test that needs a real pod, and @pytest.mark.benchpod_capability("name") skips it unless the connected pod advertises that capability (e.g. dac_deep_replay).

Options

Connection strings

Resolution order: --benchpod-connection → the benchpod_connection ini option → the BENCHPOD_CONNECTION env var → --benchpod-discover. The CLI's saved connection only with --benchpod-connection=saved (see One saved connection). A cloud pod is shared, so the SDK holds an exclusive lease on it for the session; a run that finds it busy queues for up to --benchpod-lease-wait seconds.

Wiring and connection defaults

The SDK, the MCP server and the CLI read the same wiring profile and the same saved connection, so a pod you set up once behaves the same from each of them.

Both need embeddedci 2.10 or newer.

One wiring profile

bp.wiring is resolved once, when the session connects. The first match wins:

1. the wiring= argument (in pytest: --benchpod-wiring, then an overridden benchpod_wiring fixture);
2. the file named by BENCHPOD_WIRING;
3. the profile stored on embeddedci.com for this pod, the one the Wiring tab of the pod's workspace shows. For embeddedci:<name> that is the named pod; over the LAN it is the pod the pod itself reports it is registered as. It needs an API key (api_key= or BENCHPOD_API_KEY) or the benchpod login session, with the pod on the same account. bp.wiring.source is then server;
4. the defaults.

Step 3 never fails the connection: when it cannot read the profile (on the LAN it gives up after about 3 seconds) it logs why and the defaults apply. Over USB the stored profile is not used, so pass the wiring yourself there. BenchPod(..., wiring_source="local") (the default is "auto") or BENCHPOD_WIRING_SOURCE=local skips step 3.

One saved connection

BenchPod() picks its connection from the argument, then BENCHPOD_CONNECTION, then the default connection the CLI saved (benchpod set-connection or benchpod discover --save) in ~/.config/benchpod-cli/config.json (or $XDG_CONFIG_HOME/benchpod-cli/config.json). The connection string saved names that default explicitly. Older releases raised ConnectionConfigError when no connection was given.

The pytest plugin never uses the saved connection on its own, so a test run only drives a pod you named: it takes --benchpod-connection, then the benchpod_connection ini option, then BENCHPOD_CONNECTION, then --benchpod-discover. Opt in with --benchpod-connection=saved; the hardware tests skip with a reason when nothing is saved.

Emulating an I2C sensor + capturing UART

The pod can pretend to be an I2C sensor (a BMP280) on two channels while you capture the DUT's UART, so you can flash an app, power-cycle it, and assert on its boot output with and without the sensor present. It also decodes the bus (i2c_sensor_capture), so you can assert what the firmware actually did on the wire (see Emulating an I2C Sensor).

from embeddedci.benchpod import BenchPod, INTERNAL, PIN1, PIN2, PIN5, PIN6, PIN11, PIN12, Sensor

with BenchPod("192.168.1.213", la_voltage=3.3) as bp:   # board I/O voltage; 1.8 for a 1V8 board
    bp.flash(file="app.elf", target="target/stm32f4x.cfg",
             swclk=PIN11, swdio=PIN12, target_power=INTERNAL)

    # I2C is open-drain: enable the pod's pull-ups on SDA/SCL. Only LA1-LA6 have
    # pull-ups (LA7/LA8 are pull-downs), so the I2C lines go on LA1/LA2.
    bp.enable_pullup(PIN1, PIN2)
    bp.enable_i2c_sensor(Sensor.BMP280, sda=PIN1, scl=PIN2,
                         temperature_c=22.5, pressure_pa=101000)

    # Power-cycle while capturing UART so the boot banner lands in the window.
    cap = bp.power_cycle_and_capture(rx=PIN5, tx=PIN6,
                                     delay=1.5, duration=6.0, until=r"APP_OK")
    assert cap.match("APP_OK")

The rest of the SDK

• Power & reset: power_on(efuse, delay=1.5) (scheduled pod-side), target_status() (eFuse enabled / tripped), power_status() (rail volts and amps), reset_target(pulse=0.1).
• UART: capture_uart(...) for a fixed window, open_uart(...) for a live two-way session, power_cycle_and_capture(...) for a boot banner.
• Logic analyzer: capture_la(n, sample_rate_hz=...) returns an LaCapture you can .decode("i2c" | "uart" | "spi", ...); la_step(la, steps=, delay=) emits step/dir pulse trains.
• Analog: named paths (analog_path("cal1")), adc_read("ext").voltage, capture_adc(...) with calibrated volts, calibrate() to store the offset of the 4-20 mA input on the pod (J8 disconnected), current_out(0.012) to hold 12 mA on the 4-20 mA output (J9, which needs an external floating loop supply), dac_output("5v", volts=2.5), generate("sine", freq_hz=, amplitude=), replay(capture, fault=Fault(...)), and capture_correlated(...) for ADC + LA from one hardware trigger. See Recording and Replaying Analog Signals.
• Sensors and GPS: enable_i2c_sensor(Sensor.BME280, humidity_pct=55) emulates a BMP280, BME280, SHT4x or MPU-6050 (i2c_sensor_types() lists each model's values); enable_gps("gps_rx", latitude_deg=..., speed_kmh=...) plays a GPS receiver printing NMEA on a second UART, and set_gps(...) moves the fix.
• GPIO: bp.gpio("READY", "input") or bp.signal("READY") claims a channel by number or wiring name and returns a pin you can read, set, wait_for and pulse; release_gpio(...) frees it. enable_pullup(...) engages the bias resistors.
• SPI: open_spi(hz=6_000_000) returns an SPI master session on four LA channels (transfers, flash_id(), flash_program(...)); spi_flash("fw.bin") erases, writes and verifies an SPI NOR flash in one call.
• CAN: open_can(bitrate=500_000, mode=...) returns a CanBus with write, expect, and firmware-side auto-responders for ECU simulation.
• Control loop: control_loop(curve=...) runs a tabulated transfer function in the FPGA, e.g. a solar-panel I-V curve.

from embeddedci.benchpod import BenchPod

with BenchPod("192.168.1.213", la_voltage=3.3) as bp:   # board I/O voltage; 1.8 for a 1V8 board
    print(bp.adc_read("ext").voltage)                     # one calibrated reading on the front SMA
    cap = bp.capture_adc(8192, sample_rate_hz=100_000)    # Capture: .volts, .mean(), .peak_to_peak()
    with bp.replay(cap, dac_path="5v"):                   # loops on the DAC until the block exits
        la = bp.capture_la(8192, sample_rate_hz=1_000_000)

    with bp.open_can(bitrate=500_000, mode="internal") as can:   # loopback self-test, no bus needed
        can.write(0x123, [1, 2, 3])
        assert can.expect(can_id=0x123, timeout=1.0).data == bytes([1, 2, 3])

Everything exported from embeddedci.benchpod is covered by semantic versioning: within 2.x, names and signatures only grow. bp.command(...), bp.transport and bp.lowlevel are escape hatches outside that promise.

Errors

Invalid arguments raise ValueError before anything is sent. Everything else raises a BenchPodError subclass, all importable from embeddedci.benchpod. The refusal classes (PodLockedError, PodBusyError, PodLeasedError, PermissionDeniedError) also subclass what a refusal raised before them, so older except FirmwareError or except DeviceBusyError handlers still catch them.

API reference

Every name in embeddedci.benchpod, by area. The SDK README has the signatures, the fields of each type and longer examples.

BenchPod

BenchPod(connection=None, *, la_voltage=None, timeout=30.0, api_key=None, api_base=None, lease=True, lease_wait=600.0, lease_ttl=120, wiring=None, wiring_source="auto", ...) is a context manager. With no connection it uses BENCHPOD_CONNECTION, then the CLI's saved connection. An argument you leave out (a channel, the baud, the power rail, the SWD target) comes from the wiring profile.

Types

Results are frozen dataclasses that keep the firmware's reply in raw; sessions such as UartSession, SpiSession and CanBus are context managers.

Capabilities

bp.capabilities (and the benchpod_capabilities fixture) is parsed from the pod's status, or from the server's cap.* map for a cloud pod. A flag that was not reported keeps its default (False, 0 or ""), so a feature check is if bp.capabilities.spi_master:, and @pytest.mark.benchpod_capability("name") takes the same names. On the pod itself, bp.supports("spi_master") checks any flag, and bp.can_spi, bp.can_trigger, bp.can_analog and the other can_* properties name the common ones (can_config … can_disable stay the CAN bus commands). A call the pod cannot serve raises UnsupportedFeatureError.

ServerApi

A thin client for the embeddedci server's HTTP API, for scripts that need the server rather than a pod. bp.server_api returns one bound to the connection's credentials, or build one with ServerApi(api_key=..., api_base=...). Failures raise ServerApiError.

Functions, constants and modules

• parse_connection(raw) and resolve_connection(connection=None) turn a connection string (or BENCHPOD_CONNECTION) into a ConnSpec.
• make_build_reporter(...) returns the build_report fixture's reporter outside pytest.
• Control-loop curves: build_panel_curve, build_linear_curve, build_constant_curve, curve_output_at, input_percent_to_code and encode_curve_b64url.
• Constants: INTERNAL, EXTERNAL, PIN1 … PIN14, BMP280_ADDR_PRIMARY, BMP280_ADDR_SECONDARY.
• Submodules: decode (off-device decoders), i2c, can, control_loop, motor_emulator (the emulator's registers) and dsp (the record-to-replay processing the server applies).

OpenHTF

The embeddedci-openhtf package wraps the same SDK as an OpenHTF plug and ready-made phases, for stations that want OpenHTF's sequencing, limits and records. It talks to a pod directly over TCP or serial (no cloud account), and benchpod_plug(..., persistent=True) keeps one connection open across DUTs (persistent plugs with the same connection settings share it).

pip install embeddedci-openhtf          # pulls in embeddedci + openhtf

import openhtf as htf
from embeddedci_openhtf import benchpod_plug, boot_banner_phase, flash_phase, power_phase

bench = benchpod_plug("192.168.1.50:8080", la_voltage=3.3)   # or "/dev/ttyACM0"

test = htf.Test(
    power_phase(bench, on=True),
    flash_phase(bench, file="fw.elf", target="target/stm32f4x.cfg",
                swclk=11, swdio=12, nreset=True),   # records flash_ok, attaches openocd.log
    boot_banner_phase(bench, rx=1, tx=2, expect="APP_OK", duration=5.0),
)
test.execute(test_start=lambda: "SN-0001")