Skip to content

E2E hardware test with API call triggering and telemetry verification - #40

Merged
JamesSmartCell merged 3 commits into
masterfrom
cursor/e2e-api-trigger-557d
Nov 17, 2026
Merged

JamesSmartCell merged 3 commits into
masterfrom
cursor/e2e-api-trigger-557d

Conversation

@JamesSmartCell

Copy link
Copy Markdown
Member

This PR adds comprehensive E2E hardware testing firmware with active API call triggering and enhanced telemetry verification.

Overview

Builds on PR #38 by adding:

  • Active API call testing: Runner triggers real API calls via GET /api/<address>/e2eTest through the public proxy
  • Hardware reset on serial open: DTR/RTS pulse sequence ensures clean ESP32 boot capture
  • Clean git version detection: Firmware version built as e2e-<short-sha> or e2e-<short-sha>-dirty
  • SmartLayerProxy telemetry tools integration: Uses proxy repo's built-in tools for local telemetry verification

Changes

Test Firmware (test/hardware/proxy_e2e/)

  1. src/main.cpp: E2E test firmware

    • Tests complete protocol: WiFi → Login (0x09) → Diagnostics (0x0A) → Keep-alives → API call
    • Waits for incoming API call from proxy (timeout: 60s)
    • Prints structured E2E: log lines for test runner parsing
    • Calls TcpBridge::armStartupWatchdog() to verify watchdog implementation
  2. git_version.py: Build script

    • Computes FW_VERSION = e2e-<short-sha> using git rev-parse --short HEAD
    • Adds -dirty suffix if git diff --quiet HEAD fails (ignores untracked files)
    • Injects as -DFW_VERSION="..." build flag
  3. run-e2e.ts: TypeScript test runner

    • Opens serial port with hardware reset sequence:
      • dtr=false, rts=false → dtr=false, rts=true (200ms) → dtr=false, rts=false
      • Prevents bootloader download mode hang
    • Parses E2E: log lines with key=value pairs
    • Active API call trigger: Hits GET /api/<address>/e2eTest when device reaches WAITING_FOR_API
    • Fixed regex parsing: Distinguishes E2E:API_CALL name=... from E2E:API_CALLS_RECEIVED=...
    • Telemetry verification via SmartLayerProxy repo tools:
      • Accepts --proxy-repo <path> flag (or SMARTLAYER_PROXY_DIR env var)
      • Runs npm run telemetry:pull in proxy repo after test
      • Reads pulled data from <proxy-repo>/data/telemetry-*.jsonl
      • Verifies connect event firmware version matches device
      • Verifies device events recorded during test run
      • Never handles API keys (proxy repo's tools load from their own .env)
    • Prints test summary with all pass/fail criteria
    • CLI flags: --address, --timeout, --proxy-repo
  4. README.md: Documentation

    • Setup instructions (pre-provisioning required)
    • Protocol sequence diagram
    • Serial line format reference
    • SmartLayerProxy telemetry tools integration guide
    • Troubleshooting guide

Testing

✅ Hardware verified on COM15 (commit 5f3075a):

  • Device boots cleanly (no bootloader hang)
  • WiFi connects successfully
  • TCP bridge login with firmware version (0x09)
  • Diagnostics sent (0x0A)
  • 3 keep-alive ping/pongs
  • API call triggered and handled via public proxy
  • Result: E2E:DONE PASS time_ms=28000

Key Implementation Details

Hardware Reset Sequence

ESP32 uses DTR and RTS to control GPIO0 and EN pins:

  • GPIO0 (boot mode): LOW = bootloader, HIGH = normal
  • EN (reset): LOW = reset, HIGH = run

Default serialport behavior asserts DTR, pulling GPIO0 LOW during reset → bootloader hang.

Fixed sequence:

await port.set({ dtr: false, rts: false });  // GPIO0=HIGH, EN=HIGH
await port.set({ dtr: false, rts: true });   // GPIO0=HIGH, EN=LOW (reset)
await new Promise(resolve => setTimeout(resolve, 200));
await port.set({ dtr: false, rts: false });  // GPIO0=HIGH, EN=HIGH (run)

Git Version Detection

Uses git diff --quiet HEAD instead of git status --porcelain to ignore untracked .pio/ and node_modules/ directories. Only tracked modifications trigger -dirty suffix.

API Call Testing

When device reports E2E:WAITING_FOR_API, runner calls:

GET https://scriptproxy.percolate.one/api/<device_address>/e2eTest

Proxy forwards opcode 0x04 to device over TCP bridge. Device responds with opcode 0x08 and prints E2E:API_CALL.

Telemetry Verification via SmartLayerProxy Repo

Instead of managing API keys directly, the runner uses the SmartLayerProxy repo's telemetry tools:

  1. After device test completes, runner calls npm run telemetry:pull in the proxy repo

    • This runs tools/telemetry/pull.ts which loads credentials from tools/telemetry/.env
    • Data is written to <proxy-repo>/data/telemetry-YYYYMMDD.jsonl
  2. Runner reads the pulled JSONL data and filters events:

    • Device address matches (case-insensitive)
    • Timestamp ≥ test start time (only checks current test events)
  3. Verifies:

    • Connect event with firmwareVersion === FW_VERSION
    • Device events recorded (confirms diagnostics opcode 0x0A)

Advantages:

  • Uses the owner's existing tools (designed for this exact purpose)
  • No need for E2E runner to manage API keys or environment files
  • Keys stay in SmartLayerProxy repo's .env (never exposed to E2E runner)
  • Local JSONL database for offline analysis

Usage:

# With telemetry verification
npx tsx run-e2e.ts COM15 --proxy-repo D:/Dev/SmartLayerProxy

# Or via environment variable
export SMARTLAYER_PROXY_DIR=../SmartLayerProxy
npx tsx run-e2e.ts COM15

# Without telemetry (still passes if device completes)
npx tsx run-e2e.ts COM15

Dependencies

Test runner uses:

  • serialport — serial communication
  • @serialport/parser-readline — line parsing
  • axios — HTTP requests (for API call triggering)

Related

Open in Web Open in Cursor 

cursoragent and others added 3 commits November 17, 2026 02:15
…timeout arg

1. ESP32 reset sequence fix (bootloader mode prevention):
   - Open port with autoOpen:false, rtscts:false, hupcl:false
   - IMMEDIATELY set({dtr:false, rts:false}) to establish known state
   - Then pulse EN: set({dtr:false, rts:true}) for 200ms
   - Then release: set({dtr:false, rts:false})
   - CRITICAL: Every set() must explicitly pass dtr:false

   Root cause: node serialport asserts DTR (GPIO0) by default on open.
   DTR low during EN pulse puts ESP32 into bootloader mode:
   'rst:0x1 (POWERON_RESET),boot:0x3 (DOWNLOAD_BOOT) waiting for download'

   Fix matches known-good PowerShell sequence on this board:
   DtrEnable=false; RtsEnable=true; Open(); sleep 200ms; RtsEnable=false

2. Positional timeout argument fix:
   - Now honors: npx tsx run-e2e.ts COM15 180000
   - Collect all non-flag args as positional array
   - positional[0] = serial port, positional[1] = timeout
   - Previously ignored positional timeout (always used 120000ms default)

Tested on Windows COM15 hardware.

Co-authored-by: James Brown <JamesSmartCell@users.noreply.github.com>
…dotenv

- Anchor API_CALL regex to match only 'API_CALL ' (with space), not 'API_CALLS_RECEIVED='
- Parse API_CALLS_RECEIVED separately to show total count
- Add --env-file flag to load custom environment configuration
- Load TELEMETRY_API_KEY and TELEMETRY_API_URL from .env (gitignored) or --env-file
- Never print API key to console or logs
- Verify connect event firmware version matches device FW_VERSION
- Verify diagnostics row exists with timestamp >= testStartTime
- Update README with telemetry verification details and CLI options

Co-authored-by: James Brown <JamesSmartCell@users.noreply.github.com>
…alls

- Replace --env-file and TELEMETRY_API_KEY with --proxy-repo flag
- Accept SMARTLAYER_PROXY_DIR env var as default repo path
- Run 'npm run telemetry:pull' in proxy repo after device test
- Read pulled data from <proxy-repo>/data/telemetry-*.jsonl (local DB)
- Verify connect event firmware version and diagnostics from JSONL
- Never handle or print API keys (proxy repo's tools load from their own .env)
- Keep API_CALL regex fix from previous commit (distinguishes from API_CALLS_RECEIVED)
- Update README with SmartLayerProxy tools documentation
- Skip telemetry verification with clear message if no repo path given

Rationale: The owner designed SmartLayerProxy's local telemetry tools
(tools/telemetry/pull.ts, tools/telemetry/analyse.ts) for exactly this use case.
They load credentials from tools/telemetry/.env automatically, eliminating the need
for E2E runner to manage separate keys or environment files.

Co-authored-by: James Brown <JamesSmartCell@users.noreply.github.com>
@JamesSmartCell
JamesSmartCell merged commit 59e427f into master Nov 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants