Skip to content

Implement streamed HTTP and verified HTTPS client library - #1

Closed
JaxCalder wants to merge 5 commits into
mainfrom
feat/http-https-client
Closed

JaxCalder wants to merge 5 commits into
mainfrom
feat/http-https-client

Conversation

@JaxCalder

@JaxCalder JaxCalder commented Oct 17, 2026 •

Copy link
Copy Markdown

Implement a DMOD HTTP/1.1 download library with streamed GET/HEAD over a nonblocking connector and verified dmtls. Initialize with the standard library/GitHub generator, without a driver/port layer.

The state machine handles Content-Length, chunked framing/extensions/trailers, EOF-delimited bodies, relative redirects, byte ranges, cancellation, bounded parser storage and a finite total transaction deadline. It rejects ambiguous/malformed framing, truncation, unsupported encodings, certificate/hostname failures and HTTPS downgrades. Normal completion and redirects send TLS close_notify. Applications own connector readiness, entropy/UTC and trust-store provisioning; the body is never accumulated in RAM. The interface covers GET/HEAD, with scope/ownership documented in include/dmhttp.h and docs/api.md.

Production dependencies are unversioned dmosi and dmtls. SDK/loader and 240-byte DMF format are unchanged. There is no custom pointer audit or additional linker flags; request callbacks are stored per instance. Native/M4/M7 builds have been validated. M7 image is 10368 bytes DMF / 8031 bytes DMFC, plus request state and separate TLS/transport memory.

Validation:

  • Actual module through dmod_loader: 4/4 on x86-64 and STM32F746G-DISCO, including HTTP 404 status/body handling.
  • Production HTTP + real dmtls backend against independent OpenSSL and fragmented/error streams: 29/29 tests, including a 2 MiB streamed binary body and close_notify.
  • MCU UART/OpenSSL scenarios: 9/9, covering HTTP framing/redirects, TLS 1.2/1.3 HTTPS, HTTPS reconnect, hostname/CA failures and downgrade rejection. This path is explicitly distinguished from Ethernet tests.
  • Real Ethernet/dmtcp: three complete HTTP downloads and 3/3 verified HTTPS downloads; lengths/hashes match Python's independent client. A separate 65536-byte HTTPS transfer also matched its reference (one full-firmware run took 50.25 seconds including handshake).
  • Post-network regressions: UTC 6/6, RTC 12/12, RNG 6/6; UTC restored and library-service unit running.
  • Reproducible real-TCP and UART/OpenSSL diagnostics are included; the TCP worker uses a semaphore plus buffered-readiness checks, not a network polling loop.

The validated MCU configuration includes choco-technologies/dmtls#2. Some longer ECC-chain public-server attempts remain intermittent after slow handshakes; this is recorded separately in choco-technologies/dmtls#3, not presented as a passing case. Verification remains mandatory. Raw logs and private fixture keys stay local. Renode was omitted as requested.

Registry: choco-technologies/dmod-registry#21. Ecosystem: choco-technologies/dmod-ecosystem#33. Publish first releases before registry binary installation.

CI and release workflows use the standard new-module.sh library/GitHub templates: native CI runs dmf-get/dmod_loader, and release architectures are discovered automatically. The only added steps install test dependencies and run HTTP interoperability CTest. No Makefile wrapper is included.

The installed dmf-man dmhttp page covers setup, options, request lifecycle, event scheduling, HTTP statuses, resuming downloads and troubleshooting. dmf-man dmhttp api documents function contracts and local errors. Both pages were checked through the actual viewer from the generated package layout. Named HTTP status constants preserve standard numeric values; local operation results remain separate.

@JohnAmadis JohnAmadis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JaxCalder Do poprawy

Comment thread .github/workflows/ci.yml Outdated
strategy:
fail-fast: false
matrix:
tools: [arch/x86_64, arch/armv7/cortex-m4, arch/armv7/cortex-m7]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nie wpisujemy zadnego matrixa recznie. Kod przeciez nie jest machine specific wiec nie ma sensu marnowac czasu CI na budowanie na kazda architekture

Comment thread .github/workflows/release.yml Outdated
contents: write
strategy:
matrix:
arch_name: [x86_64, armv7/cortex-m4, armv7/cortex-m7]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

od tego masz skrypt new-module.sh aby on poprawnie zainicjalizowal CI. To jest zle

Comment thread CMakeLists.txt
)

dmhttp_audit_module(${DMOD_MODULE_NAME})
dmod_link_modules(${DMOD_MODULE_NAME} dmosi dmtls)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

co to jest ten audit module?

Comment thread cmake/pointer-audit.cmake Outdated
@@ -0,0 +1,11 @@
find_package(Python3 REQUIRED COMPONENTS Interpreter)
function(dmhttp_audit_module target)
# Metadata is checked at build time and excluded from the binary by objcopy.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to nie jest potrzebne. Jesli masz problemy z globalbymi pointerami to je usun. Jesli uzywasz zewnetrznych bibliotek to zrob ich forka i z nich tez to usun

Comment thread docs/README.md Outdated
@@ -0,0 +1,4 @@
# dmhttp documentation

- [API and contracts](api.md)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to jest glowna dokumentacja, ktora uzytkownicy zobacza po wpisaniu dmf-man. Serio?! uwazasz ze ma tak wygladac? czy ty w ogole sie zainteresowales czym jest ten dmf-man i po co ten folder w ogole istnieje?!

Comment thread include/dmhttp.h
DMHTTP_ERR_PROTOCOL = -7,
DMHTTP_ERR_LIMIT = -8,
DMHTTP_ERR_TRUNCATED = -9,
DMHTTP_ERR_TIMEOUT = -10,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

czemu one sie nie zgadzaja ze standardem? np z tego co pamietam 404 to not found a nie widze go tu

@JohnAmadis JohnAmadis closed this Oct 17, 2026
@JohnAmadis

Copy link
Copy Markdown
Contributor

tak brzydkiej implementacji to ja dawno nie widzialem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants