Skip to content

Add client-side runner - #106

Open
allsey87 wants to merge 4 commits into
exercism:runner-script-fixesfrom
allsey87:clientside-runner
Open

allsey87 wants to merge 4 commits into
exercism:runner-script-fixesfrom
allsey87:clientside-runner

Conversation

@allsey87

@allsey87 allsey87 commented Oct 5, 2026 •

Copy link
Copy Markdown

Runs bin/run.sh in the browser on the wasm kernel, as jq-test-runner does.

  • exercism_parser is compiled in GHA inside the kernel by the sysroot's clang, linked without --export-all and run through wasm-opt (55KB, reproducible). bin/build-clientside-parser.sh does this.

  • Publish runs the suite in the kernel against the exact tarball it uploads.

  • run-in-kernel.mjs gains --extract, supports precompile and the canvas handlers newer kernels require.

  • bin/run-tests.sh now fails when results differ ($? was read after rm), and compares build errors by status only, since GCC and clang word them differently.

  • helpers.h drops std::ptr_fun (removed in C++17, absent from libc++). example-tbb and libtbb-dev are removed.

  • The workflows fetch with curl --compressed (artifacts are stored brotli-encoded), and run-in-kernel.mjs passes the kernel's fetches of stubbed binaries (/test-runners/...overlay/...) through to exercism.org, so CI runs on exactly what students get.

The client-side check needs exercism/clientside-tooling#3 merged and published first.

Closes #104

Merge order

  1. https://github.com/exercism/clientside-tooling/pull/3: publishes the cpp kernel and sysroot.
  2. Accept precompile in boot.json, add canvas handlers website#10692: must be deployed before any track's boot.json uses precompile.
  3. Add client-side runner #106 (this PR): its client-side check needs 1 published and 2 deployed (it fetches the brotli-encoded objects), and its first publish writes cpp/latest.json, so it merges last.

@vaeng

vaeng commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Am I right to assume that you still work ok that, until all tests are green?

@allsey87

allsey87 commented Oct 6, 2026

Copy link
Copy Markdown
Author

I think the test is only red because of the dependent PRs

@ahans

ahans commented Oct 7, 2026

Copy link
Copy Markdown

Started a discussion in the forum, maybe you can weigh in there, @allsey87. The parts affecting just C++ we can discuss here.

  1. Is the Docker runner kept or will this eventually go away? If it is kept, I think it's important to keep builds for Docker as well as the browser in sync where possible.
  2. Related, I understand the tests requiring TBB are removed because that is not supported by the browser version. We have those tests to make sure an exercise that involves threading behaves identically when run locally as well as via the online runner. Removing the tests here could make a regression in the Docker image go unnoticed somewhen in the future. To keep things in sync, TBB would have to be removed from the Docker image again, but then maybe those exercises involving parallelism should be removed as well.

@allsey87

allsey87 commented Oct 7, 2026

Copy link
Copy Markdown
Author

Started a discussion in the forum, maybe you can weigh in there, @allsey87. The parts affecting just C++ we can discuss here.

  1. Is the Docker runner kept or will this eventually go away? If it is kept, I think it's important to keep builds for Docker as well as the browser in sync where possible.
  2. Related, I understand the tests requiring TBB are removed because that is not supported by the browser version. We have those tests to make sure an exercise that involves threading behaves identically when run locally as well as via the online runner. Removing the tests here could make a regression in the Docker image go unnoticed somewhen in the future. To keep things in sync, TBB would have to be removed from the Docker image again, but then maybe those exercises involving parallelism should be removed as well.
  1. On whether the Docker runner will be kept, that is really Jeremy's call to make. Regarding keeping things in sync, I align what I am doing to Ubuntu 26.04 and try to build things the same way. In this PR series, for example, I actually run the kernel in GHAs to compile exercism_parser to match the Docker workflow.
  2. Curious what behaves identically here means since threads can start and finish in any order, including sequentially and will differ between OS versions, load, uptime etc etc. We can add TBB back, but the execution policy for transform in the reference solution should be par and not par_unseq to the best of my knowledge.

@iHiD iHiD mentioned this pull request Oct 22, 2026
@iHiD
iHiD force-pushed the clientside-runner branch from 77b7798 to 395fbb0 Compare October 22, 2026 04:13
@iHiD
iHiD changed the base branch from main to remove-tbb October 22, 2026 04:13
@iHiD
iHiD force-pushed the clientside-runner branch from 395fbb0 to 69b8e6b Compare October 22, 2026 04:25
@iHiD
iHiD changed the base branch from remove-tbb to runner-script-fixes October 22, 2026 04:25
@iHiD
iHiD force-pushed the clientside-runner branch from 69b8e6b to 54b3cf0 Compare October 22, 2026 05:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants