Repository navigation
security: prevent account takeover and secure federation - #197
NetworkTheoryAppliedResearchInstitute merged 83 commits into
Conversation
NetworkTheoryAppliedResearchInstitute
left a comment
There was a problem hiding this comment.
Requesting changes on 3f8c128. Thanks, Calvin. This closes the account-takeover paths we found while reviewing #193, and most of it is solid. Two things block merge: the DCO check, and federation import of listings and transactions. Details below.
What we tested
- Commit: head 3f8c128. Its merge ref 8525c54 is main 9373310 plus this branch, with no conflicts against main.
- Baseline, main 9373310 run locally on Node 24: node:test 61/61, Jest 46/46 (19 suites).
- This PR, from CI run 41678759143 (backend-regression, Node 22) on 8525c54: node:test 60/60, Jest 65/65 (25 suites), and git diff --check is clean. Jest 65/65 also reproduced locally on a copy of 3f8c128.
- No new failures. The count changes make sense: five API-key chat retry tests were removed on purpose, one filter test went with the role/location filters, and there are 19 new Jest tests in 6 new suites.
- CodeQL on the merge ref shows 0 open alerts, and the 20 missing-rate-limiting alerts are fixed.
- We also ran the PR's own modules against an in-memory DynamoDB stand-in, and broke some protections on purpose to see whether the tests notice.
- Frontend not run: no frontend files changed.
What's good
- POST /users is admin-only with a verified JWT. It rejects body ids and unknown fields, generates the id on the server, creates conditionally, returns 409 on collision and returns the public DTO. It's well tested.
- Register reads only four fields, so id, role and verified can't come from the body.
- Public, account and admin DTOs are enforced twice, by ProjectionExpression and by allowlist.
- requireAdmin covers all of /api/admin, and the shared key alone can't pass.
- Federation:
- HMAC over the raw body, compared with timingSafeEqual.
- Per-peer secrets that fail closed.
- sourceNodeId comes from the authenticated peer.
- Signed export responses are checked against NodeRegistry.
- Exports use allowlists.
- Imported users are sanitized and created conditionally.
Hashes and contact fields no longer leave the node.
- /events, /stream and /uploads fail closed until #191/#192.
- This adds the repo's first backend test workflow, and the PR body is upfront about deployment prerequisites.
Blocking
- DCO: none of the 83 commits has Signed-off-by. DCO is a required check with no bypass, and squash merging won't fix it. On your machine, as Calvin Secrest dev@calvinsecrest.com:
git switch security/account-federation-hardening # at 3f8c128
git rebase --signoff 9373310
git rev-parse HEAD^{tree} # should print 01f68149c3a88bece9b7a7edd790f57b8881626c
git push --force-with-lease=security/account-federation-hardening:3f8c1285d00edbaf482992d409882c516ebc921c origin security/account-federation-hardening
- We tried this on a throwaway copy: no conflicts, same tree.
- Please don't use the DCO app's "Set DCO to pass" button. It waives the sign-off instead of adding it.
- Jodson needs to confirm that CLAUDE.md rule 4 (never force-push) allows this for an author's own unmerged branch. If it doesn't, squash onto a new branch instead (
git switch -c security/account-federation-hardening-signed 9373310 && git merge --squash 3f8c128 && git commit -s) and open a PR that supersedes this one. - Push before anyone approves, because a push dismisses approvals.
- Any code fixes should be signed commits (
git commit -s). The tree check above only applies to a push that adds sign-offs and nothing else.
- Federation import still overwrites local listings and transactions, and with this PR it happens in normal operation.
- backend/sync/importHelper.js:39-44 writes each peer listing and transaction whole: no allowlist, no sourceNodeId, no condition.
- On main this never ran, because the sync job posted to a route that doesn't exist. This PR makes the 6-hourly sync import for real (federation/federationSyncJob.js:60) and narrows the export (federation/federationRoutes.js:38-43). The export also re-sends rows a node imported earlier.
- Result: once two nodes sync both ways, each node's own marketplace transactions come back as {id, status} and replace the originals. That wipes buyerId, sellerId, the escrow flags and the dialog fields. Listings lose category, media, tags and availability.
- After that, release-escrow refuses that transaction for good, and with #196 both parties lose access to it.
- These writes happen before the users step, so they land even when the run then fails.
- sync/importHelper.test.js:15-17 asserts the unconditional put.
- This is the CLAUDE.md pair "drop provenance on federation import" and "silent last-write-wins without an audit trail", and it touches the escrow invariant.
Smallest fix: stop importing listings and transactions for now, and note that as an open item in CONFORMANCE.md. Fuller fix:
- per-type allowlists that never accept escrow, party or dialog fields;
- stamp sourceNodeId from the authenticated peer;
- export originNodeId and skip rows that originated locally;
- skip and log collisions row by row instead of throwing;
- validate the whole payload before the first write.
Either way, replace the test assertion with a round trip: A exports, B imports, B exports, A imports, and A's rows come out unchanged.
Should fix (here, or in a tracked follow-up before any deployment)
- Repeat syncs stop at the first user already imported. importHelper.js:45-52 throws on an existing id, and the export re-sends every user, so the second sync with any peer fails:
- users after the collision never import;
- lastSyncAt stops updating (federationSyncJob.js:62-67);
- POST /federation/import returns 409 after the listing and transaction writes have already happened.
In a two-way pair, a node's own users come back to it on the first pull. Fix:
- treat a row with the same sourceNodeId/remoteUserId as already imported (skip it, or update guarded on sourceNodeId);
- skip users that originated locally;
- add a test that syncs twice with non-empty users. Today's sync tests only use
users: [].
-
trust proxy is never set (server.js:11). The HTTPS rule for peers means running behind nginx or another TLS proxy (docs/FEDERATION_GUIDE.md:20). Every IP-keyed limiter would then key on the proxy address, so login/register and federation attempts become one budget shared by all clients. express-rate-limit won't warn, because the limiters use custom keyGenerators. Add a TRUST_PROXY setting (hop count or CIDR, unset by default, refuse
true), document it, and test it with X-Forwarded-For. -
The docs overstate Redis sharing.
- authRoutes.js:10-17 and the inline chat limiters use the in-memory store. With REDIS_URL set, login/register limits are still per process.
- docs/security-hardening.md:35 and .env.example:36-37 say the counters are shared and that startup fails without the store, but the code only checks that REDIS_URL is set.
- When Redis is down, limited routes return 500, and ioredis logs unhandled error events.
Build these limiters with createRequestLimiter, each with its own prefix (that also fixes the mixed draft-6/draft-8 headers), or correct the docs. Add a redis 'error' listener.
- Tests that don't guard the new protections:
- authRoutes.test.js is unchanged. Spreading req.body into register, or dropping its ConditionExpression or its 409 branch, all keep CI green. Add tests that a body id, role, verified or reputationScore is ignored, that the condition is set, and that a collision returns 409.
- Removing the final HMAC comparison in verifyPeerResponse (federationPeerAuth.js:109) keeps all 65 Jest tests green, because the negative test changes the node id and the signature together. Dropping digest, path or method from both signer and verifier also passes. Add fixed known-answer signature vectors for requests and responses, plus negative tests: tampered body, path or method; another peer's secret; timestamp ±301 s; a response with the right node id but a bad signature; a stale response.
- /conversations API-key rejection and the pre-auth limiters aren't tested through the real routes.
- Docs:
- security-hardening.md:55 says no upload-creation endpoint exists, but POST /messages/:conversationId with
file.datastill writes under backend/uploads. The fix is pending in #185/#191. - :29 cites a Locations table that only exists in #193. Here, coordinates live on Users rows, and the export projection is what keeps them out.
- :63 names #193's workflow. This PR adds backend-regression.
- Please also add a CONFORMANCE.md note (CLAUDE.md:5) covering: how peers are admitted in each direction, users being create-only with provenance, the listings/transactions open item, and reputation still crossing as one number.
-
streamConcurrency.js isn't mounted yet, but the docs tell #192 to use it. A reservation lasts until the token expires (7 days) and is released only when the stream closes, so a deploy or crash leaves slots taken for days, and logging in again doesn't help. Before #192 wires it, use a short lease renewed while the stream is open, capped at exp, and release on shutdown.
-
chatRateLimit.js:4-10 always uses the prefix
agrinet:chat, and rateLimit.js:62 exports an unused copy. With Redis, every route that uses it shares one counter with different limits, so 60 chat requests would also block /stream and /uploads. Require a per-route prefix and delete the unused export.
Federation policy: decided
- #194 set the opposite federation policy on the same files. No federation exists yet to decide between them, so Jodson has decided it as a founder-board bootstrap act (bylaws §16.1): this PR's stance stands, with scheduled import on and your HMAC peer auth. #194 should rebase onto this and keep its pagination. Because scheduled import stays on, item 2 has to be fixed before this merges.
Separately, reputationScore still crosses as one number, in the public /users DTO (models/user.js:23) and in the export (JFA line 8). That is a separate open question and isn't part of this decision.
Nits / follow-ups (non-blocking)
- requireAdmin.js:4 trusts the role inside a 7-day token, and POST /users can grant admin with no record of who did it. Before any role-change route lands, check the stored role. Also, authMiddleware returns on x-api-key before it reads the bearer token, so an admin client that sends both (frontend/user_login.js does) gets 403. Have requireAdmin verify the bearer token itself, like chatAuth does, and add one test through the real server.js middleware order.
- Registration doesn't enforce unique emails, and login takes the first match of one unpaginated Scan (authRoutes.js:55-60). This predates the PR and will get its own PR, which touches authRoutes.js and userRoutes.js and uses your 409 wording; whichever lands second rebases.
- /trends and the AI insight helper also read Users rows outside the new DTOs. This predates the PR and will get its own PR, so nothing is needed here.
- JWT_SECRET has no length check. Mirror the 32-character federation rule at startup, and pin HS256 in authMiddleware.
- .env.example:31-32: the placeholder secrets are long enough to pass the 32-character check. Use empty values, or reject
replace-with. - Federation signing, before any peer goes live:
- require different secrets per direction (security-hardening.md:17,23 currently allow one);
- add a version tag, both node ids and the sorted query string to the signed text (#194 adds query parameters);
- add stale/future timestamp tests.
- federationSyncJob.js:42-44: set
maxRedirects: 0,validateStatusto accept only 200, and a generousmaxContentLength. - Export sourceNodeId next to originNodeId/originUserId so downstream nodes can see a relay.
- /deposit is mounted (server.js:16) before apiRateLimit. Move it into the route list.
- security-hardening.yml: add a push trigger on main,
timeout-minutes, and current action majors (CI warns about Node 20). Mock ioredis in a test to cover RedisRateLimitStore. - Older docs still describe x-api-key chat, live /events and /stream, and REDIS_HOST/PORT (backend/README.md, docs/API_TESTING.md, docs/FEDERATION_GUIDE.md). A pointer to security-hardening.md would do. Also document how the first admin gets created.
- server.js lines 13 and 39 are 421 and 740 characters long. Please wrap them.
- PR body: please say it supersedes #186 and the auth-route move in #183, credit Nirmal, and list the conflicting PRs.
Overlap
This branch merges cleanly with #196 and conflicts with #183, #186, #191, #192, #193 (.env.example only), #194, #195 (server.js) and #161. #186 is fully covered here. #183's serverAuth.test.js is worth porting with the public-DTO expectation, because nothing in this PR checks that register and login work through the real server.
We'll re-test whatever new head you push. Any approval will be pinned to that commit.
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
…rite Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
…rite Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
…rite Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
…mits Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
… contract Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
…cation Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
Signed-off-by: Calvin Secrest <dev@calvinsecrest.com>
3f8c128 to
6b3e2fd
Compare
|
Follow-up verification on head I rechecked the outstanding federation-import blocker from the November 4 review. It is still present: Please keep #197 unmerged until that import path is disabled or made provenance-aware, allowlisted, collision-safe, validated before writes, and covered by the requested A→B→A round-trip regression test. Once pushed, request a fresh review on the new commit. |
NetworkTheoryAppliedResearchInstitute
left a comment
There was a problem hiding this comment.
Approving 6b3e2fd. Every commit now carries Calvin's sign-off. The code tree is identical to the head reviewed on 11-04 (01f6814), and DCO, CodeQL, hardcoded-urls and backend-regression all pass.
Review item 2 (federation import overwriting local listings and transactions) is waived as a merge condition, as a founder-board bootstrap act recorded in the governance registry (Entry 4, annotating Entry 3's 11-04 federation decision). It is still open and has to be fixed before the first peer node is registered. The other should-fix items from the 11-04 review stay open as follow-ups.
Resolve the backend/server.js conflict with #197 by taking main's server.js and reapplying this branch's three changes: require the JSON error handler, give the CORS refusal status 403 with an exposed message, and register the handler after every route. Full backend suite on the merge: node:test 64/64, Jest 65/65. Signed-off-by: Jodson Graves <info@ntari.org> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
AWS verification (re: Calvin's Nov 7 Slack question on the operating conditions for this waiver) Checked the NTARI AWS account across every region today (2026-11-10):
So both conditions hold: no federation peers are registered, and no federation HMAC credentials are configured — because there is no live Agrinet AWS deployment to hold either right now. Caveat: this covers the NTARI AWS account only. It does not cover NTARIHQ (the self-hosted environment NTARI has been migrating workloads to). If Agrinet's backend is actually running there, the equivalent check needs to be done against that environment before the first peer node is registered. Checked by: Jodson Graves (via Claude), 2026-11-10. |
Summary
This is a security-first remediation for the account takeover and federation data exposure identified during review of #193. It is based on current
mainand contains implementation and regression tests, not a patch artifact.Account and user data
POST /usersadministrator-only using a verified JWT principal; the shared browser/API key does not qualify as admin identity./api/auth/register, also using server-generated IDs and conditional writes./api/admin/usersand/api/admin/keyswith explicit administrator authorization and project only required fields.Federation
backend/federation/federationRoutes.js.sourceNodeIdfrom the authenticated peer rather than the request body. Apply an IP-based attempt budget before peer authentication and a peer-keyed budget after it.NodeRegistryidentity./federation/exportand the correct/federation/importcontract. The sync job verifies the peer response before importing.Rate limiting and temporary fail-closed behavior
express-rate-limitandioredis; limit by authenticated account, authenticated peer, or IP when no individual identity exists.RATE_LIMIT_REDIS_REQUIRED=truefor multi-instance deployments./eventsand/stream/:conversationIdendpoints with 410 responses and temporarily disable/uploadsdownloads. Do not re-enable these until Restrict private chats and attachments to their owners #191/Authenticate conversation streams and retire the global feed #192 protections are present..env.exampleandbackend/docs/security-hardening.md.Regression tests added
Verification status
6b3e2fd3f35bc8313a6e91f21ad7cdeccf4794e6(updated 2026-11-07).backend/sync/importHelper.jsstill imports peer listings and transactions using unconditional whole-row DynamoDBputcalls. There is no per-type allowlist, authenticated provenance stamp, origin-node skip, or collision-safe row handling for these records. The currentimportHelper.test.jsstill asserts those unconditional writes. Do not merge while scheduled federation import can overwrite local marketplace and escrow data.CONFORMANCE.mdhas not yet been updated to record the federation admission/provenance rules and the listings/transactions import limitation.CHANGES_REQUESTEDreview remains the active human review. After fixing the federation import boundary and updating its round-trip tests/conformance note, request a fresh review on the new head.Deployment and follow-up requirements
REDIS_URLplusRATE_LIMIT_REDIS_REQUIRED=trueif more than one backend instance runs.CORS_ALLOWED_ORIGINSto the exact deployed web/chat origins.FEDERATION_LOCAL_NODE_ID,FEDERATION_OUTBOUND_PEERS_JSON(outbound per-peer secrets), andFEDERATION_PEERS_JSON(authorized inbound peer secrets). Each peer pair must configure matching secrets on the sender's outbound and receiver's inbound side. Use distinct random secrets of at least 32 characters, keep them server-side, and use HTTPS between production peers.createConcurrentStreamLimit({ maximum: 5 })afterchatAuthand the shared chat limiter, beforeconversationStreams.open, replacing its process-local count. Restore upload serving only behindchatAuth, rate limiting, andattachmentOwner.Review requested
Please independently review administrator authorization, peer credential provisioning, federation provenance semantics, export allowlists, and the Redis deployment assumptions. Do not merge until CI is green and a different authorized maintainer approves.