Skip to content

chore: absorb abuse into packages/abuse - #13954

Merged
ChiragAgg5k merged 499 commits into
mainfrom
chore/absorb-abuse
Sep 29, 2026
Merged

ChiragAgg5k merged 499 commits into
mainfrom
chore/absorb-abuse

Conversation

@ChiragAgg5k

Copy link
Copy Markdown
Member

Summary

Moves utopia-php/abuse into packages/abuse so Appwrite loads it directly. Wave 4 of #13828, following the playbook in rfc/monorepo.md. abuse is a standalone repository, so it gets the full playbook. The mirror head (821a188) is the 2.0.1 release Appwrite already locks, so there is no version gap and Appwrite runs the same code.

Caution

The mirror's main still has classic branch protection that will reject the post-merge Split push. utopia-php/abuse main requires 1 approving review and restricts pushes to the owners team and @abnegate; the split app is not an allowed actor and classic protection has no bypass for it (admins are not enforced, but the app is not an admin). The new ruleset already lets the app bypass, so an org owner must delete the classic protection on utopia-php/abuse main before this PR merges. Not changed by this PR.

Merge with a merge commit, not a squash.

Commits

Commit What
Add 'packages/abuse/' from commit '821a188…' Subtree import from the mirror main with full history
chore(abuse): mirror plumbing mirror.yml replaces tests.yml, linter.yml, codeql-analysis.yml, bench.yml; issue templates, pint.json, composer.lock, Dockerfile.php-8.4 and Dockerfile.php-8.5 removed; hoisted QA stripped from composer.json; README banner added and the dead Travis badge dropped
refactor: load abuse from packages/ Root wiring, step A shape, e2e compose, QA config, RFC baseline line

What changed

Area Change
Root composer.json Autoload, autoload-dev and replace entries added; utopia-php/abuse: 2.0.* removed from require
Hoisting appwrite/appwrite: ^27.1 (the PHP SDK the TablesDB adapter uses, already locked at 27.1.0) added to root require. ext-pdo, ext-curl, ext-redis were already there; utopia-php/database stays vendored and utopia-php/pools is in packages/
composer.lock Only utopia-php/abuse removed (+1/−54, content hash)
src/ src/Abuse/* moved up to src/. Byte-identical to 2.0.1 otherwise (Pint made no changes)
tests/ Utopia\Tests\… → Utopia\Abuse\Tests\…. NoneTest is the unit tier; every Redis, Redis Cluster, Redis pool, MySQL and Appwrite suite moved to tests/E2E/. Hosts now come from tests/E2E/Services.php (offset host ports) instead of compose service names. AppwriteTablesDBTest renamed to TablesDBTest to match its file, and it skips unless APPWRITE_ENDPOINT points at a disposable project, so automated runs never call a hosted Appwrite
tests/bench/ phpbench cases moved from tests/Abuse/Bench, excluded from both tiers and from PHPStan
docker-compose.yml Rewritten for the host-run e2e tier, project name utopia-abuse: mysql:8 on 13308, redis:7-alpine on 16386, grokzen/redis-cluster on 17010–17015, all with healthchecks. The in-compose tests container, the Dockerfiles it built and redis-insight are gone
phpunit.xml PHPUnit 12, unit over tests (excluding E2E and bench) and e2e over tests/E2E
phpstan.neon Standalone, level max over src and tests, with phpstan-baseline.neon holding 44 findings: 35 in src (the RedisCluster and RedisPool log readers of all three strategies, plus one curl_setopt() argument in ReCaptcha), 9 in the e2e tests. Added to the RFC phase 8 list. One unmatched @phpstan-ignore-next-line removed from RedisPoolClusterTest
rector.php Standalone; skips AddArrayFunctionClosureParamTypeRector, AddArrowFunctionReturnTypeRector, AddClosureVoidReturnTypeWhereNoReturnRector, ClassPropertyAssignToConstructorPromotionRector, ClosureReturnTypeRector, ClosureToArrowFunctionRector, NullCoalescingOperatorRector and RemoveUnusedVariableInCatchRector for src, and skips TimeLimit/Database.php and TimeLimit/Appwrite/TablesDB.php entirely because Rector's PHP 8.4 printer drops the parentheses in (new \DateTime())-> whenever it reprints them
Docs CONTRIBUTING.md removed; README's pointer to the nonexistent data/schema.sql replaced with setup()

Validation

Check Result
bin/monorepo validate all packages valid
bin/monorepo check abuse Rector, Pint, PHPStan level max (with baseline) pass
bin/monorepo test abuse unit 3 tests / 6 assertions OK; e2e 109 tests / 743 assertions OK, 10 skipped (TablesDBTest, no endpoint). Compose stack torn down afterwards
vendor/bin/phpunit --testsuite packages in appwrite/appwrite:2.3.0 Completes: 8709 tests. Utopia\Abuse 3 / 6 OK. Unrelated: 5 agents DiffCheckTest errors (git cannot resolve a worktree's .git inside the container) and 1 client Swoole reconnect test that passes on rerun
Host PHPStan on app/realtime.php, app/init/resources.php, app/controllers/shared/api.php, Workers/Deletes.php, Functions/Http/Functions/Create.php, Mqtt/Handler.php No errors
Utopia\Abuse\Adapters\TimeLimit\Redis resolves to packages/abuse/src/Adapters/TimeLimit/Redis.php; no vendor/utopia-php/abuse
composer validate --no-check-publish valid (existing utopia-php/platform exact-version warning)
composer lint passes
bin/monorepo split abuse --dry-run 8b93c40, fast-forward of the mirror head 821a188 (+2 commits)
git diff --check clean

Mirror state

Ruleset None before; absorb created the canonical main ruleset (id 24125972) with the split app as always-bypass
Classic branch protection on main Present: 1 required review, push restricted to owners and @abnegate, admins not enforced. Blocks Split; must be removed before merge (see the caution above)

After merge

Step
Remove classic protection first An org owner deletes the classic branch protection on utopia-php/abuse main
Merge with a merge commit A squash drops the subtree annotation and orphans the package from its mirror
Confirm Split The Split run for abuse is green and the mirror fast-forwards from 821a188
utopia-php/abuse#124 (PR, +1103/−430) Adapts the TimeLimit schema to the query library's Attribute and Index value objects. Port here against packages/abuse (and packages/query), then close with a pointer
utopia-php/abuse#117 (PR, +68/−10) Circuit breaker support for RedisPool. Port here if still wanted, then close with a pointer
utopia-php/abuse#112 (PR) Bumps the SDK to 23.1.1; superseded by ^27.1. Close
utopia-php/abuse#19 (issue, 2021) hCaptcha adapter request. Move here or close with a pointer
Monorepo PRs None open for abuse in utopia-php/monorepo

🤖 Generated with Claude Code

TorstenDittmann and others added 30 commits April 27, 2023 10:44
@ChiragAgg5k ChiragAgg5k added the absorb History-preserving package absorption; merge commit required label Sep 28, 2026
@hansi-codes

hansi-codes Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

🟢 Tier S · Ready to merge

No actionable defects were found in the newest changes.

Absorbs the abuse, platform, audit, and usage libraries into packages/ and loads them through the root Composer autoloader instead of their Composer dependencies. The absorbed packages include mirror plumbing and package-level test and QA configuration, and the monorepo RFC records their PHPStan baselines.

Latest changes: The newest commits wire packages/usage into the root Composer autoloader and replace list, remove the external usage requirement, and record its PHPStan baseline in the RFC.

Verdict New comments Fixed Still open
✅ Approved 0 0 0
📂 Walkthrough · 7
File Change
composer.json, composer.lock Wire absorbed packages into root autoloading and replace their external Composer dependencies, including usage in the newest commits.
packages/abuse/ Import the abuse library with mirror plumbing, package QA, and unit and host-run E2E tests.
packages/platform/ Import the platform library with its package metadata, tests, mirror workflow, and QA configuration.
packages/audit/ Import the audit library with its tests, mirror workflow, and package QA configuration.
packages/usage/ Add the usage library with its tests, mirror workflow, and package QA configuration.
rfc/monorepo.md Record PHPStan baselines for the absorbed packages, including usage.
composer.json Loads and replaces the absorbed abuse package and explicitly requires the PHP SDK.

Reviewed the commits since 6131e11 · Details · Comment @hansi-codes review to re-run, or mention @hansi-codes with a question.

@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[High risk] Absorbs external abuse-prevention package into monorepo.

The PR should not merge until the explicit test requirement is satisfied and the outstanding abuse findings are addressed.

Fix All in Claude CodeFindings

  1. P2 Security Test services exposed to network ▶
  2. P2 Test clears unrelated rate limits ▶
  3. P2 Test database is left behind ▶
  4. P2 Schema assertions mirror implementation ▶
  5. P2 Log assertion pins storage format ▶
  6. P2 Test mirrors clock calculation ▶
  7. P2 Tests mirror clock details ▶
Fix with agent prompt
### Issue 1
packages/abuse/docker-compose.yml:undefined-9
If a developer runs this stack on a network-reachable host, the MySQL port is accessible with the password in this file. The unauthenticated Redis port, and the cluster ports, are also published without a localhost restriction. The tests connect only to `127.0.0.1`, so binding these ports to localhost would avoid exposing the test services to other network users.

**How this was verified:** The service ports have no host-address restriction, while the test clients use localhost.

### Issue 2
packages/abuse/tests/E2E/RedisPoolTest.php:85-91
This setup scans and deletes every `abuse__*` key, not just keys created by this test. If the Redis service is shared with another run or consumer, the test clears their rate-limit counters too. The cluster test does the same across its masters. Limit cleanup to keys owned by the test.

### Issue 3
packages/abuse/tests/E2E/Appwrite/TablesDBTest.php:181-183
When this suite is enabled, class setup creates a uniquely named database in the configured Appwrite project, but teardown does not delete it. Each run leaves another database behind, even if the tests pass. Delete the database during teardown.

### Issue 4
packages/abuse/tests/E2E/Appwrite/TablesDBTest.php:58-80
This test pins the adapter’s exact column sizes, types, bounds, and index layout. A schema change that preserves rate-limiting behavior would still fail it. The repository requires tests of observable behavior rather than assertions that mirror configuration; the repair test already checks that limiting works. This requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 5
packages/abuse/tests/E2E/RedisPoolTest.php:undefined-60
The fixture writes an internal Redis key format, and this assertion expects that exact format back. Changing how keys are stored would fail the test even if `getLogs()` still paginates correctly. The cluster test repeats the pattern. The repository requires tests of observable behavior rather than assertions that mirror implementation details, and that requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 6
packages/abuse/tests/E2E/SlidingWindow/Base.php:120-121
`testTimeFormat()` calculates the expected value using the adapter’s own clock-alignment formula, then checks a type already declared by `time()`. This violates the repository requirement to test observable behavior rather than mirror source code or configuration. The assertion can also fail if the clock advances between reading `$now` and calling the adapter. This requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

### Issue 7
packages/abuse/tests/E2E/Base.php:96-97
These assertions check that `time()` equals the current wall clock and returns an integer, rather than testing whether rate limiting works. The equality can also fail if the clock advances between calls. `TokenBucket/Base.php` repeats the type-only assertion. The repository requires tests of observable behavior, not implementation details; that requirement must be satisfied before merging.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR moves abuse into the monorepo and wires it into root Composer. Since the previous review, it also imports usage and replaces the locked usage dependency with the in-tree package.

  • The newly imported usage tests add implementation-coupled assertions that need revision.
  • All seven earlier abuse findings remain in their existing, unresolved threads; none is reposted here.

Reviews (5) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..."

Comment thread packages/abuse/docker-compose.yml
Comment thread packages/abuse/tests/E2E/RedisPoolTest.php
Comment thread packages/abuse/tests/E2E/Appwrite/TablesDBTest.php
Comment thread packages/abuse/tests/E2E/Appwrite/TablesDBTest.php
Comment thread packages/abuse/tests/E2E/RedisPoolTest.php

@hansi-codes hansi-codes Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Tier S · Looks good to merge. Summary

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

✨ Benchmark results

Comparing main (before) → chore/absorb-abuse (after).

Metric Before After Change
🚀 Requests/sec 191.35 190.17 ⚪ -0.6%
⏱️ Latency P50 89.52 ms 90.06 ms ⚪ +0.6%
⏱️ Latency P95 213.4 ms 213.19 ms ⚪ -0.1%
Per-scenario breakdown & investigation details

Metrics below reflect the current branch (after). Δ P95 compares against the base.

Scenario P50 (ms) P95 (ms) Requests RPS Δ P95 (ms)
API total 90.06 213.19 12,027 190.17 -0.21
Account 171.66 323.33 633 10.47 -5.33
TablesDB 87.18 166.03 6,541 106 +1.71
Storage 83.28 179.55 3,165 52.84 -0.95
Functions 128.48 258.78 1,688 28.82 -7.15

Top API waits (after)

API request Max wait (ms)
storage.files.create 458.17
functions.variables.update 444.88
account.name.update 443.82
functions.create 428.72
tablesdb.rows.delete 418.22

Comment thread packages/abuse/tests/E2E/SlidingWindow/Base.php
# Conflicts:
#	composer.json
#	composer.lock
#	rfc/monorepo.md
Comment thread packages/abuse/tests/E2E/Base.php
@ChiragAgg5k
ChiragAgg5k merged commit 1f88932 into main Sep 29, 2026
50 checks passed
abnegate added a commit that referenced this pull request Sep 29, 2026
utopia-php/abuse#124 moved the TimeLimit database adapter onto
utopia-php/database 8 (typed Attribute and Index models behind
attributes() and indexes()) and made the TablesDB fixture clean up a
failed setup. Main absorbed abuse into packages/abuse (#13954), so this
applies #124's head (cf0ed5d129) to the package with its tests,
adapted to the package layout: main's test namespaces and TablesDBTest
name, the schema test in the unit tier, and the discovery test running
the PHPUnit that runs it, since the monorepo hoists PHPUnit out of the
package. The package requires database ^8.0 with dev stability until
8.0.0 is tagged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

absorb History-preserving package absorption; merge commit required

Projects

None yet

Development

Successfully merging this pull request may close these issues.