From 6080cf70e7caa5afb02612873ff691cbbad843a4 Mon Sep 17 00:00:00 2001 From: jmgasper Date: Wed, 7 Oct 2026 15:11:23 +1100 Subject: [PATCH] Reduce public forum catalog latency with SQL pagination and batched authorization --- docs/public-forums.md | 78 ++++- src/forums/forums-access-policy.service.ts | 143 +++++++-- src/forums/forums-topic-context.service.ts | 63 ++++ src/forums/forums.module.ts | 9 +- src/forums/public-forum-access.service.ts | 82 ++++- src/forums/public-forums-query.service.ts | 194 ++++++++++++ src/forums/public-forums.integration.spec.ts | 309 ++++++++++++++++++- src/forums/public-forums.service.ts | 257 ++++++--------- 8 files changed, 946 insertions(+), 189 deletions(-) create mode 100644 src/forums/public-forums-query.service.ts diff --git a/docs/public-forums.md b/docs/public-forums.md index fdfac40..4aa7e84 100644 --- a/docs/public-forums.md +++ b/docs/public-forums.md @@ -119,11 +119,85 @@ Use a disposable local database whose name ends in `_http_test`, apply Prisma migrations, then run: ```sh -PUBLIC_FORUMS_TEST_DATABASE_URL=postgresql://postgres@127.0.0.1:55439/public_forums_http_test \ +TZ=UTC PUBLIC_FORUMS_TEST_DATABASE_URL=postgresql://postgres@127.0.0.1:55439/public_forums_http_test \ pnpm test --runInBand ``` That suite truncates its dedicated local test database, rejects remote hosts and other database names, uses real SQL and policies, and substitutes JWT validation and email publishing so it cannot notify real users. The standalone unit suite -runs without a database. +runs without a database. Use `TZ=UTC` for the database suite so PostgreSQL +timestamps without time zones are compared consistently by the import checks. + +## Catalog query behavior + +Public catalog reads batch category ancestry, inherited ACLs, and imported role +memberships once per request. Topic lists apply visibility, direct topic roles, +search, watches, and sorting in PostgreSQL before pagination. Only the selected +page receives full post/participant summaries; totals remain available for an +out-of-range page. Search treats `%` and `_` as literal substring characters. + +Category statistics are aggregated in SQL over authorized threads and their +non-deleted posts. Each response contains at most five participant previews per +category, selected by latest activity with deterministic ID tie-breaking. Counts +include all distinct authorized participants. Latest activity and unread state +exclude restricted or deleted descendants. No shared cache stores personalized +ACLs, watches, or unread state, so permission and membership changes are visible +on the next request. + +## Development latency remediation — 7 October 2026 + +The public legacy import exposed full-catalog reads, per-category authorization +queries, CPU saturation and a Node heap crash. The read-path changes above are +implemented in source commit `a4691d4` on `codex/forum-catalog-latency`. + +The development ECS service `tc-forums-serverless/forums-api-v6` now has a desired +count of **two tasks**, each retaining **1 vCPU / 2,048 MiB**. The existing deploy +suite updates only the task definition and therefore preserves the service's +desired count. There is no service autoscaling target overriding that count. +Both tasks retain the existing subnet placement; this adds process redundancy, +not multi-AZ redundancy. + +The dev `services-alb` HTTPS listener default action now returns HTTP **404** with +`application/json` body `{"message":"Not found"}`. Its non-default route rules +are unchanged. Previously unmatched requests reached the empty retired +`tc-email-service-svc-tg` target group and generated HTTP 503. No monitoring or +alarm configuration was changed. + +Deployment artifact: + +- Task definition: `forums-api-v6:27`. +- Image tag: `forums-api-v6:catalog-latency-a4691d4`. +- Image digest: `sha256:5b38d36f40f2287d9b04a7d57beaeeef2aa12d707cca7cf231a274e8af3b4a65`. +- Runtime base: the existing `legacy-jive-catalog-4bee3af` image, pinned to + `sha256:7e108dd776dba91546a2cfb057010f3b5bbf8a2871fceb9317508dec2f75e4fb`. + +The release reuses the unchanged runtime/dependencies and copies the compiled +JavaScript and source maps for `forums-access-policy.service`, +`forums-topic-context.service`, `forums.module`, `public-forum-access.service`, +`public-forums.service`, and `public-forums-query.service` into +`/usr/src/app/dist/forums/`. Every copied artifact's SHA-256 was checked against +the local validated build. No database migration is required by this change. +The standard source Docker build includes the same implementation for subsequent +releases. Task revision 24 remains available as the previous application version; +an application rollback can preserve the two-task count and the default 404. + +Validation includes `pnpm lint`, `pnpm build`, 173 passing tests (four optional +tests skipped), real local PostgreSQL/HTTP permission and pagination checks, +and read-only dev verification against the immutable migration source. The +source audit matched all 844 guest-visible and 854 member-visible legacy +categories, the respective thread/post counts, and private-thread denial. + +The focused PR branch `codex/forum-catalog-latency-develop` carries the same +catalog implementation onto `develop` without the earlier Jive migration tools +or dependency changes. Its lint and build checks pass, and its UTC/local +PostgreSQL test run passes 170 tests across 21 suites (three optional tests +skipped); the additional Jive tests above belong to the original deployment +branch. + +Initial live checks of the optimized read path measured topic pages at +1.7–2.2 seconds versus 14.6 seconds before remediation, and categories at +4.3–5.0 seconds versus 10.9 seconds. The category response still aggregates the +full authorized tree and is approximately 700 KB; these are sampled timings, +not a sustained-load benchmark. Verification artifacts and pre-change AWS +snapshots are retained in `/tmp/dev-latency-triage` on the operator workstation. diff --git a/src/forums/forums-access-policy.service.ts b/src/forums/forums-access-policy.service.ts index ac02890..b89dcf5 100644 --- a/src/forums/forums-access-policy.service.ts +++ b/src/forums/forums-access-policy.service.ts @@ -19,7 +19,10 @@ import { ResourceAccessService, } from './resource-access.service'; -import { PublicForumAccessService } from './public-forum-access.service'; +import { + PublicForumAccess, + PublicForumAccessService, +} from './public-forum-access.service'; const CHALLENGE_CHILD_TOPIC_DENY_REASON = 'Child-topic creation is only allowed for non-challenge effective contexts.'; @@ -71,6 +74,63 @@ export class ForumsAccessPolicyService { if (!this.isTopicTargetAvailable(context)) { return this.denyTopicDecisions('Topic not found.'); } + const categoryAccess = await this.publicAccess.decide( + this.resolveEvaluationPrincipal(principal, options), + context.topic.id, + ); + return this.evaluateTopicAccess( + principal, + context, + categoryAccess, + options, + ); + } + + /** Evaluates category actions in a batch while preserving the shared topic policy. + * @param principal Catalog reader. @param contexts Preloaded category contexts. + * @returns Topic decisions keyed by ID, including inherited ACLs and restrictions. + * @throws Database errors from ACL and challenge access lookups. + */ + async decideForTopics( + principal: ForumsPrincipal, + contexts: ForumsTopicContext[], + ): Promise> { + const access = await this.publicAccess.decideMany( + principal, + contexts.map((context) => context.topic.id), + ); + return new Map( + await Promise.all( + contexts.map( + async (context) => + [ + context.topic.id, + await this.evaluateTopicAccess( + principal, + context, + access.get(context.topic.id)!, + {}, + ), + ] as const, + ), + ), + ); + } + + /** Applies the shared topic action rules to preloaded public ACL facts. + * @param principal Actor. @param context Target context. @param categoryAccess Resolved category ACL. + * @param options Optional on-behalf identity. @returns Topic action decisions. + * @throws Database errors from challenge/resource policy adapters. + */ + private async evaluateTopicAccess( + principal: ForumsPrincipal, + context: ForumsTopicContext, + categoryAccess: PublicForumAccess, + options: ForumsAccessEvaluationOptions, + ): Promise { + if (!this.isTopicTargetAvailable(context)) { + return this.denyTopicDecisions('Topic not found.'); + } const evaluation = await this.evaluateRestrictions( principal, @@ -81,13 +141,20 @@ export class ForumsAccessPolicyService { }, options, ); - const categoryAccess = await this.publicAccess.decide(this.resolveEvaluationPrincipal(principal, options), context.topic.id); - if (!categoryAccess.view) return this.denyTopicDecisions('Required public forum access is missing.'); - if (!evaluation.visibility.allowed) return this.denyTopicDecisions(evaluation.visibility.reason ?? 'Insufficient forums access.'); + if (!categoryAccess.view) + return this.denyTopicDecisions( + 'Required public forum access is missing.', + ); + if (!evaluation.visibility.allowed) + return this.denyTopicDecisions( + evaluation.visibility.reason ?? 'Insufficient forums access.', + ); const authenticated = !!principal.memberId || principal.isMachine; const canParticipate = evaluation.visibility.allowed && authenticated; const canOwnContent = - evaluation.visibility.allowed && context.isTopicAuthor && (!categoryAccess.category || evaluation.isElevated); + evaluation.visibility.allowed && + context.isTopicAuthor && + (!categoryAccess.category || evaluation.isElevated); const canMutateContent = evaluation.isElevated || canOwnContent ? this.allow() @@ -100,15 +167,29 @@ export class ForumsAccessPolicyService { return { canView: evaluation.visibility, canCreateTopLevelTopic: this.deny('Use create-topic evaluation.'), - canCreateChildTopic: canParticipate && categoryAccess.create ? this.allow() : this.deny('Topic creation is not allowed.'), - canCreatePost: canParticipate && categoryAccess.reply ? this.allow() : this.deny('Replying is not allowed.'), + canCreateChildTopic: + canParticipate && categoryAccess.create + ? this.allow() + : this.deny('Topic creation is not allowed.'), + canCreatePost: + canParticipate && categoryAccess.reply + ? this.allow() + : this.deny('Replying is not allowed.'), canUpdateTopic: canMutateContent, canDeleteTopic, - canAddWatch: canParticipate ? this.allow() : this.deny('Authenticated member required.'), - canRemoveWatch: canParticipate ? this.allow() : this.deny('Authenticated member required.'), - canMarkRead: canParticipate ? this.allow() : this.deny('Authenticated member required.'), + canAddWatch: canParticipate + ? this.allow() + : this.deny('Authenticated member required.'), + canRemoveWatch: canParticipate + ? this.allow() + : this.deny('Authenticated member required.'), + canMarkRead: canParticipate + ? this.allow() + : this.deny('Authenticated member required.'), canControlAnnouncement, - canReceiveNotification: canParticipate ? this.allow() : this.deny('Authenticated member required.'), + canReceiveNotification: canParticipate + ? this.allow() + : this.deny('Authenticated member required.'), }; } @@ -132,7 +213,10 @@ export class ForumsAccessPolicyService { options, ); - if (!this.isTopicTargetAvailable(context) || !topicDecisions.canView.allowed) { + if ( + !this.isTopicTargetAvailable(context) || + !topicDecisions.canView.allowed + ) { return { ...topicDecisions, canUpdatePost: this.deny('Post not found.'), @@ -199,9 +283,16 @@ export class ForumsAccessPolicyService { return this.denyTopicDecisions('Topic not found.'); } - if (!principal.memberId && !principal.isMachine) return this.denyTopicDecisions('Authenticated member required.'); - if (parentContext && !(await this.publicAccess.decide(principal, parentContext.topic.id)).create) { - return this.denyTopicDecisions('Topic creation is not allowed in this category.'); + if (!principal.memberId && !principal.isMachine) + return this.denyTopicDecisions('Authenticated member required.'); + if ( + parentContext && + !(await this.publicAccess.decide(principal, parentContext.topic.id)) + .create + ) { + return this.denyTopicDecisions( + 'Topic creation is not allowed in this category.', + ); } const parentVisibility = parentContext @@ -337,7 +428,15 @@ export class ForumsAccessPolicyService { target: ForumsRestrictionVisibilityTarget, options: ForumsAccessEvaluationOptions = {}, ): Promise { - if (target.topicId && !(await this.publicAccess.decide(this.resolveEvaluationPrincipal(principal, options), target.topicId)).view) { + if ( + target.topicId && + !( + await this.publicAccess.decide( + this.resolveEvaluationPrincipal(principal, options), + target.topicId, + ) + ).view + ) { return this.deny('Required public forum access is missing.'); } return (await this.evaluateRestrictions(principal, target, options)) @@ -363,8 +462,16 @@ export class ForumsAccessPolicyService { options, ); - if (!evaluationPrincipal.memberId && !evaluationPrincipal.isMachine && (input.challengeId || input.roleName)) { - return { visibility: this.deny('Authenticated member required.'), isElevated: false, isChallengeCopilot: false }; + if ( + !evaluationPrincipal.memberId && + !evaluationPrincipal.isMachine && + (input.challengeId || input.roleName) + ) { + return { + visibility: this.deny('Authenticated member required.'), + isElevated: false, + isChallengeCopilot: false, + }; } if ( diff --git a/src/forums/forums-topic-context.service.ts b/src/forums/forums-topic-context.service.ts index 93dfe13..9c500f0 100644 --- a/src/forums/forums-topic-context.service.ts +++ b/src/forums/forums-topic-context.service.ts @@ -48,6 +48,56 @@ export class ForumsTopicContextService { return this.buildTopicContext(topic, principal); } + /** Loads catalog authorization contexts with two batched database reads. + * @param topicIds Category topic IDs to resolve. Missing topics are omitted. + * @param principal Reader whose ownership flags are computed. + * @returns Contexts keyed by topic ID, using the same inherited rules as individual reads. + * @throws Prisma errors when either read fails. + */ + async loadTopicContexts( + topicIds: string[], + principal: ForumsPrincipal, + ): Promise> { + if (!topicIds.length) return new Map(); + const closures = await this.db.topicClosure.findMany({ + where: { descendantTopicId: { in: topicIds } }, + select: { descendantTopicId: true, ancestorTopicId: true, depth: true }, + orderBy: { depth: 'asc' }, + }); + const allIds = [ + ...new Set([...topicIds, ...closures.map((row) => row.ancestorTopicId)]), + ]; + const topics = await this.db.topic.findMany({ + where: { id: { in: allIds } }, + }); + const byId = new Map(topics.map((topic) => [topic.id, topic])); + const ancestors = new Map(); + for (const closure of closures) { + const ancestor = byId.get(closure.ancestorTopicId); + if (!ancestor) continue; + const group = ancestors.get(closure.descendantTopicId) ?? []; + group.push(ancestor); + ancestors.set(closure.descendantTopicId, group); + } + return new Map( + topicIds.flatMap((id) => { + const topic = byId.get(id); + return topic + ? [ + [ + id, + this.contextFromAncestors( + topic, + ancestors.get(id) ?? [topic], + principal, + ), + ] as const, + ] + : []; + }), + ); + } + /** * Loads context for a post target. * @@ -103,6 +153,19 @@ export class ForumsTopicContextService { const ancestors = closureRows.length ? closureRows.map((closure) => closure.ancestorTopic) : [topic]; + return this.contextFromAncestors(topic, ancestors, principal); + } + + /** Resolves a previously loaded ancestor chain for both single and batch reads. + * @param topic Target topic. @param ancestors Target-first ancestor chain. + * @param principal Reader identity. @returns Effective restrictions and ownership. + * @throws Never; this is an in-memory projection. + */ + private contextFromAncestors( + topic: Topic, + ancestors: Topic[], + principal: ForumsPrincipal, + ): ForumsTopicContext { const ancestorTopics = ancestors.filter( (ancestor) => ancestor.id !== topic.id, ); diff --git a/src/forums/forums.module.ts b/src/forums/forums.module.ts index 548bb44..afcd6f9 100644 --- a/src/forums/forums.module.ts +++ b/src/forums/forums.module.ts @@ -17,6 +17,7 @@ import { ModerationController } from './moderation.controller'; import { PostsController } from './posts.controller'; import { ResourceAccessService } from './resource-access.service'; import { PublicForumsController } from './public-forums.controller'; +import { PublicForumsQueryService } from './public-forums-query.service'; import { PublicForumsService } from './public-forums.service'; import { PublicForumAccessService } from './public-forum-access.service'; import { TopicsController } from './topics.controller'; @@ -32,7 +33,12 @@ import { TopicsController } from './topics.controller'; */ @Module({ imports: [DbModule], - controllers: [TopicsController, PostsController, ModerationController, PublicForumsController], + controllers: [ + TopicsController, + PostsController, + ModerationController, + PublicForumsController, + ], providers: [ ChallengeAccessService, ChallengeApiService, @@ -40,6 +46,7 @@ import { TopicsController } from './topics.controller'; ForumsAccessPolicyService, PublicForumAccessService, PublicForumsService, + PublicForumsQueryService, ForumsCommandService, ForumsMemberDirectoryService, ForumsModerationService, diff --git a/src/forums/public-forum-access.service.ts b/src/forums/public-forum-access.service.ts index 0fc8cac..f0bc787 100644 --- a/src/forums/public-forum-access.service.ts +++ b/src/forums/public-forum-access.service.ts @@ -1,8 +1,14 @@ import { Injectable } from '@nestjs/common'; -import { PublicForumCategory } from '../../prisma/generated/client'; +import { Prisma, PublicForumCategory } from '../../prisma/generated/client'; import { DbService } from '../db/db.service'; import { ForumsPrincipal } from './forums-access.types'; +/** Only ACL fields are loaded; migration provenance must not be copied per ancestor. */ +type CategoryRules = Pick< + PublicForumCategory, + 'topicId' | 'displayAs' | 'readRoles' | 'createRoles' | 'replyRoles' +>; + export interface PublicForumAccess { view: boolean; create: boolean; @@ -37,13 +43,51 @@ export class PublicForumAccessService { JOIN "forums"."TopicClosure" closure ON closure."ancestorTopicId" = category."topicId" WHERE closure."descendantTopicId" = ${topicId} ORDER BY closure.depth ASC`; if (!categories.length) - return { - view: true, - create: true, - reply: true, - category: false, - managed: false, - }; + return this.evaluate(principal, topicId, [], new Set()); + const roles = await this.roles(principal); + return this.evaluate(principal, topicId, categories, roles); + } + + /** Evaluates catalog ACLs with one ancestor query and one membership query. + * @param principal Reader identity. @param topicIds Existing category topic IDs. + * @returns Access decisions keyed by topic ID; inherited rules match decide(). + * @throws Database errors; callers must fail closed. + */ + async decideMany( + principal: ForumsPrincipal, + topicIds: string[], + ): Promise> { + if (!topicIds.length) return new Map(); + const [categories, roles] = await Promise.all([ + this.db.$queryRaw< + (CategoryRules & { descendantTopicId: string })[] + >(Prisma.sql` + SELECT category."topicId", category."displayAs", category."readRoles", category."createRoles", category."replyRoles", closure."descendantTopicId" + FROM "forums"."PublicForumCategory" category + JOIN "forums"."TopicClosure" closure ON closure."ancestorTopicId" = category."topicId" + WHERE closure."descendantTopicId" IN (${Prisma.join(topicIds)}) + ORDER BY closure.depth ASC`), + this.roles(principal), + ]); + const grouped = new Map(); + for (const category of categories) { + const group = grouped.get(category.descendantTopicId) ?? []; + group.push(category); + grouped.set(category.descendantTopicId, group); + } + return new Map( + topicIds.map((id) => [ + id, + this.evaluate(principal, id, grouped.get(id) ?? [], roles), + ]), + ); + } + + /** Loads forum-only memberships once without elevating identity roles. + * @param principal Reader identity. @returns Identity plus imported forum roles. + * @throws Prisma errors when the membership read fails. + */ + private async roles(principal: ForumsPrincipal): Promise> { const roles = new Set(principal.roles); if (principal.memberId) { const memberships = await this.db.publicForumRoleMember.findMany({ @@ -51,6 +95,28 @@ export class PublicForumAccessService { }); memberships.forEach((row) => roles.add(row.roleName)); } + return roles; + } + + /** Applies inherited category rules to already-loaded facts. + * @param principal Reader. @param topicId Target. @param categories Nearest-first category ancestors. + * @param roles Identity and imported forum roles. @returns Read/create/reply decisions. + * @throws Never; this is an in-memory policy evaluation. + */ + private evaluate( + principal: ForumsPrincipal, + topicId: string, + categories: CategoryRules[], + roles: Set, + ): PublicForumAccess { + if (!categories.length) + return { + view: true, + create: true, + reply: true, + category: false, + managed: false, + }; const elevated = principal.isAdmin || principal.isMachine; const matches = (allowed: string[]): boolean => elevated || diff --git a/src/forums/public-forums-query.service.ts b/src/forums/public-forums-query.service.ts new file mode 100644 index 0000000..bc551a8 --- /dev/null +++ b/src/forums/public-forums-query.service.ts @@ -0,0 +1,194 @@ +import { Injectable } from '@nestjs/common'; +import { Prisma } from '../../prisma/generated/client'; +import { DbService } from '../db/db.service'; +import { PublicForumsQueryDto } from './dto/public-forums.dto'; +import { ForumsPrincipal } from './forums-access.types'; +import { ForumsTopicSummaryRow } from './forums-read-query.service'; + +/** Bounded category aggregate returned by SQL; no thread or post arrays leave the database. */ +export interface PublicCategoryRollup { + categoryId: string; + topicsCount: number; + postsCount: number; + participantsCount: number; + participants: ForumsTopicSummaryRow['participants']; + latestPostId: string | null; + latestPostAuthorMemberId: string | null; + latestPostAuthorHandle: string | null; + latestActivityAt: Date | null; + unread: boolean; +} + +/** Executes public catalog aggregation and pagination after category authorization. + * Callers supply only category IDs approved by the shared inherited access policy. + * Personalized watches/read state are computed per request; no cross-user cache is used. + */ +@Injectable() +export class PublicForumsQueryService { + /** @param db Forums database. @throws Never; supplied by Nest dependency injection. */ + constructor(private readonly db: DbService) {} + + /** Restricts non-category threads to authorized parents and applicable direct roles. + * Parent restrictions have already been checked by the shared policy. Deleted ancestors + * and challenge topics are excluded before counts, search, sorting, and pagination. + * @param principal Reader. @param categoryIds Authorized category IDs (nonempty). + * @returns Parameterized predicate for the Topic alias t. @throws Never. + */ + private visibleThreads( + principal: ForumsPrincipal, + categoryIds: string[], + ): Prisma.Sql { + // The shared restriction policy requires a member ID before a human admin can bypass a role. + const role = + principal.isMachine || (principal.isAdmin && !!principal.memberId) + ? Prisma.sql`true` + : principal.memberId && principal.roles.length + ? Prisma.sql`(t."roleName" IS NULL OR t."roleName" = '' OR t."roleName" IN (${Prisma.join([...principal.roles])}))` + : Prisma.sql`(t."roleName" IS NULL OR t."roleName" = '')`; + return Prisma.sql` + t."parentTopicId" IN (${Prisma.join(categoryIds)}) + AND t."challengeId" IS NULL AND t."deletedAt" IS NULL AND ${role} + AND NOT EXISTS (SELECT 1 FROM "forums"."PublicForumCategory" c WHERE c."topicId" = t.id) + AND NOT EXISTS ( + SELECT 1 FROM "TopicClosure" c JOIN "Topic" a ON a.id = c."ancestorTopicId" + WHERE c."descendantTopicId" = t.id AND a."deletedAt" IS NOT NULL + )`; + } + + /** Aggregates authorized descendants into one row per category entirely in PostgreSQL. + * Counts, participant previews, activity and unread flags exclude inaccessible threads. + * @param principal Reader. @param categoryIds Authorized category IDs. + * @returns Category rollups (including empty categories) with at most five participants each. + * @throws Prisma errors when aggregation fails. + */ + async categoryRollups( + principal: ForumsPrincipal, + categoryIds: string[], + ): Promise { + if (!categoryIds.length) return []; + // IDs use bytewise grouping/order; locale-aware text comparisons dominate + // large catalog joins without changing which identifiers are equal. + return this.db.$queryRaw(Prisma.sql` + WITH visible_threads AS MATERIALIZED ( + SELECT t.id COLLATE "C" AS id, t."parentTopicId" COLLATE "C" AS "parentTopicId" FROM "Topic" t + WHERE ${this.visibleThreads(principal, categoryIds)} + ), thread_counts AS ( + SELECT "parentTopicId", COUNT(*)::integer AS "topicsCount" + FROM visible_threads GROUP BY "parentTopicId" + ), post_counts AS ( + SELECT t."parentTopicId", COUNT(p.id)::integer AS "postsCount", + COALESCE(BOOL_OR(${principal.memberId}::text IS NOT NULL AND p.id IS NOT NULL + AND (r."lastReadAt" IS NULL OR p."createdAt" > r."lastReadAt")), false) AS unread + FROM visible_threads t + LEFT JOIN "Post" p ON p."topicId" = t.id AND p."deletedAt" IS NULL + LEFT JOIN "TopicReadState" r ON r."topicId" = t.id AND r."memberId" = ${principal.memberId}::text + GROUP BY t."parentTopicId" + ), totals AS ( + SELECT c."ancestorTopicId" COLLATE "C" AS "categoryId", SUM(t."topicsCount")::integer AS "topicsCount", + SUM(p."postsCount")::integer AS "postsCount", BOOL_OR(p.unread) AS unread + FROM thread_counts t JOIN post_counts p ON p."parentTopicId" = t."parentTopicId" + JOIN "TopicClosure" c ON c."descendantTopicId" = t."parentTopicId" + WHERE c."ancestorTopicId" IN (${Prisma.join(categoryIds)}) + GROUP BY c."ancestorTopicId" + ), direct_authors AS MATERIALIZED ( + SELECT DISTINCT ON (t."parentTopicId", p."authorMemberId" COLLATE "C") + t."parentTopicId", p."authorMemberId" COLLATE "C" AS "authorMemberId", p."authorHandle", p."createdAt", p.id COLLATE "C" AS id + FROM visible_threads t JOIN "Post" p ON p."topicId" = t.id AND p."deletedAt" IS NULL + ORDER BY t."parentTopicId", p."authorMemberId" COLLATE "C", p."createdAt" DESC, p.id COLLATE "C" DESC + ), rollup_authors AS MATERIALIZED ( + SELECT c."ancestorTopicId" COLLATE "C" AS "categoryId", a.* + FROM direct_authors a JOIN "TopicClosure" c ON c."descendantTopicId" = a."parentTopicId" + WHERE c."ancestorTopicId" IN (${Prisma.join(categoryIds)}) + ), authors AS ( + SELECT DISTINCT ON ("categoryId", "authorMemberId") * FROM rollup_authors + ORDER BY "categoryId", "authorMemberId", "createdAt" DESC, id DESC + ), ranked_authors AS ( + SELECT *, ROW_NUMBER() OVER ( + PARTITION BY "categoryId" ORDER BY "createdAt" DESC, id DESC, "authorMemberId" + ) AS rank FROM authors + ), participants AS ( + SELECT "categoryId", COUNT(*)::integer AS "participantsCount", + JSONB_AGG(JSONB_BUILD_OBJECT('memberId', "authorMemberId", 'handle', "authorHandle") + ORDER BY rank) FILTER (WHERE rank <= 5) AS participants + FROM ranked_authors GROUP BY "categoryId" + ), latest AS ( + SELECT DISTINCT ON ("categoryId") "categoryId", id AS "latestPostId", + "authorMemberId" AS "latestPostAuthorMemberId", "authorHandle" AS "latestPostAuthorHandle", + "createdAt" AS "latestActivityAt" + FROM rollup_authors ORDER BY "categoryId", "createdAt" DESC, id DESC + ) + SELECT category."topicId" AS "categoryId", + COALESCE(t."topicsCount", 0) AS "topicsCount", COALESCE(t."postsCount", 0) AS "postsCount", + COALESCE(p."participantsCount", 0) AS "participantsCount", COALESCE(p.participants, '[]'::jsonb) AS participants, + l."latestPostId", l."latestPostAuthorMemberId", l."latestPostAuthorHandle", l."latestActivityAt", + COALESCE(t.unread, false) AS unread + FROM "forums"."PublicForumCategory" category + LEFT JOIN totals t ON t."categoryId" = category."topicId" + LEFT JOIN participants p ON p."categoryId" = category."topicId" + LEFT JOIN latest l ON l."categoryId" = category."topicId" + WHERE category."topicId" IN (${Prisma.join(categoryIds)}) + `); + } + + /** Selects and counts visible topic IDs before the caller loads expensive summary fields. + * Watch and literal substring search filters run before SQL pagination. Category watches + * are included only for the watching view, matching the existing public API contract. + * @param principal Reader. @param categoryIds Authorized/listable categories. + * @param query Validated search/watch/sort/page inputs. + * @returns A bounded page of IDs and the total visible matching count, even beyond the last page. + * @throws Prisma errors when the read fails. + */ + async topicPage( + principal: ForumsPrincipal, + categoryIds: string[], + query: PublicForumsQueryDto, + ): Promise<{ ids: string[]; totalCount: number }> { + if ( + !categoryIds.length || + (query.watching === 'true' && !principal.memberId) + ) { + return { ids: [], totalCount: 0 }; + } + const page = query.page ?? 1; + const perPage = query.perPage ?? 20; + // Preserve empty out-of-range pages even when the validated integer exceeds PostgreSQL bigint. + const offset = Math.min(Number.MAX_SAFE_INTEGER, (page - 1) * perPage); + const search = query.search?.trim().toLowerCase(); + const watching = query.watching === 'true'; + const candidates = watching + ? Prisma.sql`((${this.visibleThreads(principal, categoryIds)}) OR t.id IN (${Prisma.join(categoryIds)}))` + : this.visibleThreads(principal, categoryIds); + const watchFilter = watching + ? Prisma.sql`AND EXISTS (SELECT 1 FROM "TopicWatch" w WHERE w."topicId" = t.id AND w."memberId" = ${principal.memberId})` + : Prisma.empty; + const searchFilter = search + ? Prisma.sql`AND (STRPOS(LOWER(t.title), ${search}) > 0 OR EXISTS ( + SELECT 1 FROM "Post" p WHERE p."topicId" = t.id AND p."deletedAt" IS NULL + AND STRPOS(LOWER(p.content), ${search}) > 0))` + : Prisma.empty; + const order = + query.sort === 'oldest' + ? Prisma.sql`"createdAt" ASC, "isAnnouncement" DESC, "latestActivityAt" DESC NULLS LAST, id ASC` + : query.sort === 'recent' + ? Prisma.sql`"createdAt" DESC, "isAnnouncement" DESC, "latestActivityAt" DESC NULLS LAST, id ASC` + : Prisma.sql`"isAnnouncement" DESC, "latestActivityAt" DESC NULLS LAST, "createdAt" DESC, id ASC`; + const rows = await this.db.$queryRaw< + { id: string | null; totalCount: number }[] + >(Prisma.sql` + WITH candidates AS MATERIALIZED ( + SELECT t.id, t."createdAt", t."isAnnouncement", + (SELECT p."createdAt" FROM "Post" p WHERE p."topicId" = t.id AND p."deletedAt" IS NULL + ORDER BY p."createdAt" DESC, p.id DESC LIMIT 1) AS "latestActivityAt" + FROM "Topic" t WHERE (${candidates}) ${watchFilter} ${searchFilter} + ), page AS ( + SELECT * FROM candidates ORDER BY ${order} LIMIT ${perPage} OFFSET ${offset} + ) + SELECT page.id, totals."totalCount" FROM (SELECT COUNT(*)::integer AS "totalCount" FROM candidates) totals + LEFT JOIN page ON true ORDER BY ${order} + `); + return { + ids: rows.flatMap((row) => (row.id ? [row.id] : [])), + totalCount: rows[0]?.totalCount ?? 0, + }; + } +} diff --git a/src/forums/public-forums.integration.spec.ts b/src/forums/public-forums.integration.spec.ts index 20171fd..56469e5 100644 --- a/src/forums/public-forums.integration.spec.ts +++ b/src/forums/public-forums.integration.spec.ts @@ -60,14 +60,19 @@ integration('public forums PostgreSQL/HTTP access boundaries', () => { if (token === 'invalid') throw new UnauthorizedException('Invalid token'); return { - userId: token === 'local' ? '202' : '101', + userId: + token === 'idless-admin' + ? undefined + : token === 'local' + ? '202' + : '101', handle: 'test-member', roles: token === 'copilot' ? ['copilot'] : token === 'reviewer' ? ['reviewer'] - : token === 'admin' + : token === 'admin' || token === 'idless-admin' ? ['administrator'] : [], isMachine: token === 'machine', @@ -238,6 +243,34 @@ integration('public forums PostgreSQL/HTTP access boundaries', () => { .get('/public/topics?categoryId=pfprivate') .expect(403); }); + it('batches catalog ACLs without changing inherited, imported, or elevated access', async () => { + for (const [identity, expected] of [ + ['member', ['pfprivate', 'pfpublic']], + ['copilot', ['pfcopilot', 'pfprivate', 'pfpublic']], + ['reviewer', ['pfprivate', 'pfpublic', 'pfreviewer']], + ['local', ['pflocal', 'pfprivate', 'pfpublic']], + [ + 'admin', + ['pfcopilot', 'pflocal', 'pfprivate', 'pfpublic', 'pfreviewer'], + ], + ] as const) { + const response = await request(app.getHttpServer()) + .get('/public/categories') + .set('Authorization', `Bearer ${identity}`) + .expect(200); + expect(response.body.map((row: { id: string }) => row.id).sort()).toEqual( + expected, + ); + const topics = await request(app.getHttpServer()) + .get('/public/topics?perPage=100') + .set('Authorization', `Bearer ${identity}`) + .expect(200); + expect(topics.body.meta.totalCount).toBe(expected.length); + expect( + topics.body.data.map((row: { id: string }) => row.id).sort(), + ).toEqual(expected.map((id) => `${id}t`)); + } + }); it('enforces roles and local memberships without granting identity privileges', async () => { await request(app.getHttpServer()) .get('/topics/pfcopilott') @@ -340,6 +373,278 @@ integration('public forums PostgreSQL/HTTP access boundaries', () => { .send({ content: 'Denied' }) .expect(403); }); + it('rolls up distinct participants across nested forums without exposing private authors', async () => { + for (const [id, parent, category, role] of [ + ['pfrollup', null, true, '$public'], + ['pfrollupchild', 'pfrollup', true, '$public'], + ['pfrolluphide', 'pfrollup', true, 'copilot'], + ['pfrollupt1', 'pfrollupchild', false, ''], + ['pfrollupt2', 'pfrollupchild', false, ''], + ['pfrollupt3', 'pfrolluphide', false, ''], + ] as const) { + await db.topic.create({ + data: { + id, + parentTopicId: parent, + title: id, + authorMemberId: '101', + authorHandle: 'member', + }, + }); + if (category) + await db.publicForumCategory.create({ + data: { + topicId: id, + legacyId: + -10 - ['pfrollup', 'pfrollupchild', 'pfrolluphide'].indexOf(id), + legacySlug: id, + description: '', + displayAs: id === 'pfrollup' ? 'Categories' : 'Discussions', + sortOrder: 2, + readRoles: [role], + createRoles: [], + replyRoles: [], + source: {}, + }, + }); + } + await sql.query(`WITH RECURSIVE paths AS ( + SELECT id AS ancestor, id AS descendant, 0 AS depth FROM forums."Topic" WHERE id LIKE 'pfrollup%' + UNION ALL SELECT p.ancestor,t.id,p.depth+1 FROM paths p JOIN forums."Topic" t ON t."parentTopicId"=p.descendant + ) INSERT INTO forums."TopicClosure" ("ancestorTopicId","descendantTopicId",depth) SELECT * FROM paths`); + for (const [id, topicId, member, handle] of [ + ['pfrollupp1', 'pfrollupt1', '101', 'shared-author'], + ['pfrollupp2', 'pfrollupt2', '101', 'shared-author'], + ['pfrollupp3', 'pfrollupt2', '202', 'second-author'], + ['pfrollupp4', 'pfrollupt3', '303', 'private-author'], + ]) + await db.post.create({ + data: { + id, + topicId, + parentType: 'TOPIC', + parentId: topicId, + authorMemberId: member, + authorHandle: handle, + content: 'test', + }, + }); + const guest = await request(app.getHttpServer()) + .get('/public/categories') + .expect(200); + const publicRoot = guest.body.find( + (c: { id: string }) => c.id === 'pfrollup', + ); + expect(publicRoot).toMatchObject({ + topicsCount: 2, + postsCount: 3, + participantsCount: 2, + }); + expect( + new Set( + publicRoot.participants.map((p: { memberId: string }) => p.memberId), + ), + ).toEqual(new Set(['101', '202'])); + expect(JSON.stringify(guest.body)).not.toContain('private-author'); + const copilot = await request(app.getHttpServer()) + .get('/public/categories') + .set('Authorization', 'Bearer copilot') + .expect(200); + expect( + copilot.body.find((c: { id: string }) => c.id === 'pfrollup'), + ).toMatchObject({ topicsCount: 3, postsCount: 4, participantsCount: 3 }); + }); + it('paginates after role/search/watch filtering and retains totals beyond the last page', async () => { + await db.topic.create({ + data: { + id: 'pfpage', + title: 'Paging category', + authorMemberId: '101', + authorHandle: 'member', + }, + }); + await db.publicForumCategory.create({ + data: { + topicId: 'pfpage', + legacyId: -90, + legacySlug: 'paging', + source: {}, + description: '', + displayAs: 'Discussions', + sortOrder: 9, + readRoles: ['$public'], + createRoles: ['$authenticated'], + replyRoles: ['$authenticated'], + }, + }); + await db.topicClosure.create({ + data: { + ancestorTopicId: 'pfpage', + descendantTopicId: 'pfpage', + depth: 0, + }, + }); + for (let i = 0; i < 15; i++) { + const id = `pfpage${String(i).padStart(2, '0')}`; + const date = new Date(Date.UTC(2026, 0, 1, 0, 0, i)); + await db.topic.create({ + data: { + id, + parentTopicId: 'pfpage', + title: `Paging ${i}`, + createdAt: date, + authorMemberId: '101', + authorHandle: 'member', + roleName: i >= 12 ? 'copilot' : null, + deletedAt: i === 14 ? date : null, + }, + }); + await db.topicClosure.createMany({ + data: [ + { ancestorTopicId: id, descendantTopicId: id, depth: 0 }, + { ancestorTopicId: 'pfpage', descendantTopicId: id, depth: 1 }, + ], + }); + await db.post.create({ + data: { + id: `ppage${i}`, + topicId: id, + parentType: 'TOPIC', + parentId: id, + authorMemberId: i >= 12 ? 'private-page-author' : String(100 + i), + authorHandle: i >= 12 ? 'hidden-paging-author' : `author-${i}`, + content: i === 2 ? 'literal_%needle' : 'ordinary body', + createdAt: date, + }, + }); + } + const page = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&sort=oldest&page=2&perPage=5') + .expect(200); + expect(page.body.meta).toEqual({ + page: 2, + perPage: 5, + totalCount: 12, + totalPages: 3, + }); + expect(page.body.data.map((row: { id: string }) => row.id)).toEqual([ + 'pfpage05', + 'pfpage06', + 'pfpage07', + 'pfpage08', + 'pfpage09', + ]); + const recent = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&sort=recent&perPage=2') + .expect(200); + expect(recent.body.data.map((row: { id: string }) => row.id)).toEqual([ + 'pfpage11', + 'pfpage10', + ]); + await db.topic.update({ + where: { id: 'pfpage00' }, + data: { isAnnouncement: true }, + }); + const active = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&perPage=2') + .expect(200); + expect(active.body.data.map((row: { id: string }) => row.id)).toEqual([ + 'pfpage00', + 'pfpage11', + ]); + const empty = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&page=100&perPage=5') + .expect(200); + expect(empty.body.data).toEqual([]); + expect(empty.body.meta.totalCount).toBe(12); + const huge = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&page=1e100&perPage=5') + .expect(200); + expect(huge.body.data).toEqual([]); + expect(huge.body.meta.totalCount).toBe(12); + const search = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&search=%25&perPage=1') + .expect(200); + expect(search.body.meta.totalCount).toBe(1); + expect(search.body.data[0].id).toBe('pfpage02'); + const categories = await request(app.getHttpServer()) + .get('/public/categories') + .expect(200); + const category = categories.body.find( + (row: { id: string }) => row.id === 'pfpage', + ); + expect(category).toMatchObject({ + topicsCount: 12, + postsCount: 12, + participantsCount: 12, + unread: false, + }); + expect(category.participants).toHaveLength(5); + expect(category.latestActivity.postId).toBe('ppage11'); + expect(JSON.stringify(category)).not.toContain('hidden-paging-author'); + const copilot = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&perPage=1') + .set('Authorization', 'Bearer copilot') + .expect(200); + expect(copilot.body.meta.totalCount).toBe(14); + const idlessAdmin = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&perPage=1') + .set('Authorization', 'Bearer idless-admin') + .expect(200); + expect(idlessAdmin.body.meta.totalCount).toBe(12); + await db.topicWatch.createMany({ + data: [ + { topicId: 'pfpage', memberId: '101' }, + { topicId: 'pfpage02', memberId: '101' }, + { topicId: 'pfpage13', memberId: '101' }, + ], + }); + const watched = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&watching=true&perPage=1') + .set('Authorization', 'Bearer member') + .expect(200); + expect(watched.body.meta.totalCount).toBe(2); + const anonymous = await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage&watching=true') + .expect(200); + expect(anonymous.body.meta.totalCount).toBe(0); + await db.topicReadState.createMany({ + data: Array.from({ length: 12 }, (_, i) => ({ + topicId: `pfpage${String(i).padStart(2, '0')}`, + memberId: '101', + lastReadAt: new Date('2026-02-01T00:00:00Z'), + })), + }); + const read = await request(app.getHttpServer()) + .get('/public/categories') + .set('Authorization', 'Bearer member') + .expect(200); + expect( + read.body.find((row: { id: string }) => row.id === 'pfpage').unread, + ).toBe(false); + await db.topicReadState.update({ + where: { topicId_memberId: { topicId: 'pfpage11', memberId: '101' } }, + data: { lastReadAt: new Date('2025-01-01T00:00:00Z') }, + }); + const unread = await request(app.getHttpServer()) + .get('/public/categories') + .set('Authorization', 'Bearer member') + .expect(200); + expect( + unread.body.find((row: { id: string }) => row.id === 'pfpage').unread, + ).toBe(true); + await db.topic.update({ + where: { id: 'pfpage' }, + data: { deletedAt: new Date() }, + }); + await request(app.getHttpServer()) + .get('/public/topics?categoryId=pfpage') + .expect(403); + const hidden = await request(app.getHttpServer()) + .get('/public/topics?search=literal_%25needle') + .expect(200); + expect(hidden.body.meta.totalCount).toBe(0); + }); it('imports atomically, preserves challenge rows, reruns safely, and refuses changed targets/source', async () => { const date = new Date('2026-01-01T00:00:00Z'); const source: PublicImportSnapshot = { diff --git a/src/forums/public-forums.service.ts b/src/forums/public-forums.service.ts index 3a81d2d..6fa7169 100644 --- a/src/forums/public-forums.service.ts +++ b/src/forums/public-forums.service.ts @@ -13,13 +13,14 @@ import { ForumsReadQueryService } from './forums-read-query.service'; import { ForumsReadService } from './forums-read.service'; import { ForumsTopicContextService } from './forums-topic-context.service'; import { ForumsAccessPolicyService } from './forums-access-policy.service'; +import { PublicForumsQueryService } from './public-forums-query.service'; /** Presents public categories and paginated threads using the existing topic/post engine. * Visibility is checked before search, watches, aggregate counts, and pagination. */ @Injectable() export class PublicForumsService { - /** @param db Forums database. @param reader Shared SQL summaries. @param reads Shared mapping/pagination. + /** @param catalog Bounded SQL catalog reads. @param db Forums database. @param reader Shared SQL summaries. @param reads Shared mapping/pagination. * @param contexts Effective restrictions. @param policy Shared action policy. @param moderation Ban checks. * @throws Never; dependencies are supplied by Nest. */ @@ -30,11 +31,12 @@ export class PublicForumsService { private readonly contexts: ForumsTopicContextService, private readonly policy: ForumsAccessPolicyService, private readonly moderation: ForumsModerationService, + private readonly catalog: PublicForumsQueryService, ) {} - /** Loads category cards in Vanilla order with visible subtree statistics. + /** Loads category cards using batched authorization and SQL subtree aggregates. * @param user Optional validated identity. @param ip Trusted request IP. - * @returns Accessible categories; private empty containers are omitted. + * @returns Accessible categories with counts/activity derived only from visible descendants. * @throws ForbiddenException for bans; database errors propagate. */ async categories( @@ -42,108 +44,40 @@ export class PublicForumsService { ip?: string, ): Promise { const principal = await this.principal(user, ip); - const metadata = await this.db.publicForumCategory.findMany({ - orderBy: { sortOrder: 'asc' }, - }); - const summaries = metadata.length - ? await this.reader.findTopicSummaryRows( - Prisma.sql`t.id IN (${Prisma.join(metadata.map((category) => category.topicId))}) AND t."deletedAt" IS NULL`, - principal.memberId, - ) - : []; - const candidates = await Promise.all( - metadata.map(async (category) => { - const context = await this.contexts.loadTopicContext( - category.topicId, - principal, - ); - const access = await this.policy.decideForTopic(principal, context); - const summary = summaries.find((row) => row.id === category.topicId); - if (!access.canView.allowed || !summary) return null; - return { - ...this.reads.mapTopicSummary(summary), + const visible = await this.authorizedCategories(principal); + if (!visible.length) return []; + const ids = visible.map((category) => category.topicId); + const [summaries, rollups] = await Promise.all([ + this.reader.findTopicSummaryRows( + Prisma.sql`t.id IN (${Prisma.join(ids)}) AND t."deletedAt" IS NULL`, + principal.memberId, + ), + this.catalog.categoryRollups(principal, ids), + ]); + const summaryById = new Map(summaries.map((row) => [row.id, row])); + const rollupById = new Map(rollups.map((row) => [row.categoryId, row])); + return visible.flatMap((category) => { + const summary = summaryById.get(category.topicId); + const rollup = rollupById.get(category.topicId); + if (!summary || !rollup) return []; + return [ + { + ...this.reads.mapTopicSummary({ ...summary, ...rollup }), description: category.description, displayAs: category.displayAs, sortOrder: category.sortOrder, - topicsCount: 0, - canCreate: access.canCreateChildTopic.allowed, - }; - }), - ); - const visible: PublicForumCategoryDto[] = candidates.filter( - (category) => category !== null, - ); - if (!visible.length) return []; - const ids = visible.map((category) => category.id); - const threads = await this.visibleThreads(principal, ids); - const authors = threads.length - ? await this.db.post.findMany({ - where: { - topicId: { in: threads.map((thread) => thread.id) }, - deletedAt: null, - }, - select: { topicId: true, authorMemberId: true, authorHandle: true }, - distinct: ['topicId', 'authorMemberId'], - }) - : []; - // Roll up only authorized rows; a private child must never affect public counts or avatars. - for (const category of visible) { - const descendantIds = new Set([category.id]); - for (let changed = true; changed; ) { - changed = false; - for (const candidate of visible) { - if ( - candidate.parentTopicId && - descendantIds.has(candidate.parentTopicId) && - !descendantIds.has(candidate.id) - ) { - descendantIds.add(candidate.id); - changed = true; - } - } - } - const descendants = threads.filter( - (thread) => - !!thread.parentTopicId && descendantIds.has(thread.parentTopicId), - ); - category.topicsCount = descendants.length; - category.postsCount = descendants.reduce( - (sum, thread) => sum + thread.postsCount, - 0, - ); - const threadIds = new Set(descendants.map((thread) => thread.id)); - const participants = new Map( - authors - .filter((author) => threadIds.has(author.topicId)) - .map((author) => [ - author.authorMemberId, - { memberId: author.authorMemberId, handle: author.authorHandle }, - ]), - ); - category.participants = [...participants.values()].slice(0, 5); - category.participantsCount = participants.size; - const newest = [...descendants].sort( - (a, b) => - (b.latestActivityAt?.getTime() ?? 0) - - (a.latestActivityAt?.getTime() ?? 0), - )[0]; - category.latestActivity = newest - ? this.reads.mapTopicSummary(newest).latestActivity - : null; - category.unread = descendants.some((thread) => thread.unread); - } - return visible.filter( - (category) => - category.displayAs !== 'Categories' || - visible.some((child) => child.parentTopicId === category.id), - ); + topicsCount: rollup.topicsCount, + canCreate: category.canCreate, + }, + ]; + }); } - /** Lists visible threads, or explicitly watched categories and threads. - * @param query Validated search, category, sort, watch and page inputs. - * @param user Optional validated identity. @param ip Trusted request IP. - * @returns Page whose totals disclose only authorized content. - * @throws ForbiddenException for bans or inaccessible categories; database failures propagate. + /** Lists visible threads or explicit watches, authorizing before SQL pagination. + * Only the selected page is enriched with posts, participants, and read state. + * @param query Validated list inputs. @param user Optional identity. @param ip Trusted IP. + * @returns Bounded summaries and visible matching totals. + * @throws ForbiddenException for bans/inaccessible categories; database errors propagate. */ async topics( query: PublicForumsQueryDto, @@ -151,76 +85,83 @@ export class PublicForumsService { ip?: string, ): Promise { const principal = await this.principal(user, ip); - const categories = await this.categories(user, ip); + const categories = await this.authorizedCategories(principal); if ( query.categoryId && - !categories.some((category) => category.id === query.categoryId) + !categories.some((category) => category.topicId === query.categoryId) ) { throw new ForbiddenException('Category is unavailable.'); } const ids = categories .filter( - (category) => !query.categoryId || category.id === query.categoryId, + (category) => + !query.categoryId || category.topicId === query.categoryId, ) - .map((category) => category.id); - if (!ids.length) return this.reads.paginateRows([], query); - let rows = await this.visibleThreads(principal, ids); - if (query.watching === 'true') { - const categoryRows = await this.reader.findTopicSummaryRows( - Prisma.sql`t.id IN (${Prisma.join(ids)}) AND t."deletedAt" IS NULL`, - principal.memberId, - ); - rows = [...rows, ...categoryRows].filter((row) => row.watching); - } - const search = query.search?.trim().toLowerCase(); - if (search && rows.length) { - const matches = await this.db.post.findMany({ - where: { - topicId: { in: rows.map((row) => row.id) }, - deletedAt: null, - content: { contains: search, mode: 'insensitive' }, - }, - select: { topicId: true }, - distinct: ['topicId'], - }); - const matchingIds = new Set(matches.map((post) => post.topicId)); - rows = rows.filter( - (row) => - row.title.toLowerCase().includes(search) || matchingIds.has(row.id), - ); - } - if (query.sort === 'oldest' || query.sort === 'recent') { - rows.sort( - (a, b) => - (a.createdAt.getTime() - b.createdAt.getTime()) * - (query.sort === 'oldest' ? 1 : -1), - ); - } - return this.reads.paginateRows(rows, query); + .map((category) => category.topicId); + const result = await this.catalog.topicPage(principal, ids, query); + const summaries = result.ids.length + ? await this.reader.findTopicSummaryRows( + Prisma.sql`t.id IN (${Prisma.join(result.ids)}) AND t."deletedAt" IS NULL`, + principal.memberId, + ) + : []; + const byId = new Map(summaries.map((row) => [row.id, row])); + const page = query.page ?? 1; + const perPage = query.perPage ?? 20; + return { + data: result.ids.flatMap((id) => { + const row = byId.get(id); + return row ? [this.reads.mapTopicSummary(row)] : []; + }), + meta: { + page, + perPage, + totalCount: result.totalCount, + totalPages: Math.ceil(result.totalCount / perPage), + }, + }; } - /** Reads thread candidates only beneath visible categories, then checks narrower topic restrictions. - * @param principal Reader identity. @param categoryIds Authorized category IDs. - * @returns Authorized summaries ordered by latest activity. @throws Database failures. + /** Loads listable category metadata and inherited policy decisions in batches. + * This intentionally omits full-catalog counts and participant rollups for topic lists. + * @param principal Reader checked for bans. @returns Authorized category metadata and create permission. + * @throws Database or external policy errors; unavailable/missing categories are omitted. */ - private async visibleThreads( - principal: ForumsPrincipal, - categoryIds: string[], - ) { - const rows = await this.reader.findTopicSummaryRows( - Prisma.sql` - t."parentTopicId" IN (${Prisma.join(categoryIds)}) AND t."challengeId" IS NULL AND t."deletedAt" IS NULL - AND NOT EXISTS (SELECT 1 FROM "forums"."PublicForumCategory" category WHERE category."topicId" = t.id) - AND NOT EXISTS (SELECT 1 FROM "TopicClosure" c JOIN "Topic" a ON a.id = c."ancestorTopicId" - WHERE c."descendantTopicId" = t.id AND a."deletedAt" IS NOT NULL)`, - principal.memberId, + private async authorizedCategories(principal: ForumsPrincipal) { + const metadata = await this.db.publicForumCategory.findMany({ + orderBy: { sortOrder: 'asc' }, + select: { + topicId: true, + description: true, + displayAs: true, + sortOrder: true, + }, + }); + const contexts = await this.contexts.loadTopicContexts( + metadata.map((category) => category.topicId), + principal, + ); + const decisions = await this.policy.decideForTopics(principal, [ + ...contexts.values(), + ]); + const visible = metadata.flatMap((category) => { + const decision = decisions.get(category.topicId); + const context = contexts.get(category.topicId); + return decision?.canView.allowed && context + ? [ + { + ...category, + parentTopicId: context.topic.parentTopicId, + canCreate: decision.canCreateChildTopic.allowed, + }, + ] + : []; + }); + const parents = new Set(visible.map((category) => category.parentTopicId)); + return visible.filter( + (category) => + category.displayAs !== 'Categories' || parents.has(category.topicId), ); - // The parent ACL was evaluated above. The only additional restrictions a thread can add are role/challenge values. - return this.reads.filterVisibleRows(principal, rows, (row) => ({ - challengeId: row.challengeId, - roleName: row.roleName, - hasRestrictionConflict: false, - })); } /** Builds an anonymous/validated reader and enforces member/IP bans before querying content.