Skip to content

STORY: Implement centralized logging with OpenSearch #91

Closed
5 of 6 issues completed
Closed

Description

@ramioooz

Outcome

Provide centralized, searchable application and infrastructure logs for the local Kubernetes SIT platform.

Delivered Scope

  • Structured JSON logs are emitted by the platform services.
  • Fluent Bit collects Kubernetes container logs and routes them to OpenSearch.
  • OpenSearch Dashboards provides reproducible operational views and alerts.
  • Sensitive values are excluded or redacted before indexing.
  • Operational logs are kept separate from immutable financial and security audit records.
  • Local access, encryption, retention, and audit boundaries are documented in the completed child tasks.

Deferred Scope

AWS OpenSearch Service mapping and the AWS deployment decision are intentionally deferred to Sprint 7. They are tracked by #97 and are not a prerequisite for closing the local SIT logging outcome.

Acceptance Evidence

No additional CI workflow or redundant test suite is required for this outcome.

Activity

  1. added theissue type on Aug 2, 2026
  2. self-assigned this
    on Aug 30, 2026
  3. added a commit that references this issue on Sep 6, 2026
  4. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    Logging workstream checkpoint (2026-09-06)\n\nThe structured-logging implementation is now split into reviewable PRs:\n\n- customer-service#37\n- account-service#43\n- auth-service#12\n- mfa-service#19\n- payment-service#12\n- notification-service#13\n- ledger-service#22\n- config-repo#46\n- api-gateway#26\n\nAll service PRs above except api-gateway#26 have passing Maven, Helm, and container checks. config-repo#46 is configuration-only and has no CI checks.\n\napi-gateway#26 is not review-ready yet because its existing GatewayResilienceIntegrationTests failed in CI with expected circuit state OPEN but observed CLOSED. The container and Helm checks passed; the implementation will remain open for a focused correction and rerun.\n\nThe PRs add the required flat application fields while preserving ECS fields. The OpenSearch collector-side nested structured.log indexing limitation remains a separate runtime concern and is not being hidden by application changes.

  5. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    Logging PRs ready for review (2026-09-06)\n\nAPI Gateway PR #26 was corrected after the first CI run exposed a pre-existing circuit-breaker test timing failure. The current commit passes Maven verification, Helm validation, and container smoke.\n\nAll nine PRs are now reviewable:\n- customer-service#37\n- account-service#43\n- auth-service#12\n- mfa-service#19\n- payment-service#12\n- notification-service#13\n- ledger-service#22\n- config-repo#46 (configuration-only; no CI workflow configured)\n- api-gateway#26\n\nKeep this story open until the merged images are rolled out in SIT and OpenSearch searchability is verified. No service PR adds a redundant test suite; only the representative structured-logging contract test was added.

  6. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    Status update - 2026-09-06

    The centralized structured-logging implementation PRs have now merged across the platform services, API Gateway, Config Repository, and Ledger Service. The remaining acceptance work is SIT rollout and OpenSearch/Fluent Bit verification; no additional service PR is currently required.

  7. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    Local SIT acceptance is complete based on the merged implementation and recorded runtime evidence in this issue and child task #94. AWS OpenSearch mapping and the deployment decision remain deferred to Sprint 7 task #97; the story scope has been narrowed accordingly. No secret values were recorded.

  8. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    Closing the local SIT centralized-logging outcome. The merged application logging, OpenSearch, Fluent Bit, dashboard, redaction, and boundary work is implemented and the recorded SIT evidence satisfies the narrowed scope. AWS OpenSearch mapping remains deferred to Sprint 7 task #97.

  9. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    Post-merge SIT verification update: fresh merged-logging images are now deployed and healthy for Customer, Account, Ledger, Auth, MFA, Payment, and Notification Services. Fresh health-request logs expose service identity, environment=sit, correlation_id, and trace_id fields. Transaction Service was intentionally left on its validated 0.0.6 image and still needs the current logging baseline rolled out; keep this task open until that final service and the Fluent Bit/OpenSearch field query are verified.

  10. ramioooz commented on Sep 6, 2026

    @ramioooz
    MemberAuthor

    2026-09-07 SIT update: Transaction Service structured logging is now runtime-verified after follow-up transaction-service PR #28 candidate deployment. Health returned X-Correlation-ID logging-fix-20260907, and the fresh log record contained service.name=transaction-service, environment=sit, correlation_id=logging-fix-20260907, event.name=http.request.completed, and HTTP 200. Centralized logging acceptance remains open pending post-merge image rollout and Fluent Bit/OpenSearch field verification.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions