Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@

- Owns service bootstrap, Config Client, health probes, OpenAPI metadata, packaging, and deployment.
- Owns the transport-neutral payment instruction lifecycle foundation and its idempotency/correlation boundary.
- Does not own customer data, account balances, ledger postings, transfer saga orchestration, Kafka behavior, persistence, or secrets.
- Owns durable payment instruction persistence, the transactional payment state outbox, and bounded Kafka publication.
- Does not own customer data, account balances, ledger postings, transfer saga orchestration, provider integrations, or secrets.
- Do not add public payment routes without a supporting issue and API contract work.

## Commands
Expand All @@ -32,7 +33,7 @@ helm lint helm --strict --values helm/values-sit.yaml

Use unit tests for isolated application/domain behavior when those layers exist. Use integration tests for HTTP and infrastructure-backed behavior. Keep `./mvnw verify` green before merge.

The payment instruction foundation currently uses an in-memory output adapter for lifecycle tests. Do not mistake it for production persistence; durable payment state and external payment-provider integration require separately tracked work.
The payment instruction lifecycle uses PostgreSQL in production and an in-memory adapter in unit tests. The transactional state outbox and Kafka publisher are part of the current payment instruction event slice; external payment-provider integration requires separately tracked work.

## Architecture

Expand Down
14 changes: 11 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Payment Service

Payment Service is the Digital Bank Java platform foundation for future payment rail workflows. This repository contains a deployable Spring Boot service plus an authenticated internal payment instruction lifecycle API; payment rail business logic, Kafka behavior, and public gateway routes remain intentionally out of scope.
Payment Service is the Digital Bank Java platform foundation for future payment rail workflows. This repository contains a deployable Spring Boot service, an authenticated internal payment instruction lifecycle API, and durable publication of governed payment instruction state events; payment rail business logic and public gateway routes remain intentionally out of scope.

## Implemented State

Expand All @@ -11,10 +11,11 @@ Payment Service is the Digital Bank Java platform foundation for future payment
- Non-root container image and hardened Helm deployment.
- Default SIT service port `8085`.
- Internal payment instruction lifecycle endpoints at `/internal/v1/payment-instructions`.
- PostgreSQL-backed transactional outbox and bounded at-least-once Kafka publication for `payment.instruction.state.v1`.

## Boundaries

This service will later own payment workflow coordination and payment rail integration boundaries. It does not currently own customer data, account balances, ledger postings, transfer saga orchestration, Kafka topics, or provider credentials.
This service will later own payment workflow coordination and payment rail integration boundaries. It does not currently own customer data, account balances, ledger postings, transfer saga orchestration, or provider credentials. Its Kafka producer publishes only payment instruction state facts; the governed AsyncAPI contract is maintained in the organization `.github` repository.

Payment rail integrations must remain behind outbound ports and adapters when that work is approved and tracked. Do not add provider credentials or payment data to this repository.

Expand All @@ -24,7 +25,7 @@ The application exposes an internal HTTP adapter over the transport-neutral paym

The application boundary also normalizes the idempotency key, amount, currency, and description before comparing retries. An equivalent retry returns the original instruction identity, while reuse of the same idempotency key with a different business request is rejected. Correlation IDs are retained for tracing and are deliberately excluded from the idempotency comparison, so a retried request may have a new trace context.

Payment instructions are stored in PostgreSQL through a Flyway-managed schema. Idempotency keys are unique in the database, so equivalent retries converge across replicas and restarts; the canonical request fields are compared before a replay is returned, and a changed payload returns `409 Conflict`. Payment-provider adapters, Kafka publication, and transaction saga orchestration are intentionally deferred to their planned stories.
Payment instructions are stored in PostgreSQL through a Flyway-managed schema. Idempotency keys are unique in the database, so equivalent retries converge across replicas and restarts; the canonical request fields are compared before a replay is returned, and a changed payload returns `409 Conflict`. Creation and real terminal transitions insert one durable outbox record in the same database transaction. The publisher sends the stored payload and governed headers with a bounded leased retry loop, preserving event identity across retries. Payment-provider adapters and transaction saga orchestration remain out of scope.

All payment instruction endpoints require a bearer JWT with the `payment.internal` scope. Missing or invalid authentication returns `401 Unauthorized`; an authenticated caller without that scope returns `403 Forbidden`. Both responses use `application/problem+json`. Health, service metadata, and the generated OpenAPI document remain public.

Expand Down Expand Up @@ -54,6 +55,13 @@ Config Server supplies the effective runtime configuration. The service reposito
| `spring.security.oauth2.resourceserver.jwt.jwk-set-uri` | JWT JWK set URI | Optional when issuer discovery is available |
| `auth.jwt.secret` | Base64 HMAC secret shared with Auth Service in SIT | none |
| `auth.jwt.issuer` | HMAC token issuer used in SIT | none |
| `PAYMENT_EVENTS_TOPIC` | Kafka topic for payment instruction state events | `payment.instruction.state.v1` |
| `PAYMENT_EVENTS_BATCH_SIZE` | Maximum events claimed per publisher poll | `25` |
| `PAYMENT_EVENTS_MAX_ATTEMPTS` | Maximum broker failure attempts before durable publication failure | `5` |
| `PAYMENT_EVENTS_RETRY_BACKOFF` | Delay before retrying a failed publication | `PT5S` |
| `PAYMENT_EVENTS_LEASE` | Claim lease for an in-flight publication batch | `PT30S` |
| `PAYMENT_EVENTS_SEND_TIMEOUT` | Maximum wait for one Kafka send | `PT5S` |
| `PAYMENT_EVENTS_POLL_DELAY_MS` | Publisher poll delay in milliseconds | `1000` |

The application fallback port is `8085`, and the Helm chart sets `SERVER_PORT` from `service.port` so the process, probes, and Service remain aligned even before a service-specific Config Repo entry is added. The SIT chart injects the `payment_service` JDBC URL and reads PostgreSQL credentials from the existing `postgres` Secret (`POSTGRES_USER` and `POSTGRES_PASSWORD`). Credentials remain outside Git.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Payment Instruction State Events Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add a transactional PostgreSQL outbox and bounded at-least-once Kafka publisher for payment-instruction state events.

**Architecture:** Extend the existing application service transaction to persist one immutable event payload for each accepted instruction state. Keep the outbox behind an outbound port, use JDBC/Flyway for PostgreSQL, and isolate Kafka delivery in a scheduled adapter with leased batch claims and durable retry state.

**Tech Stack:** Java 21, Spring Boot 4, Spring JDBC transactions, Flyway PostgreSQL migrations, Spring Kafka, Jackson, JUnit 5, AssertJ, Testcontainers PostgreSQL.

**Spec:** `docs/superpowers/specs/2026-09-06-payment-instruction-state-events-design.md`

## Global Constraints

- Use topic `payment.instruction.state.v1` and event type `PaymentInstructionStateChanged.v1`.
- Payload status is exactly `PENDING`, `COMPLETED`, or `FAILED`.
- Reuse the persisted `eventId` and serialized payload on every retry.
- Create and terminal transition plus outbox insert must share one database transaction.
- Do not mutate accounts or ledger, add provider credentials, add public routes, or add extra CI jobs.
- Keep tests focused on load-bearing persistence and publication behavior.
- Add the organization AsyncAPI contract in a separate `.github` PR; do not embed a conflicting schema in this service.

---

### Task 1: Add the event and outbox application boundaries

**Files:**
- Create: `src/main/java/com/digitalbank/paymentservice/application/port/out/PaymentInstructionStateEvent.java`
- Create: `src/main/java/com/digitalbank/paymentservice/application/port/out/PaymentInstructionOutbox.java`
- Create: `src/main/java/com/digitalbank/paymentservice/application/port/out/PaymentInstructionStateEventFactory.java`
- Modify: `src/main/java/com/digitalbank/paymentservice/application/port/out/PaymentInstructionRepository.java`
- Modify: `src/main/java/com/digitalbank/paymentservice/application/port/out/PaymentInstructionSaveResult.java`
- Test: `src/test/java/com/digitalbank/paymentservice/application/service/PaymentInstructionServiceTest.java`

**Interfaces:**
- `PaymentInstructionStateEvent` exposes `eventId`, `eventType`, `schemaVersion`, `producer`, `occurredAt`, `aggregateId`, `correlationId`, `causationId`, `instructionId`, `idempotencyKey`, `amount`, `currency`, `status`, and nullable `failureReason`, plus serialized payload and header values.
- `PaymentInstructionOutbox.save(PaymentInstructionStateEvent event)` is the transactional outbound port; the JDBC adapter implements it later.
- `PaymentInstructionRepository.saveIfAbsent` accepts the instruction and a PENDING event; `transition` returns a result that distinguishes a real state change from an idempotent replay.

- [ ] **Step 1: Write the failing service tests** for one PENDING event on a new instruction, one terminal event on a real transition, and no duplicate event on equivalent create or repeated terminal replay.
- [ ] **Step 2: Run the focused test** with `./mvnw --batch-mode --no-transfer-progress -Dtest=PaymentInstructionServiceTest test`; confirm it fails because the new event/outbox contract is absent.
- [ ] **Step 3: Implement the immutable event record, event factory, and repository transition result** using the approved envelope and deterministic causation fallback.
- [ ] **Step 4: Update the service transaction** to build an event once per real mutation and call the outbox port only after the corresponding instruction write succeeds in the same transaction.
- [ ] **Step 5: Run the focused test** again and confirm all service tests pass.
- [ ] **Step 6: Commit** with `git add src/main/java src/test/java && git commit -m "feat: define payment instruction state events"`.

### Task 2: Add PostgreSQL outbox persistence and atomic integration

**Files:**
- Create: `src/main/resources/db/migration/V2__create_payment_instruction_outbox.sql`
- Create: `src/main/java/com/digitalbank/paymentservice/adapter/out/persistence/PostgresPaymentInstructionOutbox.java`
- Create: `src/main/java/com/digitalbank/paymentservice/adapter/out/persistence/PaymentInstructionOutboxRecord.java`
- Modify: `src/main/java/com/digitalbank/paymentservice/adapter/out/persistence/PostgresPaymentInstructionRepository.java`
- Modify: `src/test/java/com/digitalbank/paymentservice/PaymentInstructionApiIT.java`

**Interfaces:**
- The adapter implements `PaymentInstructionOutbox` and provides `claimBatch`, `markPublished`, and `markFailedOrRetry` for the publisher.
- The migration creates immutable event identity and payload columns, a unique `(instruction_id, event_status)` key, and indexed eligible-row lookup.

- [ ] **Step 1: Write the failing PostgreSQL integration assertions** that creation leaves exactly one PENDING outbox row with matching instruction fields and that completion/failure leaves exactly one matching terminal row.
- [ ] **Step 2: Run `./mvnw --batch-mode --no-transfer-progress -Dtest=PaymentInstructionApiIT test`** and confirm the new SQL assertions fail because the table/adapter is absent.
- [ ] **Step 3: Implement the Flyway migration** with check constraints for event and publication fields, unique identity, and retry/lease indexes.
- [ ] **Step 4: Implement the JDBC outbox adapter** with serialized payload storage, PostgreSQL `FOR UPDATE SKIP LOCKED` claims, leases, and durable publication/error updates.
- [ ] **Step 5: Wire the existing JDBC instruction repository and service transaction** so a failed outbox insert rolls back the instruction mutation.
- [ ] **Step 6: Run the focused integration test** and confirm atomic creation, terminal transition, and replay behavior pass.
- [ ] **Step 7: Commit** with `git add src/main src/test && git commit -m "feat: persist payment instruction outbox events"`.

### Task 3: Add bounded Kafka publication

**Files:**
- Create: `src/main/java/com/digitalbank/paymentservice/adapter/out/kafka/PaymentInstructionEventPublisher.java`
- Create: `src/main/java/com/digitalbank/paymentservice/configuration/PaymentEventProperties.java`
- Modify: `pom.xml`
- Modify: `src/main/resources/application.properties`
- Modify: `src/main/java/com/digitalbank/paymentservice/PaymentServiceApplication.java`
- Test: `src/test/java/com/digitalbank/paymentservice/adapter/out/kafka/PaymentInstructionEventPublisherTest.java`

**Interfaces:**
- `PaymentInstructionEventPublisher.publishBatch()` claims at most `batch-size` eligible records, sends each `ProducerRecord<String,String>` to the configured topic, and marks each outcome through the outbox adapter.
- Kafka record key is `aggregateId`; required headers are `event-id`, `correlation-id`, `causation-id`, `producer`, `schema-version`, and `occurred-at`.
- Defaults are topic `payment.instruction.state.v1`, batch size `25`, max attempts `5`, fixed retry backoff `PT5S`, and lease `PT30S`; all are externalizable under `payment.events.*`.

- [ ] **Step 1: Write the failing publisher tests** for successful publication with exact topic/key/headers/payload, retry after a sender failure, and durable final failure at max attempts.
- [ ] **Step 2: Run `./mvnw --batch-mode --no-transfer-progress -Dtest=PaymentInstructionEventPublisherTest test`** and confirm it fails because the publisher is absent.
- [ ] **Step 3: Add `spring-kafka`, typed properties, scheduling enablement, and safe defaults** without adding provider or Kafka credentials to the repository.
- [ ] **Step 4: Implement the publisher** with bounded claim size, blocking send result handling, lease-safe updates, and bounded error text.
- [ ] **Step 5: Run the focused publisher test** and confirm all publication cases pass.
- [ ] **Step 6: Commit** with `git add pom.xml src/main src/test && git commit -m "feat: publish payment instruction state events"`.

### Task 4: Document operations and verify the whole service

**Files:**
- Modify: `README.md`
- Modify: `AGENTS.md`
- Test: existing Maven integration suite and Helm chart

- [ ] **Step 1: Add concise operational documentation** for the topic, outbox retry visibility, externalized publisher settings, and SIT evidence query; state that account/ledger/provider/public-route behavior remains out of scope.
- [ ] **Step 2: Run `./mvnw --batch-mode --no-transfer-progress test`**.
- [ ] **Step 3: Run `./mvnw --batch-mode --no-transfer-progress verify -DskipUnitTests=true`** for integration/package verification.
- [ ] **Step 4: Run `helm lint helm --strict --values helm/values-sit.yaml` and render the chart with `helm template`**.
- [ ] **Step 5: Run `git diff --check` and scan the diff for forbidden account/ledger mutations, credentials, public routes, and CI changes.**
- [ ] **Step 6: Commit** with `git add README.md AGENTS.md && git commit -m "docs: document payment state event operations"`.
- [ ] **Step 7: Create the separate `.github` contract worktree/PR** from `.github` `origin/main`, add the exact approved AsyncAPI contract, run its existing validation, push, and open a normal non-draft PR. Link it explicitly from the service PR.
- [ ] **Step 8: Request review of the service diff, fix Critical/Important findings, then push the service branch and open a normal non-draft PR linked to `.github#250` and `.github#31`; do not merge either PR.
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Payment Instruction State Events Design

**Issue:** `.github#250` under `.github#31`

## Goal

Publish durable, versioned payment-instruction state events for `PENDING`,
`COMPLETED`, and `FAILED` without changing account or ledger state, adding
provider integrations, or adding public payment routes.

## Contract

The organization contract will be added in a separate `.github` pull request.
The service publishes one Kafka topic:

```text
payment.instruction.state.v1
```

Every message uses the event type `PaymentInstructionStateChanged.v1` and the
following JSON payload envelope:

```json
{
"eventId": "uuid",
"eventType": "PaymentInstructionStateChanged.v1",
"schemaVersion": "1.0.0",
"producer": "payment-service",
"occurredAt": "2026-09-06T19:00:00Z",
"aggregateId": "payment-instruction-uuid",
"correlationId": "trace-or-workflow-id",
"causationId": "stable-command-or-transition-id",
"instructionId": "payment-instruction-uuid",
"idempotencyKey": "client-idempotency-key",
"amount": "125.50",
"currency": "AED",
"status": "PENDING",
"failureReason": null
}
```

`failureReason` is present only for `FAILED` events. Required Kafka headers use
kebab-case and repeat `event-id`, `correlation-id`, `causation-id`, `producer`,
`schema-version`, and `occurred-at`. `aggregateId` is the Kafka partition key.
`eventId` is generated once for a state transition, persisted with the outbox
payload, and reused for every delivery retry.

The existing lifecycle API does not carry a causal command identifier. Until a
dedicated command-header contract is introduced, creation uses the normalized
idempotency key as `causationId`; a terminal transition uses
`<instructionId>:<status>`. The instruction's existing correlation ID is
preserved unchanged.

## Persistence And Atomicity

Flyway adds `payment_instruction_outbox` with one row per instruction state,
enforced by a unique `(instruction_id, event_status)` constraint. The row
stores the serialized JSON payload plus the common header values and publisher
control data: attempts, next-attempt time, lease expiry, publication time, and
last error. `PENDING`, `COMPLETED`, and `FAILED` are business event statuses;
publication status is represented by `published_at` and retry metadata.

`PaymentInstructionService` owns the application transaction. On creation it
persists the instruction and PENDING outbox record together. On a real terminal
transition it updates the instruction and inserts exactly one terminal outbox
record together. Idempotent replays return the existing state and do not create
another outbox row. A failed outbox insert rolls back the instruction mutation.

## Publication

The publisher is a Spring scheduled adapter backed by `KafkaTemplate<String,
String>`. It claims at most the configured batch size using PostgreSQL row locks
and a time-bounded lease, increments the attempt count before sending, and
publishes the stored payload and headers. Success sets `published_at`; failure
sets `last_error` and a bounded retry time, or leaves the row durably marked
failed once `max-attempts` is reached. Lease expiry makes an abandoned claim
eligible again, preserving at-least-once delivery. A crash after Kafka accepts
the record and before the success update may duplicate delivery, but never
creates a second business transition and always reuses the same event ID.

## Testing And Scope

Focused tests cover the application transaction/outbox behavior against the
existing PostgreSQL Testcontainers pattern and publisher success/retry/final
failure using a test sender. Existing API, unit, Maven, and Helm checks remain
the quality gates. No account, ledger, provider credential, public route,
redundant CI job, or direct synchronous downstream call is introduced.
Loading
Loading