[#336] Tamper-evident CSV audit: write and read the CSV files in UTF-8 on every path

[vharseko]

Fixes #336.

Charset

RotatableWriter writes in UTF-8, and CsvSecureUtils.dataToSign() computes each HMAC over the UTF-8 bytes of the values. TextWriter.Stream, which writes when rotation is disabled, and CsvSecureVerifier used the platform default charset. With a default charset other than UTF-8 (JDK 11–17 on Windows, or -Dfile.encoding), an untampered file with a non-ASCII value failed verification with "The HMac at row N is not correct", the live file too, so the writer could not resume it. With rotation disabled, a character that the platform charset cannot encode, such as a Cyrillic letter in windows-1252, was written as ?.

Every reader and writer of the audit files now uses UTF-8:

• TextWriter.Stream writes in UTF-8, as RotatableWriter does. In this repository only the two CSV writers use it.
• CsvSecureVerifier reads in UTF-8.
• CsvAuditEventHandler reads the audit file in UTF-8 for readEvent and queryEvents, which returned non-ASCII values garbled.
• StandardCsvWriter reads the header of the file it resumes in UTF-8. A non-ASCII column name failed with "Resuming an existing CSV file but the headers do not match."

Malformed input

CsvSecureVerifier decodes with UTF_8.newDecoder(), which reports malformed input instead of replacing it with U+FFFD. A file that is not UTF-8 fails with "The file is not valid UTF-8" instead of an HMAC failure, and a byte sequence that is not UTF-8 can no longer pass for a U+FFFD that the writer logged.

Upgrade

On a host whose default charset is not UTF-8 (Windows, JDK 11–17, or -Dfile.encoding), the files that the previous version wrote with rotation disabled are in that charset:

• A tamper-evident file with a non-ASCII value that this charset encodes (ü or é in windows-1252) passed verification before and now fails with "The file is not valid UTF-8". When its bytes happen to be valid UTF-8, as some sequences of a multi-byte charset such as MS932 or GBK are, it fails with "The HMac at row N is not correct." instead. This includes the live file, which the writer verifies before it resumes it. Until [#340] Tamper-evident CSV audit: close the file of a writer that fails to resume it #341, this failure stops the startup of the whole audit service, not only of the topic: SecureCsvWriter wraps it in a RuntimeException, which neither CsvAuditEventHandler.startup() nor AuditServiceImpl.startup() catches. It also leaves the live file open (Tamper-evident CSV audit: a writer that fails to resume its file leaks it, and stops the audit service from starting #340).
• A plain CSV file is resumed as it is, so the rows appended after the upgrade are in UTF-8. readEvent and queryEvents read the rows written before with U+FFFD in place of the bytes that are not UTF-8, and a query no longer matches such a value. A file whose header holds a non-ASCII column name, which only a custom topic schema has, is not resumed: the topic fails with "Resuming an existing CSV file but the headers do not match."

On such a host, before the upgrade, stop the service and move aside the files of each topic written with rotation disabled. For a tamper-evident topic, rename tamper-evident-<topic>.csv and tamper-evident-<topic>.csv.keystore to tamper-evident-<topic>.csv<timestamp> and tamper-evident-<topic>.csv<timestamp>.keystore, with <timestamp> in the format of the CLI's --suffix without .gz (-yyyy.MM.dd-HH.mm.ss by default, as in tamper-evident-access.csv-2026.10.19-12.00.00): the file is not compressed. Verify them on that host with the CLI of the previous version, which reads in the platform charset. Do not use the rotate action of a version up to 3.2.1 for such a topic: with rotation disabled, it renames only the CSV file, under a .gz name although it does not compress it, and the writer of the new file overwrites the keys of the archive in the keystore that it leaves; #332 fixes this in this version. For a plain CSV topic, the rotate action is enough.

Files written with rotation enabled were in UTF-8 already: they failed verification on such a host, and now pass. On such a host, the files written with rotation disabled are now in UTF-8 too: a tool that reads them must decode them as UTF-8.

Rolling back on such a host: the previous version reads in the platform charset, so it fails the tamper-evident files that this version wrote with rotation disabled and a non-ASCII value, the live file included. It reads the non-ASCII values of a plain CSV file that this version wrote with rotation disabled garbled, and does not resume such a file whose header holds a non-ASCII column name. Run the rotate action of this version before the rollback, and verify the archives written after the upgrade with the CLI of this version.

TextWriter.Stream is public: audit-core exports org.forgerock.audit.events.handlers.writers. With a default charset other than UTF-8, a caller of new TextWriter.Stream(out) outside this repository now gets the UTF-8 bytes of a non-ASCII character, and getBytesWritten() counts them; a file that such a caller appends to then mixes two encodings. With a UTF-8 default charset, the output is unchanged.

Tests

• CsvWriterTest.shouldWriteAndVerifyNonAsciiValuesInUtf8, with rotation disabled and enabled: the file is in UTF-8 and passes verification.
• CsvWriterTest.shouldNotVerifyFileNotInUtf8: a file re-encoded in windows-1252 fails with "The file is not valid UTF-8".
• CsvWriterTest.shouldNotResumeFileNotInUtf8: SecureCsvWriter does not resume such a file.
• CsvWriterTest.shouldResumeCsvFileWithNonAsciiHeader.
• CsvAuditEventHandlerTest.testReadingAuditLogEntryWithNonAsciiValue.
• The plain CSV case of the Upgrade section: CsvAuditEventHandlerTest.testReadingAuditLogEntryNotInUtf8 resumes a file re-encoded in windows-1252, reads its old row back with U+FFFD and the row appended after the resume as written; CsvWriterTest.shouldNotResumeCsvFileWithNonAsciiHeaderNotInUtf8 does not resume a file whose header, with a non-ASCII column name, it writes in windows-1252.

The bug shows only with a default charset other than UTF-8, so surefire runs the tests of handler-csv with JAVA_TOOL_OPTIONS=-Dfile.encoding=windows-1252, on every platform and JDK. This replaces a JAVA_TOOL_OPTIONS of the environment in these test JVMs. CsvWriterTest.shouldRunWithDefaultCharsetOtherThanUtf8 fails when the tests run with a UTF-8 default charset, as in a run that does not apply this setting: there, six of the new cases pass against the unfixed code. With windows-1252, every new case but that check fails against the unfixed code, and each of the five changes is caught when reverted alone: the writer, the verifier, the strict decoder, the reader of the handler and the header check. audit/core and audit/handler-csv pass locally on JDK 11 and 27.

Not in this PR

Found by reading the code in the review of this change, older than it, and not reproduced; filed separately:

• Tamper-evident CSV audit: the row HMAC does not cover the column boundaries #344: the row HMAC does not cover the column boundaries, so an edit that moves characters between adjacent columns keeps it.
• CSV audit: resuming a file with more columns than the topic schema throws ArrayIndexOutOfBoundsException #345: StandardCsvWriter throws ArrayIndexOutOfBoundsException when it resumes a file with more columns than the topic schema.
• Tamper-evident CSV audit: reading or querying a topic with no event yet throws NullPointerException #346: reading or querying a tamper-evident topic with no event yet throws NullPointerException. Fixed since by [#346] Tamper-evident CSV audit: read and query a topic with no event yet #354.

TextWriter.Stream still swallows write failures through PrintWriter; that is #349.

[maximthomas]

praise: Every reader and writer of the audit files now agrees on UTF-8, and the tests prove it under a non-UTF-8 default charset.

• CsvSecureVerifier.newBufferedCsvMapReader decodes with StandardCharsets.UTF_8.newDecoder(), so non-UTF-8 bytes can no longer pass for a logged U+FFFD. The plain and the .gz branches share that one reader.
• The surefire JAVA_TOOL_OPTIONS=-Dfile.encoding=windows-1252 fork makes the new cases bite on every CI JDK, 27 included. Each change reverted on its own fails at least one new test: TextWriter.Stream, the verifier's charset, the strict decoder, createCsvMapReader and the StandardCsvWriter header check. Measured on JDK 11 and 17: HEAD green 5/5, all four files at BASE red 5/5.

---

question (non-blocking): Are the U+FFFD read-back of pre-upgrade plain CSV rows and the failed resume of a non-ASCII column name accepted?

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/CsvAuditEventHandler.java:348, StandardCsvWriter.java:60, :69-71

The setup is a Windows JDK 11–17 host (or -Dfile.encoding) with a plain topic and rotation disabled, the default. The old version wrote "Grüße" as FC DF. readEvent/queryEvents now read that file with FileReader(auditFile, UTF_8), which replaces bad bytes, so the old row comes back as "Gr��e" with no error, and an eq query on it stops matching. The header check also reads in UTF-8, so a non-ASCII column name the old version wrote now throws "Resuming an existing CSV file but the headers do not match." where BASE resumed the file. The Upgrade note says only that such a file "is resumed as it is". If both effects are accepted, this is a Minor: say both in the Upgrade note (rotate or move such a file before upgrading), and pin the HEAD outcome with a test that writes a header and a "Grüße" row in windows-1252, resumes the file with StandardCsvWriter and asserts what readEvent returns. If "resumed as it is" was meant to promise intact read-back, this is a Major: the resume needs a strict UTF-8 check that moves a non-UTF-8 file aside.

---

suggestion (non-blocking): The Upgrade note should say that, at this head, the documented failure stops the whole audit service's startup, not one topic.

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/SecureCsvWriter.java:133-135, :176-177, CsvAuditEventHandler.java:223, commons/audit/core/src/main/java/org/forgerock/audit/AuditServiceImpl.java:399-400

The resume throws IOException("The CSV file was tampered: The file is not valid UTF-8") inside the constructor's outer try, and the constructor rethrows it as RuntimeException("Error when initializing a secure CSV writer"). CsvAuditEventHandler.startup() catches only IOException per topic, and AuditServiceImpl.startup() catches only ResourceException. So on an affected host the RuntimeException escapes both, and the audit service never reaches RUNNING. A really tampered file already took this path. This PR sends untampered legacy files down it too. #341 makes the failure per topic, so either say this in the note or merge after #341.

---

suggestion (non-blocking): The Upgrade note should also cover the downgrade direction.

commons/audit/core/src/main/java/org/forgerock/audit/events/handlers/writers/TextWriter.java:73

The previous version's verifier reads with new InputStreamReader(in), the platform charset, and signs the UTF-8 bytes of what it decoded (CsvSecureUtils.java:87). After a rollback on a Windows JDK 11–17 host, a rotation-disabled secure file this version wrote with "ü" (C3 BC) decodes as "ü". The previous version then refuses the live file ("The CSV file was tampered: The HMac at row N is not correct."), and its CLI fails the archives written after the upgrade. One line would cover it: before rolling back on such a host, run the rotate action, and verify the archives written after the upgrade with the new CLI.

---

suggestion (non-blocking): Name the charset change of the public TextWriter.Stream in audit-core's release notes.

commons/audit/core/src/main/java/org/forgerock/audit/events/handlers/writers/TextWriter.java:73

audit-core is published, and its MANIFEST exports org.forgerock.audit.events.handlers.writers. When the default charset is not UTF-8, outside callers of new TextWriter.Stream(out) (OpenIDM, OpenAM, custom handlers) now get 63 61 66 c3 a9 0a for "café\n" instead of 63 61 66 e9 0a. getBytesWritten() returns 6 instead of 5, a default-charset reader gets "café", and appending to an older file mixes two encodings. Under a UTF-8 default the output is unchanged (probed on JDK 11/17/21/26). No code change is needed.

---

suggestion (non-blocking): Nothing asserts that the handler-csv fork really runs under a non-UTF-8 default charset.

commons/audit/handler-csv/pom.xml:120, commons/audit/handler-csv/src/test/java/org/forgerock/audit/handlers/csv/CsvWriterTest.java:183

Under a UTF-8 default, BASE's reader and writer agree with each other. That happens in an IDE run that does not apply surefire's environmentVariables, or after a pom edit that drops the block. Then shouldWriteAndVerifyNonAsciiValuesInUtf8[false,true], shouldResumeCsvFileWithNonAsciiHeader and testReadingAuditLogEntryWithNonAsciiValue pass against the unfixed code, and only shouldNotVerifyFileNotInUtf8 stays red.

    @Test
    public void shouldRunUnderDefaultCharsetOtherThanUtf8() {
        // surefire sets JAVA_TOOL_OPTIONS=-Dfile.encoding=windows-1252; under a UTF-8 default charset the non-ASCII
        // tests pass against a reader or a writer that uses the default charset
        assertThat(java.nio.charset.Charset.defaultCharset()).isNotEqualTo(UTF_8);
    }

Pin: this test turns red as soon as the pom block is gone or not applied. That includes an IDE run, which is intended.

---

suggestion (non-blocking): No test pins the documented refusal to resume a pre-upgrade live file that is not UTF-8.

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/SecureCsvWriter.java:133-135, CsvWriterTest.java:193

shouldNotVerifyFileNotInUtf8 calls the verifier directly. No test drives SecureCsvWriter's resume with such a file, and that resume is the behaviour the Upgrade note warns operators about. The constructor wraps the failure in a RuntimeException, so the pin asserts the root cause (add import static org.assertj.core.api.Assertions.assertThatThrownBy;):

    @Test
    public void shouldNotResumeFileNotInUtf8() throws Exception {
        final File csvFile = writeSecureCsvFile(false, "Grüße");
        // the live file as the writer wrote it with a windows-1252 default charset, before it wrote in UTF-8
        final String content = new String(Files.readAllBytes(csvFile.toPath()), UTF_8);
        Files.write(csvFile.toPath(), content.getBytes("windows-1252"));
        final CsvSecurity csvSecurity = new CsvSecurity();
        csvSecurity.setEnabled(true);
        csvSecurity.setSignatureInterval("5 minutes");
        final CsvAuditEventHandlerConfiguration configuration = new CsvAuditEventHandlerConfiguration();
        configuration.setSecurity(csvSecurity);
        configuration.setRotationRetentionCheckInterval("5 minutes");

        assertThatThrownBy(() -> new SecureCsvWriter(csvFile, new String[] { "child1", "child2", "child3" },
                CsvPreference.EXCEL_PREFERENCE, configuration, keyStoreHandler, random))
                .hasRootCauseMessage("The CSV file was tampered: The file is not valid UTF-8");
    }

Pin: if the writer resumes the file instead of verifying it, or verification passes, this test turns red. Not compiled or run by this review.

---

suggestion (non-blocking): "The file is not valid UTF-8" names no row, and the decoder's read-ahead can mask an earlier row's HMAC failure.

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/CsvSecureVerifier.java:152-154

InputStreamReader decodes ahead about 8 KB at a time. In a small file with a wrong HMAC at row 2 and a 0xFC byte in row 4, the exception surfaces first, and the result says "not valid UTF-8" instead of "The HMac at row 2 is not correct.". Verification still fails; the operator just can't locate the bad row. Optionally, track the last row that verified and say "… not valid UTF-8 after row R". csvReader.getRowNumber() at the throw does not give the malformed row, because of the read-ahead.

---

nitpick (non-blocking): The pom comment should say that this setting replaces an inherited JAVA_TOOL_OPTIONS.

commons/audit/handler-csv/pom.xml:117-121

A developer or CI image that sets JAVA_TOOL_OPTIONS (proxy, truststore, -Xmx, an agent) loses it in handler-csv's test forks only. The project's CI sets none. Moving the setting to argLine does not work as a drop-in: the root argLine is a property of the jdk16.options profile only (pom.xml:1468-1476), so ${argLine} in this pom would stay literal on JDK 11.

---

nitpick (non-blocking): The Upgrade note gives one failure message, but a legacy file can also fail with "The HMac at row N is not correct.".

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/CsvSecureVerifier.java:152-154

The strict decoder rejects only malformed bytes. Some platform-charset bytes are valid UTF-8: windows-1252 "é" is C3 A9, which decodes to "é", and multi-byte charsets such as MS932 or GBK hit this more often. The HMAC was computed over the UTF-8 bytes of "é" (CsvSecureUtils.java:87), so such a file fails with a message that reads as tampering.

[vharseko]

Addressed in b51fdf0, and the description is updated. The branch is rebased onto the current master first, without conflicts; the change of the first commit is as you reviewed it.

• question (U+FFFD read-back, non-ASCII column name): accepted as a Minor. The Upgrade section names both effects and says to run the rotate action on each topic before the upgrade; with rotation disabled, the action goes through rotateAndReopenWriter(). Moving a file that is not UTF-8 aside on resume would move the operators' files at startup, a new behaviour outside Tamper-evident CSV audit: archives are written in UTF-8 but verified in the platform charset #336. CsvAuditEventHandlerTest.testReadingAuditLogEntryNotInUtf8 pins the read-back: the row written before reads back as Gr��e, the row appended after the resume as written. CsvWriterTest.shouldNotResumeCsvFileWithNonAsciiHeaderNotInUtf8 pins the header. A column name is a JSON pointer of the topic schema, so only a custom schema has a non-ASCII one, and that IOException fails the topic only.
• startup of the whole service: the Upgrade section says that until [#340] Tamper-evident CSV audit: close the file of a writer that fails to resume it #341 the failure of the live file stops the startup of the audit service, and why.
• downgrade: added to the Upgrade section.
• public TextWriter.Stream: added to the Upgrade section. commons/audit/CHANGELOG.md stops at 6.2.0 and is not kept up, so the PR description is the release note.
• default charset of the fork: CsvWriterTest.shouldRunWithDefaultCharsetOtherThanUtf8, as suggested.
• resume of a pre-upgrade live file: CsvWriterTest.shouldNotResumeFileNotInUtf8. It asserts hasStackTraceContaining instead of hasRootCauseMessage: [#340] Tamper-evident CSV audit: close the file of a writer that fails to resume it #341 throws the IOException as is, without a cause, and hasRootCauseMessage would fail on it once both are merged.
• pom comment: done, with the reason it is not in argLine.
• second failure message: the Upgrade section names "The HMac at row N is not correct." for a file whose bytes happen to be valid UTF-8.
• row number of "not valid UTF-8": not taken. The file fails verification either way, and the message is true of it. Because of the read-ahead, "after row R" would name the last row verified before the decoder threw, up to 8 KB before the malformed byte, and a byte offset needs a counting stream under the decoder. Neither locates the bad bytes much better than the file name does.

[maximthomas]

praise: the charset is fixed at the readers and writers themselves, and a test run that cannot show the bug fails instead of passing.

• CsvSecureVerifier.java:192 decodes with StandardCharsets.UTF_8.newDecoder(), so a file that is not UTF-8 fails as such (:152-154) instead of passing for a U+FFFD that the writer logged.
• TextWriter.java:73 writes UTF-8, as RotatableWriter.java:353 does: rotation-disabled and rotated files now have one encoding.
• CsvWriterTest.shouldRunWithDefaultCharsetOtherThanUtf8 (CsvWriterTest.java:182) makes a run without the windows-1252 fork fail. In a UTF-8 run of this head it is the only failure.

---

issue (blocking): The Upgrade note's workaround (rotate before the upgrade, then verify the archives with the previous CLI) fails on every release: their rotate action leaves the archive without its keystore.

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/CsvSecureVerifier.java:152-154

The keystore rename on the rotate action is #332 (0d6ddfc), and no release contains it: git tag --contains 0d6ddfccf prints nothing, and the newest tag is 3.2.1. In 3.2.1, the rotate action with rotation disabled renames only tamper-evident-<topic>.csv (CsvAuditEventHandler.java:439, :460-473 at 3.2.1) and reopens the writer on the same .keystore. The 3.2.1 CLI then opens <archive>.keystore (CsvSecureArchiveVerifier.java:80 at 3.2.1), which does not exist, so the archive fails with "Expecting to find an initial key into the keystore." This version's CLI fails it with "The file is not valid UTF-8". #332's own comment adds that the reopened file overwrites the archive's keys in the keystore left behind. An operator who follows the note loses the verifiability that skipping it would have kept. I traced this by reading the code; I did not run it end to end against a 3.2.1 build.

On such a host, before the upgrade, stop the service and move the files of each topic aside. For a
tamper-evident topic, move `tamper-evident-<topic>.csv` together with
`tamper-evident-<topic>.csv.keystore`, renamed `<archive>` and `<archive>.keystore`, and verify them
with the CLI of the previous version. Do not use the `rotate` action of a version up to 3.2.1 for a
tamper-evident topic with rotation disabled: it renames only the CSV file, and the new file
overwrites the keys of the archive in the keystore. For a plain CSV topic, the `rotate` action is
enough.

Or: merge after a release that carries #332, and name that release as the minimum for the rotate advice. Not checked by this review: which <archive> names the 3.2.1 CLI lists. That depends on the TimestampFilenameFilter of TimeStampFileNamingPolicy and on the CLI's prefix and suffix options, so take the names from that code.

---

issue (non-blocking): A pre-upgrade live file that fails the new check leaves the topic's FileOutputStream open.

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/SecureCsvWriter.java:104, :131-135, :176-177, :215

The constructor opens the file (:104 → :215) before verify() (:131-135), and the catch at :176-177 wraps the failure without closing anything. The base resumed an untampered windows-1252 live file and kept the stream as the live writer. This head throws "The file is not valid UTF-8" and leaves the stream open until GC; on Windows that handle has no share-delete. CsvWriterTest.shouldNotResumeFileNotInUtf8 (CsvWriterTest.java:215) leaks it the same way. This is #340's mechanism and #341 removes it, so there is nothing to change here, only the merge order with #341 to keep in mind.

---

suggestion (non-blocking): The Rollback paragraph covers only tamper-evident files.

commons/audit/handler-csv/src/main/java/org/forgerock/audit/handlers/csv/StandardCsvWriter.java:60, commons/audit/core/src/main/java/org/forgerock/audit/events/handlers/writers/TextWriter.java:73

The previous version reads the header and rows of a plain rotation-disabled file in the platform charset (StandardCsvWriter.java:58 at the base). On windows-1252, suppose a file that this version created has a custom column "Grüße". The previous version's resume fails with "Resuming an existing CSV file but the headers do not match." I measured this with this head's writer and the base's header read: shouldResumeCsvFileWithNonAsciiHeader:272 fails with that message. Non-ASCII values written after the upgrade also read back as "Grüße" through readEvent and queryEvents.

A plain CSV topic that this version wrote with rotation disabled is in UTF-8: the previous version
reads the rows written after the upgrade garbled, and does not resume a file whose header holds a
non-ASCII column name. Run the `rotate` action before the rollback.

---

suggestion (non-blocking): The Upgrade section does not tell readers outside the product that rotation-disabled files are now UTF-8.

commons/audit/core/src/main/java/org/forgerock/audit/events/handlers/writers/TextWriter.java:73

Rotated files were in UTF-8 already (RotatableWriter.java:353). Rotation-disabled files move from the platform charset (TextWriter.java:71 at the base) to UTF-8. A log shipper or script that decodes them in windows-1252 reads "café" in a row written after the upgrade as "café" (probed on JDK 11, 17 and 21), and a resumed file mixes both encodings. The description names the mixed encoding only for callers of new TextWriter.Stream(out).

Files written with rotation disabled are now in UTF-8, as rotated files already were: a tool that
reads them must decode them as UTF-8, and a file resumed across the upgrade mixes both encodings.

---

nitpick (non-blocking): "four of the new cases pass against the unfixed code" under a UTF-8 default charset: six of them do.

commons/audit/handler-csv/src/test/java/org/forgerock/audit/handlers/csv/CsvWriterTest.java:182

With the base's main code, this head's tests and -Dfile.encoding=UTF-8 (JDK 17.0.11), only shouldNotVerifyFileNotInUtf8, shouldNotResumeFileNotInUtf8 and the guard fail, 3 of 9. testReadingAuditLogEntryNotInUtf8 and shouldNotResumeCsvFileWithNonAsciiHeaderNotInUtf8 pass there too. No coverage is lost, because the guard fails any UTF-8 run.

---

nitpick (non-blocking): "Each new case of the charset fails without its fix" is not true of shouldNotResumeCsvFileWithNonAsciiHeaderNotInUtf8 against the whole unfixed code.

commons/audit/handler-csv/src/test/java/org/forgerock/audit/handlers/csv/CsvWriterTest.java:277

In the windows-1252 fork with the base's main code, 7 of 9 cases fail and this one passes. reencodeInWindows1252 decodes the base's windows-1252 header as UTF-8 and writes U+FFFD back as ?, so the header mismatches anyway. The case fails only when StandardCsvWriter.java:60 is reverted alone (:286). Write "each fails when its change is reverted alone", or name this case as the exception.

[vharseko]

Addressed in c0105c9, and the description is updated. The branch is on the current master; nothing to rebase.

• issue (blocking), the rotate workaround: confirmed at 3.2.1. With rotation disabled, resetAndReopenWriter(topic, true) renames only the CSV file, and the writer of the new file writes its initial and current keys (SecureCsvWriter.java:197-198 at 3.2.1) into the same tamper-evident-<topic>.csv.keystore. The Upgrade section now says to stop the service and move the CSV file and its keystore aside under an archive name, and not to use the rotate action of a version up to 3.2.1 for such a topic. Merging after a release that carries [#331] Tamper-evident CSV audit: verify compressed archives, rename the keystore on the rotate action #332 would not help the operators who upgrade from 3.2.1 directly. Two details differ from the review:
  • The CLI of this version does not get to "The file is not valid UTF-8" on such an archive: the CsvSecureVerifier constructor fails first, on the missing keystore, with "Expecting to find an initial key into the keystore." (CsvSecureVerifier.java:82), as the 3.2.1 CLI does.
  • The 3.2.1 rotate action names the archive with the default suffix -yyyy.MM.dd-HH.mm.ss.gz (new TimeStampFileNamingPolicy(auditLogFile, null, null)), .gz included, without compressing it. Even with its keystore, the CLI of this version would read it through GZIPInputStream. So the note names the archive without .gz. A copy of the accept() of the 3.2.1 TimestampFilenameFilter, built as the 3.2.1 CLI builds it with the default --prefix and --suffix, lists tamper-evident-access.csv-2026.10.19-12.00.00 and not its .keystore.
• issue (non-blocking), open file on a failed resume: the Upgrade section adds that the failure also leaves the live file open (Tamper-evident CSV audit: a writer that fails to resume its file leaks it, and stops the audit service from starting #340). No code change, as you said; the note about [#340] Tamper-evident CSV audit: close the file of a writer that fails to resume it #341 already covers the merge order.
• Rollback of a plain CSV topic: added. The rollback advice keeps the rotate action, as this version's action carries [#331] Tamper-evident CSV audit: verify compressed archives, rename the keystore on the rotate action #332.
• Files without rotation are now UTF-8: added, for a host whose default charset is not UTF-8. The plain CSV bullet already says that a resumed file mixes both encodings.
• "four" → "six": fixed. Measured with the main code of master, this head's tests and the pom of master (UTF-8 default, JDK 27): only shouldNotVerifyFileNotInUtf8, shouldNotResumeFileNotInUtf8 and shouldRunWithDefaultCharsetOtherThanUtf8 fail.
• shouldNotResumeCsvFileWithNonAsciiHeaderNotInUtf8 against the whole unfixed code: fixed in the test rather than in the sentence. The case now writes the header in windows-1252 itself, with CsvFormatter.formatHeader(), as the writer wrote it before, instead of re-encoding the file of the writer under test. In the windows-1252 fork with the main code of master, 8 of the 9 new cases fail now, all but shouldRunWithDefaultCharsetOtherThanUtf8; with only StandardCsvWriter reverted, both header cases fail, as before. The description says so.

[maximthomas]

praise: The new head closes the last gap between the tests and the description, and the upgrade advice now holds against the released CLI.

• CsvWriterTest.shouldNotResumeCsvFileWithNonAsciiHeaderNotInUtf8 now writes the header bytes itself, with CsvFormatter.formatHeader and Files.write in windows-1252. Run against the unfixed main code in the windows-1252 fork, it now fails at :288 ("Expecting code to raise a throwable."); before, it passed. Every new case except shouldRunWithDefaultCharsetOtherThanUtf8 now fails against the unfixed code, as the Tests paragraph says (JDK 17, 8 of 9).
• The 3.2.1 CLI handles the archive naming that the Upgrade section asks for: TimestampFilenameFilter.accept lists tamper-evident-<topic>.csv-yyyy.MM.dd-HH.mm.ss and skips its .keystore, and CsvSecureArchiveVerifier opens <archive>.keystore.