Skip to content

Prevent invalid releases - additional tests - #215

Open
pietrygamat wants to merge 3 commits into
java-native:masterfrom
pietrygamat:verify-natives-during-release
Open

pietrygamat wants to merge 3 commits into
java-native:masterfrom
pietrygamat:verify-natives-during-release

Conversation

@pietrygamat

@pietrygamat pietrygamat commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Addresses #211: additional tests focused on preventing #205 (wrong natives packaged into jar and sent to Maven Central).

I added multiple checks that run on different points of the lifecycle: because the release cannot just be built in a single run - we need to build natives in different environments, then there is an issue of tagging the git commit that the release version was built from. For this reason there are separate tests run against a binaries that were built this run, and separate that are run against natives we store in src and ship. The two commits (ignoring the required mac runner update) are:

1. Check the ident of the natives built in a run

#213 puts an ident into every native, but nothing reads it yet. This makes every build verify it's own output. NativeIdentTest class reads the marker so we can be sure the binary exist, is placed in right dir, built for arch we targeted - even in cross-compile build, where it cannot be loaded by java. Cross builds can't run the regular tests, so until now they ran none; they now run this one, and only this one. When build for host arch, the SerialNativeInterfaceTest also checks if the version reported by the loaded library is as expected. Some reorganization and additional properties in pom and ant/build were required to make the test trigger correctly only when natives are rebuilt, and also when natives are being rebuild with version different than the pom.xml - which happens during release preparation.

One aspect of the change is that it also verifiy if src/main/cpp/jssc_SerialNativeInterface.h is up-to-date against the SerialNativeInterface class. It was not - the header file in src (kept since #195) was obsolete, but keeping it up-to-date not only allows builds outside of java, but also allows us to identify native binaries as stale (if the header file was changed after commit imprinted in the rev field, we can tell without running). To avoild problems with keeping the file up to date, the build now puts the updated version of header file in src (only after significant change affecting C compilation). So the author of the change in SerialNativeInterfaceTest.class will be prompted to commit the header as well, and if they don't - the CI will reject the PR.

2. Verify the version and provenance of the natives before a release

A release ships the precompiled natives committed under src/main/resources-precompiled as they are. This adds the check that they were built for this release: PrecompiledNativesTest. Disabled by default, requires passing a flag to the mvn command. It is run in cross-compile.yml to verify all binaries are a matching set, and also during a release to Maven Central.

Run some test runs from my fork, the tests caught inconsistencies where expected (current master for example) and passed up to the maven central staging when natives are correct.

The macos-14 runner image retires on November 2, 2026, and its brownouts
already cancel the macOS jobs. Move them to macos-15, which is arm64 too
and still sets XCODE_16_DEVELOPER_DIR.

macos-15 defaults to Xcode 16.4, whose xcrun shims (/usr/bin/make,
/usr/bin/c++) read SDKROOT and cannot resolve MacOSX10.9.sdk, so CMake
fails to configure. The workflows now export only CMAKE_OSX_SYSROOT,
which ant/build.xml passes to CMake as -DCMAKE_OSX_SYSROOT, and the shims
keep the default SDK.

This does not change the compatibility of the natives: they are still
built against the same MacOSX10.9.sdk and MacOSX11.0.sdk with the same
MACOSX_DEPLOYMENT_TARGET values.
Every native carries a JSSC-IDENT line; check it where natives are built:

- NativeIdentTest reads the ident of the library built in this run without
  loading it, with the parser in NativeIdent, and compares it with what
  ant/build.xml passed to CMake. Cross builds now run it instead of skipping
  all tests; it is the only test they run. SerialNativeInterfaceTest checks
  that a host build loads the library it built.
- ant/build.xml computes the ident values once, in ident-props, and exports
  them together with the test selection to maven, which passes them to the
  tests.
- The committed jssc_SerialNativeInterface.h stands for the JNI surface, so a
  JNI change is a change to src/main/cpp like any other native change.
  cmake-build rewrites it when the header generated from
  SerialNativeInterface.java differs, and build.yml fails when a build
  changed it, so an updated header can't be left out of a commit. The
  written header is the same on every OS and JDK: javac on Windows writes
  long constants with MSVC's i64 suffix, which GCC and Clang reject, so they
  get LL as elsewhere, and line endings are LF. The committed copy is
  refreshed (OS_UNKNOWN was missing).
- CMakeLists.txt refuses ident values that are not printable ASCII without
  spaces or quotes.
PrecompiledNativesTest checks the precompiled natives on the classpath, which
a release ships as they are: every target the caller names in
jssc.native.targets (comma separated, required) has one library with one
ident, there are no other directories, and each has the expected version,
target = directory, one CI rev and origin, no -dirty or unknown rev and no
origin=local, and no change to the native sources since rev. It reads the
idents with NativeIdent.

- The test runs on request only: the verify-natives surefire execution runs
  it, and only it, when -Djssc.native.targets is given (the verify-natives
  profile) or with -Djssc.verify.natives.skip=false, which fails without the
  targets. The package profile now skips the tests with skipTests instead of
  maven.test.skip, so the test classes are still compiled.
- release:prepare runs it through the release plugin's arguments, refusing
  natives not built by this repository's CI for the release version from
  these sources. The expected origin defaults to GITHUB_REPOSITORY. The
  releaseVersion input now reaches its scripts through the environment
  instead of being substituted into them.
- release:perform passes the same arguments to its build in a fresh clone of
  the tag, so the test runs there again; that clone is no longer shallow, so
  the rev the natives were built from is in its history.
- cross-compile.yml runs it before the squashed branch is pushed, with the
  rev and origin of the run.
- Both workflows name the targets in NATIVE_TARGETS; the one in
  cross-compile.yml lists what its matrices build.
Comment thread ant/build.xml
Comment on lines +206 to +216
<target name="ident-props">
<!-- Build ident: blank lets CMake take the revision from git and set origin "local" -->
<property description="fallback value" name="jssc.build.rev" value=""/>
<property description="fallback value" name="jssc.build.origin" value=""/>
<!-- Native version: jssc.native.version if given, else project.version,
without its qualifier if jssc.native.release=true -->
<condition property="jssc.native.version" value="${release.version}" else="${project.version}">
<istrue value="${jssc.native.release}"/>
</condition>
</target>

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this section from cmake-generate section below. These props are now exported back to maven and to java/junit.

Comment thread ant/build.xml
Comment on lines +310 to +335
<!-- The committed JNI header follows the generated one, so a JNI change is a change in src/main/cpp.
Unlike the natives, the build writes it to the source tree: it is a source, and CI fails when a build
changed it (build.yml), so an updated header can't be left out of a commit -->
<property name="jni.header.generated" location="${cmake.generated.directory}/jssc_SerialNativeInterface.h"/>
<property name="jni.header.committed" location="${ant.project.basedir}/src/main/cpp/jssc_SerialNativeInterface.h"/>
<fail message="No JNI header was generated at ${jni.header.generated}">
<condition>
<not><available file="${jni.header.generated}"/></not>
</condition>
</fail>
<!-- The header as it is committed, the same on every OS and JDK: javac on Windows writes long constants
with MSVC's i64 suffix, which GCC and Clang reject, so they get LL as elsewhere -->
<property name="jni.header.expected" location="${cmake.generated.directory}/committed/jssc_SerialNativeInterface.h"/>
<copy file="${jni.header.generated}" tofile="${jni.header.expected}" overwrite="true">
<filterchain>
<fixcrlf eol="lf"/>
<tokenfilter><replaceregex pattern="^(#define \S+ -?[0-9]+)i64$" replace="\1LL"/></tokenfilter>
</filterchain>
</copy>
<!-- Line endings don't count, so a checkout with CRLF is left alone -->
<condition property="jni.header.outdated">
<not><filesmatch file1="${jni.header.expected}" file2="${jni.header.committed}" textfile="true"/></not>
</condition>
<copy if:set="jni.header.outdated" file="${jni.header.expected}" tofile="${jni.header.committed}" overwrite="true"/>
<echo if:set="jni.header.outdated" level="warning"
message="The JNI surface of SerialNativeInterface.java changed: updated ${jni.header.committed}, commit it"/>

@pietrygamat pietrygamat Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our java builds generate jni header and use it for compilation, so the committed one is unused. This part detects that the generated output is the same as the committed, and if not - replaces the committed one so the author of the change may commit it and keep in sync. I run tests against JDK 8, 11, 17, 21 and 25 and confirm, they all produce identical output, so this should not randomly change, but with one quirk:

The header generator distinguishes between Windows and other OSes, so on Windows the headers have 1 difference:

            case LONG:
                // Visual C++ supports the i64 suffix, not LL.
                valueStr = value.toString() + ((isWindows) ? "i64" : "LL");
                break;

This is a compatibility layer that to my research, is a relic of 1998 era of MSVC compilers, and the LL format works on any modern compiler including Windows. Our repo has a header with LL commited, and non java raw cmake build works.

// ours:     #define jssc_SerialNativeInterface_ERR_PORT_BUSY -1LL
// windows:  #define jssc_SerialNativeInterface_ERR_PORT_BUSY -1i64

We don't want Windows users see this as a change, and especially to make them commit i64 which I found is NOT accepted on Linux, so the code above replaces i64 -> LL - I could not find a pattern that would cause a false positive.

Anyway, the header that current build generates is still one that the compiler uses, so the replacement is a form of tracking changes in the repo rather than a rewrite of the compilation process.

@pietrygamat
pietrygamat requested a review from tresf November 1, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant