Repository navigation
Prevent invalid releases - additional tests - #215
pietrygamat wants to merge 3 commits into
Conversation
The macos-14 runner image retires on November 2, 2026, and its brownouts already cancel the macOS jobs. Move them to macos-15, which is arm64 too and still sets XCODE_16_DEVELOPER_DIR. macos-15 defaults to Xcode 16.4, whose xcrun shims (/usr/bin/make, /usr/bin/c++) read SDKROOT and cannot resolve MacOSX10.9.sdk, so CMake fails to configure. The workflows now export only CMAKE_OSX_SYSROOT, which ant/build.xml passes to CMake as -DCMAKE_OSX_SYSROOT, and the shims keep the default SDK. This does not change the compatibility of the natives: they are still built against the same MacOSX10.9.sdk and MacOSX11.0.sdk with the same MACOSX_DEPLOYMENT_TARGET values.
Every native carries a JSSC-IDENT line; check it where natives are built: - NativeIdentTest reads the ident of the library built in this run without loading it, with the parser in NativeIdent, and compares it with what ant/build.xml passed to CMake. Cross builds now run it instead of skipping all tests; it is the only test they run. SerialNativeInterfaceTest checks that a host build loads the library it built. - ant/build.xml computes the ident values once, in ident-props, and exports them together with the test selection to maven, which passes them to the tests. - The committed jssc_SerialNativeInterface.h stands for the JNI surface, so a JNI change is a change to src/main/cpp like any other native change. cmake-build rewrites it when the header generated from SerialNativeInterface.java differs, and build.yml fails when a build changed it, so an updated header can't be left out of a commit. The written header is the same on every OS and JDK: javac on Windows writes long constants with MSVC's i64 suffix, which GCC and Clang reject, so they get LL as elsewhere, and line endings are LF. The committed copy is refreshed (OS_UNKNOWN was missing). - CMakeLists.txt refuses ident values that are not printable ASCII without spaces or quotes.
PrecompiledNativesTest checks the precompiled natives on the classpath, which a release ships as they are: every target the caller names in jssc.native.targets (comma separated, required) has one library with one ident, there are no other directories, and each has the expected version, target = directory, one CI rev and origin, no -dirty or unknown rev and no origin=local, and no change to the native sources since rev. It reads the idents with NativeIdent. - The test runs on request only: the verify-natives surefire execution runs it, and only it, when -Djssc.native.targets is given (the verify-natives profile) or with -Djssc.verify.natives.skip=false, which fails without the targets. The package profile now skips the tests with skipTests instead of maven.test.skip, so the test classes are still compiled. - release:prepare runs it through the release plugin's arguments, refusing natives not built by this repository's CI for the release version from these sources. The expected origin defaults to GITHUB_REPOSITORY. The releaseVersion input now reaches its scripts through the environment instead of being substituted into them. - release:perform passes the same arguments to its build in a fresh clone of the tag, so the test runs there again; that clone is no longer shallow, so the rev the natives were built from is in its history. - cross-compile.yml runs it before the squashed branch is pushed, with the rev and origin of the run. - Both workflows name the targets in NATIVE_TARGETS; the one in cross-compile.yml lists what its matrices build.
| <target name="ident-props"> | ||
| <!-- Build ident: blank lets CMake take the revision from git and set origin "local" --> | ||
| <property description="fallback value" name="jssc.build.rev" value=""/> | ||
| <property description="fallback value" name="jssc.build.origin" value=""/> | ||
| <!-- Native version: jssc.native.version if given, else project.version, | ||
| without its qualifier if jssc.native.release=true --> | ||
| <condition property="jssc.native.version" value="${release.version}" else="${project.version}"> | ||
| <istrue value="${jssc.native.release}"/> | ||
| </condition> | ||
| </target> | ||
|
|
There was a problem hiding this comment.
Moved this section from cmake-generate section below. These props are now exported back to maven and to java/junit.
| <!-- The committed JNI header follows the generated one, so a JNI change is a change in src/main/cpp. | ||
| Unlike the natives, the build writes it to the source tree: it is a source, and CI fails when a build | ||
| changed it (build.yml), so an updated header can't be left out of a commit --> | ||
| <property name="jni.header.generated" location="${cmake.generated.directory}/jssc_SerialNativeInterface.h"/> | ||
| <property name="jni.header.committed" location="${ant.project.basedir}/src/main/cpp/jssc_SerialNativeInterface.h"/> | ||
| <fail message="No JNI header was generated at ${jni.header.generated}"> | ||
| <condition> | ||
| <not><available file="${jni.header.generated}"/></not> | ||
| </condition> | ||
| </fail> | ||
| <!-- The header as it is committed, the same on every OS and JDK: javac on Windows writes long constants | ||
| with MSVC's i64 suffix, which GCC and Clang reject, so they get LL as elsewhere --> | ||
| <property name="jni.header.expected" location="${cmake.generated.directory}/committed/jssc_SerialNativeInterface.h"/> | ||
| <copy file="${jni.header.generated}" tofile="${jni.header.expected}" overwrite="true"> | ||
| <filterchain> | ||
| <fixcrlf eol="lf"/> | ||
| <tokenfilter><replaceregex pattern="^(#define \S+ -?[0-9]+)i64$" replace="\1LL"/></tokenfilter> | ||
| </filterchain> | ||
| </copy> | ||
| <!-- Line endings don't count, so a checkout with CRLF is left alone --> | ||
| <condition property="jni.header.outdated"> | ||
| <not><filesmatch file1="${jni.header.expected}" file2="${jni.header.committed}" textfile="true"/></not> | ||
| </condition> | ||
| <copy if:set="jni.header.outdated" file="${jni.header.expected}" tofile="${jni.header.committed}" overwrite="true"/> | ||
| <echo if:set="jni.header.outdated" level="warning" | ||
| message="The JNI surface of SerialNativeInterface.java changed: updated ${jni.header.committed}, commit it"/> |
There was a problem hiding this comment.
Our java builds generate jni header and use it for compilation, so the committed one is unused. This part detects that the generated output is the same as the committed, and if not - replaces the committed one so the author of the change may commit it and keep in sync. I run tests against JDK 8, 11, 17, 21 and 25 and confirm, they all produce identical output, so this should not randomly change, but with one quirk:
The header generator distinguishes between Windows and other OSes, so on Windows the headers have 1 difference:
case LONG:
// Visual C++ supports the i64 suffix, not LL.
valueStr = value.toString() + ((isWindows) ? "i64" : "LL");
break;
This is a compatibility layer that to my research, is a relic of 1998 era of MSVC compilers, and the LL format works on any modern compiler including Windows. Our repo has a header with LL commited, and non java raw cmake build works.
// ours: #define jssc_SerialNativeInterface_ERR_PORT_BUSY -1LL
// windows: #define jssc_SerialNativeInterface_ERR_PORT_BUSY -1i64
We don't want Windows users see this as a change, and especially to make them commit i64 which I found is NOT accepted on Linux, so the code above replaces i64 -> LL - I could not find a pattern that would cause a false positive.
Anyway, the header that current build generates is still one that the compiler uses, so the replacement is a form of tracking changes in the repo rather than a rewrite of the compilation process.
Addresses #211: additional tests focused on preventing #205 (wrong natives packaged into jar and sent to Maven Central).
I added multiple checks that run on different points of the lifecycle: because the release cannot just be built in a single run - we need to build natives in different environments, then there is an issue of tagging the git commit that the release version was built from. For this reason there are separate tests run against a binaries that were built this run, and separate that are run against natives we store in src and ship. The two commits (ignoring the required mac runner update) are:
1. Check the ident of the natives built in a run
#213 puts an ident into every native, but nothing reads it yet. This makes every build verify it's own output.
NativeIdentTestclass reads the marker so we can be sure the binary exist, is placed in right dir, built for arch we targeted - even in cross-compile build, where it cannot be loaded by java. Cross builds can't run the regular tests, so until now they ran none; they now run this one, and only this one. When build for host arch, theSerialNativeInterfaceTestalso checks if the version reported by the loaded library is as expected. Some reorganization and additional properties in pom and ant/build were required to make the test trigger correctly only when natives are rebuilt, and also when natives are being rebuild with version different than the pom.xml - which happens during release preparation.One aspect of the change is that it also verifiy if
src/main/cpp/jssc_SerialNativeInterface.his up-to-date against theSerialNativeInterfaceclass. It was not - the header file in src (kept since #195) was obsolete, but keeping it up-to-date not only allows builds outside of java, but also allows us to identify native binaries as stale (if the header file was changed after commit imprinted in the rev field, we can tell without running). To avoild problems with keeping the file up to date, the build now puts the updated version of header file in src (only after significant change affecting C compilation). So the author of the change inSerialNativeInterfaceTest.classwill be prompted to commit the header as well, and if they don't - the CI will reject the PR.2. Verify the version and provenance of the natives before a release
A release ships the precompiled natives committed under
src/main/resources-precompiledas they are. This adds the check that they were built for this release:PrecompiledNativesTest. Disabled by default, requires passing a flag to the mvn command. It is run incross-compile.ymlto verify all binaries are a matching set, and also during a release to Maven Central.Run some test runs from my fork, the tests caught inconsistencies where expected (current master for example) and passed up to the maven central staging when natives are correct.