Skip to content

Throw LinkageError from ReflectionUtil.newInstance for unlinkable classes - #191

Merged
vharseko merged 3 commits into
OpenIdentityPlatform:masterfrom
vharseko:fix-connector-config-instantiation
Oct 19, 2026
Merged

vharseko merged 3 commits into
OpenIdentityPlatform:masterfrom
vharseko:fix-connector-config-instantiation

Conversation

@vharseko

@vharseko vharseko commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Summary

#134 replaced Class.newInstance() with ReflectionUtil.newInstance(), which goes through MethodHandles.Lookup.findConstructor. That changed how a connector configuration class with a missing dependency fails:

  • before: Class.newInstance() threw NoClassDefFoundError. The catch (Exception) in createDefaultAPIConfiguration let it through, and LocalConnectorInfoManagerImpl (catch (NoClassDefFoundError), likewise the OSGi managers) skipped that connector and logged an error;
  • now: findConstructor reports the same NoClassDefFoundError as IllegalAccessException: no such constructor. That is an Exception, so it is wrapped in a ConnectorException that nobody catches, and the whole bundle scan fails.

Found through OpenIdentityPlatform/OpenIDM#259 (bump to openicf 2.1.0-SNAPSHOT): ScriptedCRESTConfiguration in groovy-connector refers to httpasyncclient, which is not on the OpenIDM test classpath, and ConnectorInfoProviderServiceTest.beforeTest fails with LocalManager initialisation failed; 126 tests of the module are skipped.

Caused by: ConnectorException: java.lang.IllegalAccessException: no such constructor: org.forgerock.openicf.connectors.scriptedcrest.ScriptedCRESTConfiguration.<init>()v
  at LocalConnectorInfoManagerImpl.createDefaultAPIConfiguration(LocalConnectorInfoManagerImpl.java:339)
Caused by: java.lang.NoClassDefFoundError: org/apache/http/nio/client/HttpAsyncClient

Fix

ReflectionUtil.newInstance rethrows the LinkageError cause of the findConstructor failure, except for the NoSuchMethodError and IllegalAccessError that come with a missing or inaccessible constructor (those are still reported as NoSuchMethodException / IllegalAccessException). This brings back the Class.newInstance() behaviour for every caller, including the OSGi info managers. One case stays different: a link failure that is itself an IllegalAccessError (a dependency that cannot access its superclass) cannot be told from an inaccessible constructor, so it is still reported as IllegalAccessException. Class.newInstance() threw the raw IllegalAccessError. The managers catch neither, so the scan fails in both cases.

Test plan

  • New ReflectionUtilTests.newInstanceThrowsLinkageErrorOfClassWithMissingDependency: a class loader that cannot load one dependency, so the class fails to link. Before the fix the test fails with the same IllegalAccessException: no such constructor
  • New ReflectionUtilTests.newInstanceFailsOnInaccessibleConstructor: a private constructor seen through publicLookup() still fails with IllegalAccessException, not with a raw IllegalAccessError
  • New ReflectionUtilTests.newInstanceFailsOnInaccessibleClass: a package-private class seen through publicLookup(), which findConstructor reports without a cause, still fails with IllegalAccessException, not with a NullPointerException
  • New LocalConnectorInfoManagerTests.testSkipsConnectorWithUnlinkableConfiguration: a bundle with two connectors, the configuration class of one refers to a class missing from the bundle; the scan lists the other connector. Without the fix it fails with the IllegalAccessException: no such constructor from OpenIDM#259; without the catch (NoClassDefFoundError) in LocalConnectorInfoManagerImpl it fails with the NoClassDefFoundError. The test closes the bundle class loader before it deletes the temp jar, which Windows cannot delete while it is open
  • connector-framework: 205/205 pass (incl. newInstanceFailsWithoutNoArgConstructor, which still gets NoSuchMethodException); connector-framework-internal: 525 run, 0 failures, 2 skipped (testConfigurationUpdate of the remote manager tests)
  • End to end: with this connector-framework installed locally, OpenIDM openidm-provisioner-openicf on openicf 2.1.0-SNAPSHOT passes 138/138 (same as on 2.0.4; without the fix beforeTest fails)

@vharseko
vharseko requested a review from maximthomas October 13, 2026 12:29
@vharseko vharseko added bug Something isn't working java Pull requests that update java code framework OpenICF-java-framework tests Test additions or fixes codeql Fixes or follow-ups for CodeQL code scanning alerts labels Oct 13, 2026

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: The fix sits at the one choke point every caller shares, and it keeps the existing contract for missing and inaccessible constructors.

  • ReflectionUtil.newInstance (ReflectionUtil.java:162-172) unwraps the LinkageError cause of the findConstructor failure, so every caller gets the link failure back without a change on its side.
  • The NoSuchMethodError exclusion keeps newInstanceFailsWithoutNoArgConstructor on NoSuchMethodException.
  • newInstanceThrowsLinkageErrorOfClassWithMissingDependency is red at the base (IllegalAccessException: no such constructor ... LinksAgainstHidden.<init>()void) and green at the head.

suggestion (non-blocking): No test reaches the !(cause instanceof IllegalAccessError) exclusion.

OpenICF-java-framework/connector-framework/src/main/java/org/identityconnectors/common/ReflectionUtil.java:168-169

Every ReflectionUtilTests fixture is a public nested class created through the test's own MethodHandles.lookup(), a nestmate that resolves even a private constructor. A mutant that drops && !(cause instanceof IllegalAccessError) makes a private constructor seen through a non-nestmate lookup throw a raw IllegalAccessError instead of IllegalAccessException, and the catch (Exception) callers let that Error escape. All 7 ReflectionUtilTests stay green against it on JDK 11.0.25, 17.0.11, 21.0.2 and 26. The case below fails against the mutant and passes at the head on the same four JDKs.

@Test(expectedExceptions = IllegalAccessException.class)
public void newInstanceFailsOnInaccessibleConstructor() throws Exception {
    // publicLookup() is no nestmate: the private constructor fails the access
    // check with an IllegalAccessError cause
    newInstance(MethodHandles.publicLookup(), PrivateConstructor.class);
}

public static class PrivateConstructor {
    private PrivateConstructor() {
    }
}

Pin: at the head it throws IllegalAccessException (cause IllegalAccessError); against the mutant it throws the raw IllegalAccessError.


suggestion (non-blocking): contains("Hidden") also matches the class under test, so the assertion does not check that the error names the missing dependency.

OpenICF-java-framework/connector-framework/src/test/java/org/identityconnectors/common/ReflectionUtilTests.java:107

The missing dependency is org/identityconnectors/common/ReflectionUtilTests$Hidden, but ReflectionUtilTests$LinksAgainstHidden contains "Hidden" too. A mutant that throws new NoClassDefFoundError(clazz.getName()) instead of rethrowing the cause passes the current assertion on JDK 11, 17, 21 and 26, and the error the info managers log when they skip the connector would no longer name the missing class. The line below fails against that mutant and passes at the head.

assertTrue(e.getMessage().contains(Hidden.class.getName().replace('.', '/')), e.getMessage());

suggestion (non-blocking): The per-connector skip on NoClassDefFoundError, which is the outcome this PR restores, is pinned only at the ReflectionUtil level.

OpenICF-java-framework/connector-framework-internal/src/main/java/org/identityconnectors/framework/impl/api/local/LocalConnectorInfoManagerImpl.java:303, OpenICF-java-framework/connector-framework-osgi/src/main/java/org/forgerock/openicf/framework/impl/api/osgi/internal/OsgiConnectorInfoManagerImpl.java:320, OpenICF-java-framework/connector-framework-server/src/main/java/org/forgerock/openicf/framework/osgi/internal/AsyncOsgiConnectorInfoManagerImpl.java:220

No test reaches these three catch (final NoClassDefFoundError e) blocks: every configuration class in testbundlev1/v2/v10/v11 links. If createDefaultAPIConfiguration's catch (Exception e) were widened to Exception | Error, or one manager's NoClassDefFoundError catch were dropped, the whole-scan failure from OpenIDM#259 would come back with CI still green. Right now the only end-to-end check is a manual OpenIDM run. This mutant was traced by reading, not run.

Pin: a LocalConnectorInfoManagerTests case with a bundle of two connectors, one whose configuration class refers to a class missing from the bundle; assert that the scan throws nothing and the manager lists the other connector. Widening createDefaultAPIConfiguration to catch Exception | Error turns it red.


note (non-blocking): Text that does not match the change.

  • PR description (the squash commit message): "brings back the Class.newInstance() behaviour for all eight callers". The fix is inside ReflectionUtil.newInstance, so it reaches every caller, but git grep 'ReflectionUtil.newInstance(' over src/main at the head finds 15-16 production call sites in 13-14 files. Saying "every caller" would be accurate.

@vharseko
vharseko force-pushed the fix-connector-config-instantiation branch from 83807ed to 20b77d4 Compare October 14, 2026 07:29
@vharseko

Copy link
Copy Markdown
Member Author

@maximthomas Added all three tests in 20b77d4 and fixed the description ("every caller"); the branch is rebased onto the current master.

  • newInstanceFailsOnInaccessibleConstructor (publicLookup() and a private constructor) fails against the mutant without the IllegalAccessError exclusion.
  • The missing-dependency test now checks for Hidden.class.getName().replace('.', '/') and fails against new NoClassDefFoundError(clazz.getName()).
  • LocalConnectorInfoManagerTests.testSkipsConnectorWithUnlinkableConfiguration builds a bundle with two connectors; the configuration class of one refers to a class that is neither in the bundle nor visible through the parent class loader. The scan lists the other connector. It fails if the fix in ReflectionUtil is reverted (the IllegalAccessException: no such constructor from OpenIDM#259) and if the catch (NoClassDefFoundError) in LocalConnectorInfoManagerImpl is dropped.

One note on the third point: widening createDefaultAPIConfiguration to catch (Exception | Error e) turns out to be an equivalent mutant, because ConnectorException.wrap rethrows any Error unchanged (ConnectorException.java:94-96), so the new test stays green against it by design. The OSGi managers are still not covered by this test.

@vharseko
vharseko requested a review from maximthomas October 14, 2026 07:31

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: the fix sits in the one place every caller goes through, and the new tests pin it end to end.

  • ReflectionUtil.newInstance (:167-171) rethrows the LinkageError cause, and the NoSuchMethodError / IllegalAccessError exclusions keep NoSuchMethodException and IllegalAccessException for a missing or inaccessible constructor.
  • LocalConnectorInfoManagerTests.testSkipsConnectorWithUnlinkableConfiguration fails with BASE's ReflectionUtil (IllegalAccessException: no such constructor ...UnlinkableConfiguration) and fails with the NoClassDefFoundError when the catch (NoClassDefFoundError) at LocalConnectorInfoManagerImpl.java:303 is removed.
  • ReflectionUtilTests.newInstanceFailsOnInaccessibleConstructor pins the IllegalAccessError exclusion through publicLookup().

issue (non-blocking): the new test's temp bundle jar is not deleted on Windows, because the bundle class loader still holds it open.

OpenICF-java-framework/connector-framework-internal/src/test/java/org/identityconnectors/framework/impl/api/LocalConnectorInfoManagerTests.java:239-245

The manager is built straight from the file: URL, so the jar itself goes on the class path of a child-first BundleClassLoader (LocalConnectorInfoManagerImpl.java:248). That loader opens the jar to define LinkableConnector, and nothing ever closes it. This manager is not in the factory cache either, so clearLocalCache never reaches it. On the windows-latest legs bundle.delete() therefore fails, its result is ignored, and every run leaves an unlinkable-bundle*.jar in java.io.tmpdir. The zip-slip test at :151 has no such leak: its bundle is refused before any loader is built. I did not run this on Windows.

        List<ConnectorInfo> infos = null;
        try {
            infos = new LocalConnectorInfoManagerImpl(Collections.singletonList(bundle.toURI()
                    .toURL()), parent).getConnectorInfos();
            assertEquals(infos.size(), 1, String.valueOf(infos));
            assertEquals(infos.get(0).getConnectorKey().getConnectorName(),
                    LinkableConnector.class.getName());
        } finally {
            // the bundle class loader keeps the jar open, and Windows cannot
            // delete an open file
            if (infos != null && !infos.isEmpty()) {
                ClassLoader loader = ((LocalConnectorInfoImpl) infos.get(0)).getConnectorClass()
                        .getClassLoader();
                if (loader instanceof Closeable) {
                    ((Closeable) loader).close();
                }
            }
            bundle.delete();
        }

This needs two imports: java.io.Closeable and org.identityconnectors.framework.impl.api.local.LocalConnectorInfoImpl.


suggestion (non-blocking): no test case has a findConstructor failure with a null cause, so the cause instanceof LinkageError check is not pinned.

OpenICF-java-framework/connector-framework/src/main/java/org/identityconnectors/common/ReflectionUtil.java:168-170

findConstructor gives an IllegalAccessException with no cause for a class the lookup cannot see ("symbolic reference class is not accessible"). It gives a NoSuchMethodException with no cause for an array class. Drop cause instanceof LinkageError && and both inputs become throw (LinkageError) null, a NullPointerException instead of the exception the @throws contract promises. Yet that mutant passes all 8 ReflectionUtilTests cases (measured on JDK 11, 17, 21 and 26), because each of their failure inputs has a non-null cause.

    static class PackagePrivate {
    }

    @Test(expectedExceptions = IllegalAccessException.class)
    public void newInstanceFailsOnInaccessibleClass() throws Exception {
        // findConstructor reports a class the lookup cannot see without a cause
        newInstance(MethodHandles.publicLookup(), PackagePrivate.class);
    }

Pin: with the mutant, this case fails with a NullPointerException.


suggestion (non-blocking): the rethrown LinkageError no longer names the class that failed to link.

OpenICF-java-framework/connector-framework/src/main/java/org/identityconnectors/common/ReflectionUtil.java:170

On the PR's own fixture, the error that escapes is NoClassDefFoundError "...$MissingDependency", caused by a ClassNotFoundException. UnlinkableConfiguration appears nowhere in that chain. The dropped IllegalAccessException "no such constructor: <class>.<init>()void" was the only exception that named it. Some callers have no context of their own: ConnectorPoolManager:181 wraps the error and ConnectorDocBuilder:207 logs it. Those callers now report only the missing dependency. This matches what Class.newInstance() did in 2.0.4. A suppressed exception would keep the class name without changing the type or message that the tests and the info managers rely on.

            if (cause instanceof LinkageError && !(cause instanceof NoSuchMethodError)
                    && !(cause instanceof IllegalAccessError)) {
                // keep the name of the class that failed to link; e has cause as
                // its own cause, so record only its message
                cause.addSuppressed(new IllegalAccessException(e.getMessage()));
                throw (LinkageError) cause;
            }

nitpick (non-blocking): the Javadoc and the catch comment say that any other LinkageError means the class failed to link, but a class whose link fails with IllegalAccessError still gets "no such constructor".

OpenICF-java-framework/connector-framework/src/main/java/org/identityconnectors/common/ReflectionUtil.java:139-143, :163-166

Verifying a class can itself fail with IllegalAccessError, for example when a dependency cannot access its superclass. The exclusion at :169 then keeps that failure as IllegalAccessException: no such constructor. A probe on JDK 11, 17, 21 and 26 showed Class.newInstance() throwing the raw IllegalAccessError, while BASE and HEAD both wrap it. The scan fails the same way in all three cases, so the code needs no change, only the text.

     * A class that cannot be linked, typically because a class it refers to is
     * missing from the connector bundle, fails with that {@link LinkageError}
     * (e.g. {@link NoClassDefFoundError}), as it did with
     * {@link Class#newInstance()}: the connector info managers skip such a
     * connector instead of failing the whole bundle. A link failure that is an
     * {@link IllegalAccessError} cannot be told from an inaccessible
     * constructor and is reported as {@link IllegalAccessException}.

nitpick (non-blocking): the comment on the hiding parent loader gives only one of the two reasons why MissingDependency is unavailable.

OpenICF-java-framework/connector-framework-internal/src/test/java/org/identityconnectors/framework/impl/api/LocalConnectorInfoManagerTests.java:225-226, :227-236

MissingDependency is in org.identityconnectors.framework.impl.api. BundleClassLoader.checkAccessAllowed already refuses every parent-loaded class in that package ("Connector may not reference class ..., it is an internal framework class"). So the test stays green with the hiding loader removed; only the ClassNotFoundException message changes. The loader would start to matter if the fixtures moved to an allowed package.

        // and hidden from the parent class loader, which has it on the test
        // class path, as a dependency missing from the class path would be
        // (BundleClassLoader refuses it anyway, as an internal framework class)

…sses

OpenIdentityPlatform#134 replaced Class.newInstance() with ReflectionUtil.newInstance(), which
goes through MethodHandles.Lookup.findConstructor. When a connector
configuration class cannot be linked because one of its dependencies is
missing (e.g. ScriptedCRESTConfiguration without httpasyncclient),
findConstructor reports the NoClassDefFoundError as an
IllegalAccessException "no such constructor". createDefaultAPIConfiguration
wraps that Exception in a ConnectorException, which the connector info
managers do not catch, so one such connector now fails the whole bundle
scan instead of being skipped with an error as before. OpenIDM's
ConnectorInfoProviderServiceTest fails with "LocalManager initialisation
failed" on 2.1.0-SNAPSHOT.

newInstance now rethrows a LinkageError cause other than the
NoSuchMethodError / IllegalAccessError that accompany a missing or
inaccessible constructor, restoring the Class.newInstance() behaviour for
all callers.
- ReflectionUtilTests: a private constructor seen through publicLookup()
  still fails with IllegalAccessException; the missing-dependency test
  checks that the error names the missing class, not the class that
  refers to it.
- LocalConnectorInfoManagerTests: a bundle with a connector whose
  configuration class cannot be linked lists its other connector
  instead of failing the scan.
- LocalConnectorInfoManagerTests: close the bundle class loader before
  deleting the temp jar, which Windows cannot delete while it is open;
  note that BundleClassLoader refuses MissingDependency anyway.
- ReflectionUtilTests: a class the lookup cannot see fails
  findConstructor without a cause and must stay an IllegalAccessException.
- ReflectionUtil: say that a link failure with an IllegalAccessError is
  reported as IllegalAccessException, and that the info managers skip a
  connector on NoClassDefFoundError.
@vharseko
vharseko force-pushed the fix-connector-config-instantiation branch from 20b77d4 to bfb989f Compare October 16, 2026 10:35
@vharseko

Copy link
Copy Markdown
Member Author

@maximthomas Round 2 is in bfb989f; the branch is rebased onto the current master.

issue: temp bundle jar on Windows. Applied as you suggested: the finally closes the bundle class loader (reached through LocalConnectorInfoImpl.getConnectorClass()) before bundle.delete(). The bundle has no message catalogs, so the loader's JarFile is the only open handle. I could not run it on Windows either; on macOS the test passes.

suggestion: null cause. Added newInstanceFailsOnInaccessibleClass (publicLookup() and a package-private class). On JDK 11, 17, 21 and 27, findConstructor reports it as IllegalAccessException: symbolic reference class is not accessible, with getCause() == null. With cause instanceof LinkageError && dropped, the test fails with NullPointerException: Cannot throw exception because "cause" is null (JDK 21).

nitpick: IllegalAccessError link failure. Reproduced on JDK 11, 17, 21 and 27 with a class whose dependency cannot access its superclass. Class.newInstance() throws the raw IllegalAccessError, and findConstructor throws IllegalAccessException: no such constructor with that cause. I took your Javadoc text and added the same point to the catch comment. I also narrowed "the connector info managers skip such a connector" to the NoClassDefFoundError case: the managers catch only NoClassDefFoundError and TypeNotPresentException around createDefaultAPIConfiguration.

nitpick: the hiding parent loader. Took your comment. The loader stays, so the test does not depend on the package restriction in BundleClassLoader.

suggestion: class name in the rethrown LinkageError. I would rather leave this one as it is:

  • The PR restores what Class.newInstance() did in 2.0.4, and you point out that this matches. The class name was not in that error either.
  • The suppressed exception would carry no such constructor: ...<init>()void. That is the text that misled the OpenIDM#259 diagnosis. In a stack trace it reads like a second failure, a missing constructor, not like context.
  • On the path this PR is about, the managers already name the connector: Unable to load configuration class of connector class ...LocalConnectorInfoManagerTests$UnlinkableConnector from bundle file:/.../unlinkable-bundle….jar ... java.lang.NoClassDefFoundError: ...$MissingDependency (from the run of the new test).
  • ConnectorPoolManager:181 is catch (Exception e), so the LinkageError passes it unwrapped, as in 2.0.4. ConnectorDocBuilder:207 logs the throwable at debug level. If those two callers need the class in their message, it belongs there: they know the connector key, and the shared utility does not.

@vharseko
vharseko requested a review from maximthomas October 16, 2026 10:36

@maximthomas maximthomas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

praise: the round closes the test-side gaps from round 2 without touching the fix.

  • testSkipsConnectorWithUnlinkableConfiguration now closes the connector's class loader (LocalConnectorInfoImpl.getConnectorClass().getClassLoader(), a URLClassLoader) before bundle.delete(), so the scan no longer holds the temp jar open.
  • ReflectionUtilTests.newInstanceFailsOnInaccessibleClass pins the cause-less findConstructor failure (publicLookup() on package-private PackagePrivate) as an IllegalAccessException.
  • The unlinkable-configuration case pins the fix on JDK 25.0.2 as well: with ReflectionUtil.java:173-177 deleted it fails with IllegalAccessException: no such constructor: ...LocalConnectorInfoManagerTests$UnlinkableConfiguration.<init>()void (1/1 red).

@vharseko
vharseko merged commit 889f242 into OpenIdentityPlatform:master Oct 19, 2026
15 checks passed
@vharseko
vharseko deleted the fix-connector-config-instantiation branch October 19, 2026 16:40
maximthomas added a commit that referenced this pull request Oct 20, 2026
…et evicted scripts unload (#196)

Fixes #165.

`runScriptOnConnector` no longer compiles a new class into the script
engine's loader on every call. Each distinct script text is compiled
once per engine and kept in a bounded LRU cache. Classes that leave the
cache, and every cached class on `release()`, are unregistered so that
they can be unloaded. With the default size an engine keeps at most 64
runScriptOnConnector scripts, each with its closures and declared
classes, however many calls it serves.

- **One class per text, in its own loader.**
`ScriptedConfiguration.getScriptOnConnectorClass` compiles a text with a
`ScriptOnConnectorLoader`, a child of the engine's loader, which keeps
the new classes out of the engine's class cache. Before, every call went
through
`getGroovyScriptEngine().getGroovyClassLoader().parseClass(...)`. Each
class then stayed in the engine loader, and in `GroovyScriptEngine`'s
script cache, until `release()`.
- **Script-root classes are the engine's.** A class whose source is in
the script roots, such as `Helper`, resolves through
`GroovyScriptEngine.loadScriptByName`, as it does in the engine's own
compilations. A text therefore uses the `Helper` that the operation
scripts use. The engine compiles it when it is first needed or after its
source changes, and a script-root class still shadows a class of the
connector's class loader. Other classes resolve through the engine's
loader.
- **A cached text runs against the classes it was compiled against.**
Its code links a class by name the first time each reference runs.
`ScriptOnConnectorLoader` keeps every class the compiler resolved for
the text, and the loaders of the script-root classes among them. Its
`loadClass` returns a resolved class, or else a class that one of those
loaders defined: a nested class, a closure or another class compiled
from the same source. Without that, code that first ran after an
operation script had recompiled an edited `Helper` linked the new
classes. `@CompileStatic` code then failed with `NoSuchMethodError`
where it was compiled against an old signature, and with a
`LinkageError` (loader constraint violation) where it reached a class
such as `Helper.Greeting` only through the return type of a `Helper`
method.
- **The engine's configuration and compiler lock.** A text compiles with
the engine's compiler configuration, the one
`getRoots(CompilerConfiguration, GroovyClassLoader)` receives, so with
the default imports and with whatever a subclass adds there, a
`SecureASTCustomizer` for example. It compiles under the lock that the
engine's loader holds while it compiles an operation script or a
script-root class, as texts compiled on master: the monitor of the
loader's `GroovyClassLoader.sourceCache`. The engine therefore cannot
recompile a script-root class while a text resolves the classes it uses,
and the customizers of the configuration are not used by two
compilations at once. The compiler resolves the class nodes of an
`ImportCustomizer` in place. `sourceCache` is protected and the loader
is a private class of `GroovyScriptEngine`, so the field is read by
reflection; Groovy 2.2.1, 2.4.21, 4.0.22 and 5.0.3 have it and lock it
in `GroovyScriptEngine`'s loader.
`getImportCustomizer(ImportCustomizer)` and `getImports()` are called
once per engine, as on master.
- **One compilation at a time, no lock for a cached text.** The
configuration monitor is held only to read the engine and the cache, and
the cache's monitor only around map operations. Texts compile one at a
time, and not while the engine compiles, as on master. A cached text
takes no lock and does not wait for a compilation. If two threads
compile the same new text, the first one cached wins and the other's
classes are unregistered.
- **What a text adds or compiles through its loader goes to the
engine.** A text's class loader delegates `addURL`, `addClasspath` and
`parseClass` to the loader that compiled the text, now its
`ScriptOnConnectorLoader`. That loader passes them on to the engine's
loader, as on master. The operation scripts and the other texts load
classes from a URL that a text adds, and see a class that a text
compiles at run time. That compilation takes the engine's compiler lock,
and the class resolves `Helper` as the operation scripts do. The text
itself compiles through a separate method, into its own loader.
- **Bounded, least recently used first.** An engine keeps 64 scripts by
default. The system property
`org.identityconnectors.framework.groovy.scriptOnConnectorCacheSize`
changes the size. It is read when an engine is created. A value that is
not a positive integer logs a WARN and falls back to 64.
- **Evicted and released classes can unload.** Eviction and `release()`
call `InvokerHelper.removeClass` on every class of the script's child
loader, closures and declared classes included, outside the cache
monitor. The cache is created with the engine once its customizer has
succeeded. `release()` clears it last in its `finally`.
- **Run from Java.** `ScriptedConnectorBase.runScriptOnConnector` now
calls `evaluateScriptOnConnector`, which runs
`InvokerHelper.createScript(...).run()` in Java. A Groovy call site
caches the metaclass of the last script it ran, and so keeps that class
alive. The arguments, the `log` binding and the `catch` blocks are
unchanged.
- **Without a cache, as on master.** While the customizer runs, the
engine is not published and there is no cache. A subclass that overrides
`getGroovyScriptEngine()` without calling super has no cache either. A
text then compiles through the engine's loader, with the engine's
configuration, as on master, and is not cached. The configuration of an
engine that a subclass builds, a `SecureASTCustomizer` for example,
therefore still applies to its texts. If `GroovyClassLoader.sourceCache`
cannot be read, there is no cache either, and a WARN is logged when
`ScriptedConfiguration` is loaded.
- **A broken script-root class still fails as a compilation error.** The
engine compiles it, and its `CompilationFailedException` reaches the
caller, as on master. A `ScriptException` from `loadScriptByName`, which
a `GroovyScriptEngine` subclass may throw, also fails the text's
compilation with a `CompilationFailedException`, not a `GroovyBugError`.

Behaviour changes:
- a script's class name is stable while its text stays cached;
- static fields, metaclass changes and classes declared in a script
survive between calls with the same text;
- classes declared in one text are no longer visible to other texts or
to the operation scripts, and a text can no longer replace a script-root
class for everyone else;
- a text's class loader answers `getURLs`, `findResource`,
`findResources`, `getResourceLoader`, `setResourceLoader`,
`getLoadedClasses` and `clearCache` for its `ScriptOnConnectorLoader`,
not for the engine's loader. `getResource` and `getResourceAsStream`
give the same answers as on master;
- a cached script runs against the classes it was compiled against,
script-root classes such as `Helper` and the classes compiled with them
included, until it is evicted or the configuration is released. This
holds even after their source changes on disk and the operation scripts
move to the new classes. Master recompiled the text on every call and so
picked up the change; a newly compiled text still does;
- a class evicted while another thread runs it loses any metaclass
changes it made. Its metaclass is rebuilt on its next dynamic call,
after which the class is only softly held;
- a compilation that finishes after `release()` goes into the orphaned
cache, so its class is only softly held.

Not changed:
- operation scripts (`loadScript`, `evaluate`) are still compiled and
cached by the engine;
- the customizer lifecycle and the exceptions `runScriptOnConnector`
maps.

## Test strength

All in `ScriptOnConnectorCacheTest`. The ed25f73, ce3a86f, 37b50d1
and 21f99b4 cells are runs of the new tests against the production code
of those earlier heads of this PR.

| arm | case | evidence |
|---|---|---|
| one class per text | `#testSameTextReusesTheCompiledClass` |
compile-red at c328553 |
| classes stay out of the engine loader |
`#testScriptClassStaysOutOfTheEngineLoader` | compile-red at c328553;
mutants "child loader's parent is `getParentLoader()`, not the engine's
loader" and "the text compiles through the `parseClass` it passes on to
the engine" killed |
| the text's configuration has the default imports |
`#testScriptSeesTheDefaultImports` | mutant "text configuration without
the import customizer" killed |
| script-root classes resolve |
`#testScriptSeesTheClassesInTheScriptRoots` | kept behaviour |
| a text shares script-root classes with the operation scripts |
`#testScriptSharesScriptRootClassesWithOperationScripts` | red at
ed25f73; mutants "no engine resolver" and "default resolver first"
killed |
| … also after a source edit, the text compiled first |
`#testNewScriptSharesTheEditedScriptRootClassWithOperationScripts` | red
at ed25f73; both resolver mutants killed |
| script roots shadow the connector's classes |
`#testScriptRootClassShadowsAClassOfTheConnectorLoader` | red at
ed25f73; both resolver mutants killed |
| a cached text keeps the class it was compiled against |
`#testCachedScriptKeepsTheScriptRootClassItWasCompiledAgainst` | kept
behaviour, green at ed25f73 |
| … also where its code first runs after the engine recompiled the
class, nested classes included |
`#testCachedScriptLinksTheScriptRootClassesItWasCompiledAgainst` | red
at ce3a86f (`hi hi`); mutant "no `loadClass` override" killed |
| … a class of a script-root source that the text names without that
source's own class |
`#testCachedScriptLinksTheEngineClassesItWasCompiledAgainst` | kept
behaviour, green at 37b50d1; mutants "no lookup of the resolved
classes" and "keep only the script-root classes" killed |
| … statically compiled code, with classes reached only through the
methods of a script-root class |
`#testCachedStaticScriptLinksTheClassesCompiledWithAScriptRootClass` |
red at 37b50d1 (`LinkageError`, loader constraint violation); mutants
"no lookup in the script-root loaders" and "nested classes only" killed
|
| texts compiled together share a script-root class |
`#testScriptsCompiledTogetherShareTheScriptRootClasses` | red at
ed25f73; mutant "no engine compiler lock" killed (five rounds of four
threads: a race, so a likely kill, not a certain one) |
| a text and an operation script that load a script-root class together
share it |
`#testTextAndOperationScriptThatLoadAScriptRootClassTogetherShareIt` |
red at 37b50d1 (two copies of `Helper`); mutant "no engine compiler
lock" killed |
| texts compile together on a new engine |
`#testScriptsCompileTogetherOnANewEngine` | red at ed25f73 (`unable to
resolve class
org$forgerock$openicf$misc$scriptedcommon$ICFObjectBuilder`); mutant "no
engine compiler lock" killed, with the engine's configuration shared |
| what a subclass adds to the engine's configuration in `getRoots`
applies to texts |
`#testScriptCompilesWithWhatASubclassAddsToTheEngineConfiguration` | red
at 37b50d1 (`unable to resolve class RootsList`); mutant "a new
configuration built as the engine's" killed |
| binding reaches the script | `#testScriptSeesItsBinding` | compile-red
at c328553 |
| a URL that a text adds reaches the operation scripts |
`#testUrlThatATextAddsToItsLoaderReachesTheOperationScripts` | red at
37b50d1 (`ClassNotFoundException: drv.Driver`); mutant "no `addURL`
override" killed |
| a class that a text compiles at run time reaches the operation scripts
| `#testClassThatATextParsesReachesTheOperationScripts` | red at
21f99b4 (`MissingPropertyException: No such property: Parsed`); mutant
"the `parseClass` override takes the engine's lock but compiles into the
text's loader" killed |
| … and waits for the engine's compilation |
`#testClassThatATextParsesWaitsForTheEngineCompilation` | red at
21f99b4 (`parseClass` returns while the engine compiles an operation
script) |
| `release()` unregisters and recompiles |
`#testReleaseUnregistersTheCachedClasses` | compile-red at c328553 |
| … also when the release closure throws |
`#testReleaseUnregistersTheCachedClassesWhenTheReleaseClosureThrows` |
mutant "clear the cache after the `try`/`finally`" killed |
| customizer before publish | `#testCustomizerCanRunAScriptOnConnector`
| mutant without the null-cache guard killed (`NullPointerException`) |
| … and its script is not cached |
`#testScriptRunByTheCustomizerIsNotCached` | mutant "cache created
before the customizer runs" killed |
| an engine built by a subclass compiles with its own configuration |
`#testEngineBuiltByASubclassCompilesTheScriptWithItsOwnConfiguration` |
red at ce3a86f (`unable to resolve class EngineList`); mutants "no
cache: compile with the child loader anyway" and "no null-cache guard"
killed |
| a hit does not wait for a compile |
`#testCachedScriptDoesNotWaitForAnotherCompilation` | mutants
`synchronized getScriptOnConnectorClass` and "cache lookup under the
engine's compiler lock" killed (`TimeoutException`) |
| size property: name, default, `5`, ` 7 `, `0`, `-3`, `many` |
`#testCacheSizeComesFromTheSystemProperty` | compile-red at f4b1286 |
| LRU, not insertion order | `#testLeastRecentlyUsedScriptIsEvicted` |
mutant `accessOrder` `false` killed |
| eviction unregisters closures too |
`#testEvictedScriptClassesAreUnregistered` | mutants "unregister only
the main class" and "no unregister in `put`" killed |
| an evicted class unloads | `#testEvictedScriptClassCanBeUnloaded` |
same two mutants killed |
| the connector reuses the class |
`#testConnectorReusesTheCompiledScript` | red at a40030c |
| arguments and `log` binding |
`#testConnectorScriptSeesItsArgumentsAndTheLogger` | kept behaviour,
green at a40030c; mutants "`log` is `Log.getLog(Object.class)`" and
"`evaluateScriptOnConnector` rebinds `log` to the script's class" killed
|
| a connector-run class unloads once evicted |
`#testScriptRunByTheConnectorCanBeUnloadedOnceEvicted` | red at
a40030c; mutant Groovy-side `createScript(...).run()` killed (the call
site keeps the class) |
| broken script-root class → compile error |
`#testBrokenScriptRootClassFailsAsACompilationError` | red at a872c5b
(`GroovyBugError`); both resolver mutants killed |
| … also from the connector |
`#testConnectorReportsABrokenScriptRootClassAsACompilationError` | red
at a872c5b (`GroovyBugError`); both resolver mutants killed |
| a `ScriptException` from the engine → compile error |
`#testScriptExceptionFromTheEngineFailsAsACompilationError` | red at
37b50d1 (`GroovyBugError`); mutants "`ConnectorException.wrap`" and "no
cause" killed |

The two unload tests run `System.gc()` for up to about 5 s and return as
soon as the class is gone. They rely on the JVM's default soft-reference
policy, which surefire leaves unchanged.

Not pinned by a test:
- the first-wins rule when two threads compile the same new text;
- the WARN for an invalid size, and reading the size once per engine;
- unregistering closure classes on `release()` (eviction pins the shared
`unregister()`);
- the fallback without a cache, and its WARN, when
`GroovyClassLoader.sourceCache` cannot be read;
- that `loadClass` takes from the loader of a script-root class only the
classes that loader defined. A mutant that takes any class the loader
returns passes: the two differ only for a class that the script-root
class linked before the engine replaced it.

`mvn -pl OpenICF-groovy-connector,OpenICF-ssh-connector verify` on JDK
26:
- groovy: `Tests run: 179, Failures: 0, Errors: 0, Skipped: 42`. The 42
skipped tests are the five `*SampleTest` classes, which need a private
configuration;
- ssh: `Tests run: 135, Failures: 0, Errors: 0, Skipped: 120`. The
skipped sample tests need a live host.

Master is at 4648e38 now. #188, #191, #194 and #197 touch none of these
files. The branch merges into it cleanly, and the groovy module then
runs `Tests run: 210, Failures: 0, Errors: 0, Skipped: 42`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working codeql Fixes or follow-ups for CodeQL code scanning alerts framework OpenICF-java-framework java Pull requests that update java code tests Test additions or fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants