Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,10 @@ It adds only the Revit execution model on top of TUnit; TUnit provides the asser
* The Revit thread is one process-wide background STA thread running a WPF `Dispatcher`. The dispatcher pumps the Win32 messages COM marshaling needs and routes `await` continuations back to the thread through `DispatcherSynchronizationContext`.
* `RevitThreadExecutor` unwraps the dispatcher operation (`operation.Task.Unwrap()`). The returned task completes only after the body and its `await` continuations finish.
* `RevitThreadExecutor` declares an `IParallelLimit` of `1` through `ITestRegisteredEventReceiver`, and TUnit applies it to every test the executor runs. The limit belongs to the executor; never reintroduce it as an attribute on the base class.
* `RevitUiThreadExecutor` declares `RevitUiParallelLimit`. TUnit groups tests by the type of their limit, and UI tests run one at a time in parallel with the tests of `RevitThreadExecutor`. A UI test outside Revit awaits its result without the Revit thread.
* UI tests in the test host use one shared `NotInParallel` key, never a parallel limit. A test that waits for a limit occupies a parallel slot. TUnit runs the tests of a limit in no class order. A UI test outside Revit awaits its result without the Revit thread.
* A UI test has one `NotInParallel` constraint: TUnit reads only the first `NotInParallel` constraint of a test. The UI key is added to the keys the test declares.
* Inside Revit, the `Before(Class)` hooks of a UI test class run only after the test host executes the first test of the class. The timeout of a `Before(Class)` hook excludes that wait.
* A base class declares its hook executor with the non-generic `HookExecutor` attribute on .NET Framework: TUnit ignores a generic hook executor attribute of a class there.
* The application connection opens in the `RevitSessionSetup` hook, a `Before(TestSession)` hook, and never lazily in the executor. TUnit creates test instances on pool threads, and a class that references a Revit API type loads that type off the Revit thread. An injection running at the same time fails with `Attempted to write protected memory`. TUnit applies no executor of the class to a test session hook, and the hook declares `[HookExecutor<RevitThreadExecutor>]` itself. `RevitSession.EnsureStarted` refuses a call off the Revit thread, and Revit activates only on an STA thread.
* The test host is the only scheduler of UI tests. Revit runs a UI test only on the request of the test host, and inside Revit no UI test keeps a parallel limit or a parallel constraint; two schedulers that order the same tests deadlock.
* A UI test waits for Revit before TUnit starts its timer. The duration of a UI test covers its hooks and body inside Revit, never the start of Revit.
Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,22 @@ using Nice3point.Revit.Injector.Attributes;
`Application.ActiveAddInId` in `RevitApiTest` and `UiApplication.ActiveAddInId` in `RevitApiUiTest` return the declared add-in.
Each property is optional: `Id` defaults to a random GUID, `Name` to the ID in uppercase, and `VendorId` to `Autodesk`.

## UI test order

The UI tests of a class run one after another https://github.com/Nice3point/RevitUnit/issues/132:

- `[Before(Class)]` of a `RevitApiUiTest` class runs right before the first test of the class.
- `[After(Class)]` runs right after the last test of the class.
A document that a class opens in `[Before(Class)]` and closes in `[After(Class)]` is open only during the tests of that class.
- The timeout of a `[Before(Class)]` hook excludes the wait for the tests of other classes.
- The duration of the first test of a class includes its `[Before(Class)]` hooks.
- A UI test that waits for its turn occupies no parallel slot.

## Bug fixes

- Fixed `[Before(Class)]` and `[After(Class)]` hooks of `RevitApiTest` and `RevitApiUiTest` classes running outside the Revit thread on Revit 2021 to 2024.
- Fixed the test host crash when `--fail-fast` stops a run with UI tests: the package depends on TUnit 1.76.45.

# 2027.0.7

- `RevitApiTest` waits for a timed-out or cancelled test body to finish before running cleanup hooks, starting another test, or closing the Revit connection.
Expand Down
2 changes: 1 addition & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
<PackageVersion Include="ILRepack" Version="2.0.48"/>

<!-- Testing -->
<PackageVersion Include="TUnit" Version="1.73.19"/>
<PackageVersion Include="TUnit" Version="1.76.45"/>
<PackageVersion Include="Shouldly" Version="4.3.0"/>

<PackageVersion Include="Microsoft.VisualStudio.SolutionPersistence" Version="1.0.52"/>
Expand Down
43 changes: 41 additions & 2 deletions source/Nice3point.TUnit.Revit/Executors/RevitUiThreadExecutor.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
using Nice3point.TUnit.Revit.Limiters;
using Nice3point.TUnit.Revit.Sessions;
using Nice3point.TUnit.Revit.Ui;
using TUnit.Core.Interfaces;
Expand All @@ -15,6 +14,11 @@ namespace Nice3point.TUnit.Revit.Executors;
/// </remarks>
public sealed class RevitUiThreadExecutor : ITestExecutor, IHookExecutor, ITestRegisteredEventReceiver
{
/// <summary>
/// The <c>NotInParallel</c> key of every UI test in the test host.
/// </summary>
private const string UiConstraintKey = "Nice3point.TUnit.Revit.Ui";

/// <inheritdoc />
public int Order => 0;

Expand Down Expand Up @@ -100,12 +104,47 @@ public ValueTask OnTestRegistered(TestRegisteredContext context)
}

RevitUiSession.Get(context.TestContext).Register(context.TestDetails.TestId);
context.SetParallelLimiter(RevitUiParallelLimit.Default);
AddUiConstraint(context);
context.TestDetails.Timeout = Timeout.InfiniteTimeSpan;
context.TestDetails.RetryLimit = 0;
return default;
}

/// <summary>
/// Adds the <see cref="UiConstraintKey" /> to the <see cref="NotInParallelConstraint" /> of the specified test.
/// </summary>
/// <param name="context">The registration context of the test.</param>
/// <remarks>
/// TUnit reads only the first <see cref="NotInParallelConstraint" /> of a test.
/// The method adds the key to the keys of a constraint the test declares.
/// The method does not change a constraint without keys: TUnit runs such a test alone.
/// </remarks>
private static void AddUiConstraint(TestRegisteredContext context)
{
var parallelism = context.TestContext.Parallelism;
var constraints = parallelism.Constraints.ToArray();
var notInParallel = constraints.OfType<NotInParallelConstraint>().FirstOrDefault();
if (notInParallel is null)
{
parallelism.AddConstraint(new NotInParallelConstraint([UiConstraintKey]));
return;
}

var keys = notInParallel.NotInParallelConstraintKeys;
if (keys.Count == 0 || keys.Contains(UiConstraintKey))
{
return;
}

context.ClearParallelConstraints();
foreach (var constraint in constraints)
{
parallelism.AddConstraint(ReferenceEquals(constraint, notInParallel)
? notInParallel with { NotInParallelConstraintKeys = [.. keys, UiConstraintKey] }
: constraint);
}
}

private static ValueTask InvokeAsync(Func<ValueTask> action)
{
return RevitUiRuntime.Application?.InvokeAsync(action) ?? default;
Expand Down
5 changes: 5 additions & 0 deletions source/Nice3point.TUnit.Revit/RevitApiTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,12 @@ namespace Nice3point.TUnit.Revit;
/// Test discovery executes no test and does not start Revit.
/// </remarks>
[TestExecutor<RevitThreadExecutor>]
#if NET
[HookExecutor<RevitThreadExecutor>]
#else
// On .NET Framework, TUnit ignores a generic hook executor attribute of a class.
[HookExecutor(typeof(RevitThreadExecutor))]
#endif
public abstract class RevitApiTest
{
/// <inheritdoc cref="Autodesk.Revit.ApplicationServices.Application" />
Expand Down
5 changes: 5 additions & 0 deletions source/Nice3point.TUnit.Revit/RevitApiUiTest.cs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,12 @@ namespace Nice3point.TUnit.Revit;
/// </remarks>
[Category(RevitUiRuntime.Category)]
[TestExecutor<RevitUiThreadExecutor>]
#if NET
[HookExecutor<RevitUiThreadExecutor>]
#else
// On .NET Framework, TUnit ignores a generic hook executor attribute of a class.
[HookExecutor(typeof(RevitUiThreadExecutor))]
#endif
public abstract class RevitApiUiTest : ITestStartEventReceiver, ITestEndEventReceiver
{
/// <inheritdoc cref="Autodesk.Revit.UI.UIApplication" />
Expand Down
25 changes: 24 additions & 1 deletion source/Nice3point.TUnit.Revit/Sessions/RevitUiSessionHooks.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
namespace Nice3point.TUnit.Revit.Sessions;

/// <summary>
/// Provides the test session hooks of the UI tests.
/// Provides the test session and class hooks of the UI tests.
/// </summary>
/// <remarks>
/// The hooks run without the Revit thread, in the test host process and in the Revit user interface application alike.
Expand All @@ -19,6 +19,29 @@ public static void RevitUiSessionSetup()
RevitUiRuntime.Application?.ReportUnregisteredTests();
}

/// <summary>
/// Waits inside Revit until the test host executes the first test of a UI test class.
/// </summary>
/// <param name="context">The context of the class.</param>
/// <param name="cancellationToken">A <see cref="CancellationToken" /> used to cancel the wait.</param>
/// <returns>A task that represents the asynchronous wait operation.</returns>
/// <remarks>
/// TUnit runs the hook before the <c>[Before(Class)]</c> hooks of the class.
/// The timeout of a <c>[Before(Class)]</c> hook excludes the wait.
/// The hook returns at once in the test host process and for a class not derived from <see cref="RevitApiUiTest" />.
/// </remarks>
[Timeout(int.MaxValue)]
[BeforeEvery(Class, Order = int.MinValue)]
public static Task RevitUiClassSetup(ClassHookContext context, CancellationToken cancellationToken)
{
if (RevitUiRuntime.Application is not { } runtime || !typeof(RevitApiUiTest).IsAssignableFrom(context.ClassType))
{
return Task.CompletedTask;
}

return runtime.WaitForAnyExecutionAsync(context.Tests.Select(static test => test.Metadata.TestDetails.TestId), cancellationToken);
}

/// <summary>
/// Closes the Revit user interface session of the finished test session.
/// </summary>
Expand Down
26 changes: 26 additions & 0 deletions source/Nice3point.TUnit.Revit/Ui/RevitUiApplicationRuntime.cs
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,32 @@ public async Task WaitForExecutionAsync(string testId)
}
}

/// <summary>
/// Waits for the test host to execute any of the specified tests.
/// </summary>
/// <param name="testIds">The identifiers of the tests.</param>
/// <param name="cancellationToken">A <see cref="CancellationToken" /> used to cancel the wait.</param>
/// <returns>A task that represents the asynchronous wait operation.</returns>
/// <exception cref="SkipTestException">The test host skipped the pending tests.</exception>
/// <remarks>
/// The call sends <see cref="RevitUiTestPending" /> for every test that is not pending yet.
/// A call without tests completes at once.
/// </remarks>
public async Task WaitForAnyExecutionAsync(IEnumerable<string> testIds, CancellationToken cancellationToken)
{
var executions = testIds.Select(testId => GetExecution(testId).Task).ToArray();
if (executions.Length == 0)
{
return;
}

var execution = await Task.WhenAny(executions).WaitAsync(cancellationToken).ConfigureAwait(false);
if (!await execution.ConfigureAwait(false))
{
throw new SkipTestException("The test host skipped the pending tests.");
}
}

/// <summary>
/// Runs the specified test body or hook on the Revit thread inside a Revit API context.
/// </summary>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
namespace Nice3point.TUnit.Revit.Tests.Abstractions;

public abstract class ClassLifetimeUiTest : RevitApiUiTest
{
private static readonly Lock StateLock = new();
private static readonly Dictionary<Type, Type?> ClassesOpenAtSetup = [];
private static Type? _openClass;

protected static Type? OpenClass
{
get
{
lock (StateLock)
{
return _openClass;
}
}
}

protected Type? ClassOpenAtSetup
{
get
{
lock (StateLock)
{
return ClassesOpenAtSetup[GetType()];
}
}
}

[Before(Class)]
public static void OpenTestClass(ClassHookContext context)
{
lock (StateLock)
{
ClassesOpenAtSetup[context.ClassType] = _openClass;
_openClass = context.ClassType;
}
}

[After(Class)]
public static void CloseTestClass(ClassHookContext context)
{
lock (StateLock)
{
if (_openClass == context.ClassType)
{
_openClass = null;
}
}
}
}
22 changes: 22 additions & 0 deletions tests/Nice3point.TUnit.Revit.Tests/ClassHookTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
namespace Nice3point.TUnit.Revit.Tests;

public sealed class ClassHookTests : RevitApiTest
{
private static int _beforeClassThreadId;

[Before(Class)]
public static void RecordBeforeClassThread()
{
_beforeClassThreadId = Environment.CurrentManagedThreadId;
}

[Test]
public async Task BeforeClassHook_DerivedClass_RunsOnTheRevitThread()
{
// Arrange & Act
var threadId = Environment.CurrentManagedThreadId;

// Assert
await Assert.That(_beforeClassThreadId).IsEqualTo(threadId);
}
}
22 changes: 22 additions & 0 deletions tests/Nice3point.TUnit.Revit.Tests/ClassHookUiTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
namespace Nice3point.TUnit.Revit.Tests;

public sealed class ClassHookUiTests : RevitApiUiTest
{
private static int _beforeClassThreadId;

[Before(Class)]
public static void RecordBeforeClassThread()
{
_beforeClassThreadId = Environment.CurrentManagedThreadId;
}

[Test]
public async Task BeforeClassHook_DerivedClass_RunsOnTheRevitThread()
{
// Arrange & Act
var threadId = Environment.CurrentManagedThreadId;

// Assert
await Assert.That(_beforeClassThreadId).IsEqualTo(threadId);
}
}
22 changes: 22 additions & 0 deletions tests/Nice3point.TUnit.Revit.Tests/FirstClassLifetimeUiTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
using Nice3point.TUnit.Revit.Tests.Abstractions;

namespace Nice3point.TUnit.Revit.Tests;

public sealed class FirstClassLifetimeUiTests : ClassLifetimeUiTest
{
[Test]
[Repeat(2)]
public async Task ClassHooks_AnotherUiClass_DoNotOverlap()
{
// Arrange & Act
var openClass = OpenClass;
var classOpenAtSetup = ClassOpenAtSetup;

// Assert
using (Assert.Multiple())
{
await Assert.That(classOpenAtSetup).IsNull();
await Assert.That(openClass).IsEqualTo(typeof(FirstClassLifetimeUiTests));
}
}
}
22 changes: 22 additions & 0 deletions tests/Nice3point.TUnit.Revit.Tests/SecondClassLifetimeUiTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
using Nice3point.TUnit.Revit.Tests.Abstractions;

namespace Nice3point.TUnit.Revit.Tests;

public sealed class SecondClassLifetimeUiTests : ClassLifetimeUiTest
{
[Test]
[Repeat(2)]
public async Task ClassHooks_AnotherUiClass_DoNotOverlap()
{
// Arrange & Act
var openClass = OpenClass;
var classOpenAtSetup = ClassOpenAtSetup;

// Assert
using (Assert.Multiple())
{
await Assert.That(classOpenAtSetup).IsNull();
await Assert.That(openClass).IsEqualTo(typeof(SecondClassLifetimeUiTests));
}
}
}