feat: add TypstCompiler.EvictCache to release the compilation cache

README.md

@@ -222,6 +222,37 @@ both its size and its modification time, such as a copy that preserves timestamp
 the same length, is not noticed; a file modified within the last two seconds is always read, so
 an ordinary write is safe even on file systems with coarse timestamps.
 
+### Memory held by the compilation cache
+
+Typst memoises its work in an incremental compilation cache, which is what makes a second compile of
+the same template fast. The cache is shared by every compiler in the process, and after each
+compilation it is trimmed to what one of the last ten compilations used. What it holds grows with
+the size of the documents, so a process that occasionally compiles a document of a few thousand
+pages keeps memory in proportion to it for the next ten compilations. Under a memory limit, a
+handful of large documents in a row can be enough to exceed it.
+
+Clear the cache after an unusually large compilation to release that memory at once:
+
+```csharp
+using (var document = compiler.CompileToDocument())
+{
+    document.CopyOutputTo(destination);
+}
+
+TypstCompiler.EvictCache();
+```
+
+`EvictCache()` clears the cache completely. Because the cache belongs to the process rather than to a
+compiler, that includes the results of every other compiler, in every other component that uses
+typstsharp: the next compilation of each template starts cold. Call it after large documents rather
+than after every compilation. It is safe to call while other threads compile. They lose cached
+results and compute them again, and documents already compiled stay valid.
+
+The packaged Linux library is statically linked against musl, whose allocator reuses freed memory for
+later compilations but rarely hands it back to the operating system, so the resident size stops growing
+rather than shrinking. Clearing the cache does not lower the peak of a single compilation either: to
+bound that, split a large document into several smaller ones and clear the cache between them.
+
 ## Prerequisites
 
 - [.NET SDK 10.0](https://dotnet.microsoft.com/) – required to build the managed projects.

RELEASENOTES.md

@@ -17,6 +17,7 @@
 - `sysInputs` parameters on `TypstCompiler` and the `SetSysInputs` argument are now `IDictionary<string, string>` instead of `Dictionary<string, string>`, so a `ReadOnlyDictionary<string, string>`, `ImmutableDictionary<string, string>` or any other implementation can be passed. Existing calls that pass a `Dictionary<string, string>` are unaffected.
 
 ### Added
+- Added `TypstCompiler.EvictCache()` to clear and release the memory held by Typst's incremental compilation cache. The cache is shared by the whole process and was only ever trimmed to what the last ten compilations used, so an application that compiled one large document kept memory in proportion to it for the next ten compilations, and could not release it any sooner. The cache is process-wide, so clearing it makes the next compilation of every compiler start cold; it is safe while other threads compile, and leaves compiled documents valid. The packaged Linux library uses musl, which reuses the freed memory rather than returning it to the operating system, and a single compilation's peak is unchanged, so a very large document still has to be split to bound that.
  - Added merged SVG export, rendering every page into a single SVG stacked top to bottom as Typst's `svg_merged` does. Use `compiler.CompileMergedSvg(gapInPoints)`, the static `TypstCompiler.CompileMergedSvg(...)` / `TypstCompiler.CompileMergedSvgFromFile(...)`, or the `svg-merged` format with the new `mergedGap` parameter on `CompileToDocument` and `Compile(outputFile, format)`. The gap between pages is given in points.
  - Added `compiler.CompileToDocument(...)`, returning a disposable `TypstDocument` that exposes the rendered output while it is still in the memory the native library allocated. `GetOutputSpan`, `OpenOutputStream`, `CopyOutputTo`, `WriteOutputToFile` and `RentOutput` read it without putting a multi-megabyte PDF on the large object heap; `GetOutputBytes` copies when a `byte[]` is what you need.
  - Added easier PDF compilation APIs on `TypstCompiler`:

src/typst_core/src/lib.rs

@@ -10,7 +10,7 @@
 //!   independent heap allocation, and none of them borrows from the compiler that produced them.
 //! - Those allocations stay valid until [`free_compile_result`] is called on the result that owns
 //!   them, whatever else happens in between: further [`compile`] calls, [`set_sys_inputs`],
-//!   [`free_compiler`] on the originating compiler, or [`reset_world`].
+//!   [`free_compiler`] on the originating compiler, [`reset_world`] or [`evict_cache`].
 //! - [`free_compile_result`] may be called from any thread, and must be called exactly once per
 //!   result. Calling it twice frees the same allocations twice.
 //!
@@ -439,9 +439,9 @@ fn compile_internal(
 /// is the distance in points between neighbouring pages, which must be finite and non-negative.
 ///
 /// The returned [`CompileResult`] owns its buffers and messages. They do not borrow from
-/// `compiler`, so they outlive further compilations, [`set_sys_inputs`], [`reset_world`] and even
-/// [`free_compiler`] on the compiler that produced them. The caller must pass the result to
-/// [`free_compile_result`] exactly once.
+/// `compiler`, so they outlive further compilations, [`set_sys_inputs`], [`reset_world`],
+/// [`evict_cache`] and even [`free_compiler`] on the compiler that produced them. The caller must
+/// pass the result to [`free_compile_result`] exactly once.
 ///
 /// A panic inside Typst is caught and reported as an error result rather than unwinding across the
 /// ABI boundary.
@@ -513,11 +513,37 @@ pub unsafe extern "C" fn free_compile_result(result: CompileResult) {
     }
 }
 
-/// Trims the process-global incremental compilation cache. It holds no references to any
-/// [`CompileResult`], so trimming it never invalidates output the caller is still holding.
+/// How many evictions a cached result survives without being used, when the cache is trimmed after
+/// a compilation. With one eviction per compilation, results stay cached for ten compilations.
+const DEFAULT_MAX_CACHE_AGE: usize = 10;
+
+/// Trims the process-global incremental compilation cache, dropping every result that has not been
+/// used in the last ten trims. The managed side calls this after each compilation. The cache holds
+/// no references to any [`CompileResult`], so trimming it never invalidates output the caller is
+/// still holding.
 #[unsafe(no_mangle)]
 pub extern "C" fn reset_world() {
-    comemo::evict(10);
+    comemo::evict(DEFAULT_MAX_CACHE_AGE);
+}
+
+/// Clears the process-global incremental compilation cache completely.
+///
+/// The cache is what makes recompiling a template fast, and what it holds grows with the size of
+/// the documents compiled. Because [`reset_world`] keeps results for ten compilations, memory taken
+/// by an unusually large document stays allocated until ten more compilations have passed. A
+/// document of a few thousand pages can hold on to gigabytes that way. Calling this right after such
+/// a compilation releases it at once.
+///
+/// The cache is shared by every compiler in the process, so the next compilation of every compiler
+/// starts cold. Evicting is safe while other threads are compiling: they lose cached results and
+/// recompute them, but their output is unaffected. The cache holds no references to any
+/// [`CompileResult`], so results the caller is still holding stay valid.
+///
+/// Returns `false` if the eviction panicked. The panic is caught rather than unwinding across the
+/// ABI boundary.
+#[unsafe(no_mangle)]
+pub extern "C" fn evict_cache() -> bool {
+    std::panic::catch_unwind(|| comemo::evict(0)).is_ok()
 }
 
 #[cfg(test)]

src/typst_core/tests/cache_eviction.rs

@@ -0,0 +1,120 @@
+//! Proves that clearing the compilation cache releases memory, rather than only that output stays
+//! correct afterwards. A cache eviction that did nothing would pass every other test.
+//!
+//! This file is its own test binary so that it can count every heap allocation in the process with
+//! a global allocator. It holds a single test on purpose: the cache and the allocator are both
+//! process-wide, so a second test running in parallel would skew the counts.
+
+use std::alloc::{GlobalAlloc, Layout, System};
+use std::ffi::{c_char, CString};
+use std::sync::atomic::{AtomicIsize, Ordering};
+
+use typst_core::{
+    compile, create_compiler, evict_cache, free_compile_result, free_compiler, set_sys_inputs,
+};
+
+/// Forwards to the system allocator and keeps a running total of the bytes currently allocated.
+struct CountingAllocator;
+
+static LIVE_BYTES: AtomicIsize = AtomicIsize::new(0);
+
+unsafe impl GlobalAlloc for CountingAllocator {
+    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
+        let ptr = unsafe { System.alloc(layout) };
+        if !ptr.is_null() {
+            LIVE_BYTES.fetch_add(layout.size() as isize, Ordering::SeqCst);
+        }
+        ptr
+    }
+
+    unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
+        let ptr = unsafe { System.alloc_zeroed(layout) };
+        if !ptr.is_null() {
+            LIVE_BYTES.fetch_add(layout.size() as isize, Ordering::SeqCst);
+        }
+        ptr
+    }
+
+    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
+        unsafe { System.dealloc(ptr, layout) };
+        LIVE_BYTES.fetch_sub(layout.size() as isize, Ordering::SeqCst);
+    }
+
+    unsafe fn realloc(&self, ptr: *mut u8, layout: Layout, new_size: usize) -> *mut u8 {
+        let new_ptr = unsafe { System.realloc(ptr, layout, new_size) };
+        if !new_ptr.is_null() {
+            LIVE_BYTES.fetch_add(new_size as isize - layout.size() as isize, Ordering::SeqCst);
+        }
+        new_ptr
+    }
+}
+
+#[global_allocator]
+static ALLOCATOR: CountingAllocator = CountingAllocator;
+
+fn live_bytes() -> isize {
+    LIVE_BYTES.load(Ordering::SeqCst)
+}
+
+#[test]
+fn clearing_the_cache_releases_the_memory_a_compilation_cached() {
+    // Enough distinct content that the memoised layout dwarfs anything else a compilation keeps.
+    // Every heading carries `sys.inputs.run`, so a compilation with a new value cannot reuse what
+    // an earlier one cached, whether or not that was evicted.
+    let source = "#set document(date: none)\n#let run = sys.inputs.at(\"run\")\n".to_owned()
+        + &(0..100)
+            .map(|page| format!("= Page {page} of run #run\n\n#lorem(150)\n\n#pagebreak()\n"))
+            .collect::<String>();
+
+    let root = CString::new(".").unwrap();
+    let sys_inputs = CString::new(r#"{"run": "warm-up"}"#).unwrap();
+    let compiler = unsafe {
+        create_compiler(
+            root.as_ptr(),
+            std::ptr::null(),
+            source.as_ptr(),
+            source.len(),
+            std::ptr::null::<*const c_char>(),
+            0,
+            std::ptr::null(),
+            sys_inputs.as_ptr(),
+            true,
+            true,
+        )
+    };
+    assert!(!compiler.is_null(), "failed to create compiler");
+
+    // A first compilation loads the fonts and parses the source into the compiler's own world.
+    // Those stay with the compiler, so clearing the cache after it gives the baseline that a
+    // compilation's cached results are measured against.
+    let warm_up = unsafe { compile(compiler, std::ptr::null(), 96.0, 0.0, std::ptr::null()) };
+    assert!(warm_up.error_ptr.is_null(), "valid document failed to compile");
+    unsafe { free_compile_result(warm_up) };
+    assert!(evict_cache(), "clearing the cache failed");
+    let measured_inputs = CString::new(r#"{"run": "measured"}"#).unwrap();
+    assert!(
+        unsafe { set_sys_inputs(compiler, measured_inputs.as_ptr()) },
+        "failed to set the inputs"
+    );
+    let before = live_bytes();
+
+    let result = unsafe { compile(compiler, std::ptr::null(), 96.0, 0.0, std::ptr::null()) };
+    assert!(result.error_ptr.is_null(), "valid document failed to compile");
+    unsafe { free_compile_result(result) };
+    let cached = live_bytes() - before;
+
+    assert!(evict_cache(), "clearing the cache failed");
+    let remaining = live_bytes() - before;
+
+    eprintln!("cached after compiling: {cached} bytes, left after clearing: {remaining} bytes");
+    assert!(
+        cached > 1024 * 1024,
+        "compiling 100 pages cached only {cached} bytes, so this test proves nothing"
+    );
+    assert!(
+        remaining < cached / 10,
+        "clearing the cache left {remaining} of {cached} cached bytes allocated"
+    );
+
+    unsafe { free_compiler(compiler) };
+}

src/typst_core/tests/compile.rs

@@ -1,6 +1,10 @@
 use std::ffi::{c_char, CString};
+use std::sync::atomic::{AtomicUsize, Ordering};
+use std::sync::{Arc, Barrier};
 
-use typst_core::{Compiler, compile, create_compiler, free_compile_result, free_compiler};
+use typst_core::{
+    Compiler, compile, create_compiler, evict_cache, free_compile_result, free_compiler,
+};
 
 /// Creates a compiler over an in-memory document. The source is handed over as
 /// raw bytes, so a test can feed content that a C string could not carry.
@@ -91,3 +95,106 @@ fn source_after_a_nul_byte_is_still_compiled() {
         "source was truncated at the NUL byte: {error}"
     );
 }
+
+/// Compiles the document to PDF and returns the bytes, failing the test on an error.
+fn compile_to_pdf(compiler: *mut Compiler) -> Vec<u8> {
+    let result = unsafe { compile(compiler, std::ptr::null(), 96.0, 0.0, std::ptr::null()) };
+    assert!(result.error_ptr.is_null(), "valid document failed to compile");
+    assert_eq!(result.buffers_len, 1, "a PDF is exported as one buffer");
+
+    let pdf = unsafe {
+        let buffer = &*result.buffers;
+        std::slice::from_raw_parts(buffer.ptr, buffer.len).to_vec()
+    };
+
+    unsafe { free_compile_result(result) };
+    pdf
+}
+
+/// A document whose PDF depends on nothing but its source, so two compilations can be compared
+/// byte for byte. Without a fixed date, Typst stamps the current one.
+fn reproducible(body: &str) -> Vec<u8> {
+    format!("#set document(date: none)\n{body}").into_bytes()
+}
+
+#[test]
+fn a_compiler_still_compiles_after_the_cache_is_cleared() {
+    let compiler = compiler_for(&reproducible("= Cached\n\nBody text."));
+
+    let first = compile_to_pdf(compiler);
+    assert!(evict_cache(), "clearing the cache failed");
+    let second = compile_to_pdf(compiler);
+
+    // The compiler keeps its sources and fonts in its own world, so only the
+    // memoised results are gone. Recomputing them gives the same document.
+    assert!(second.starts_with(b"%PDF-"), "recompiled output is not a PDF");
+    assert!(first == second, "recompiling after the eviction changed the document");
+
+    unsafe { free_compiler(compiler) };
+}
+
+#[test]
+fn a_result_stays_readable_after_the_cache_is_cleared() {
+    let compiler = compiler_for(b"= Held across an eviction");
+
+    let result = unsafe { compile(compiler, std::ptr::null(), 96.0, 0.0, std::ptr::null()) };
+    assert!(result.error_ptr.is_null(), "valid document failed to compile");
+
+    // The result owns its buffers, so the cache it came from may go away entirely.
+    assert!(evict_cache(), "clearing the cache failed");
+    unsafe { free_compiler(compiler) };
+
+    let pdf = unsafe {
+        let buffer = &*result.buffers;
+        std::slice::from_raw_parts(buffer.ptr, buffer.len)
+    };
+    assert!(pdf.starts_with(b"%PDF-"), "held output was invalidated by the eviction");
+
+    unsafe { free_compile_result(result) };
+}
+
+#[test]
+fn the_cache_can_be_cleared_while_other_threads_compile() {
+    const THREADS: usize = 4;
+
+    // The main thread joins the barrier too, so it only starts evicting once
+    // every thread is about to compile, and keeps evicting until all are done.
+    let started = Arc::new(Barrier::new(THREADS + 1));
+    let finished = Arc::new(AtomicUsize::new(0));
+
+    let compiling: Vec<_> = (0..THREADS)
+        .map(|index| {
+            let started = Arc::clone(&started);
+            let finished = Arc::clone(&finished);
+            std::thread::spawn(move || {
+                let source = reproducible(&format!("= Thread {index}\n\n#lorem(200)"));
+                let compiler = compiler_for(&source);
+                started.wait();
+
+                let first = compile_to_pdf(compiler);
+                for _ in 0..10 {
+                    assert!(
+                        compile_to_pdf(compiler) == first,
+                        "an eviction during compilation changed the document"
+                    );
+                }
+
+                unsafe { free_compiler(compiler) };
+                finished.fetch_add(1, Ordering::SeqCst);
+            })
+        })
+        .collect();
+
+    started.wait();
+    let mut evictions = 0;
+    while finished.load(Ordering::SeqCst) < THREADS {
+        assert!(evict_cache(), "clearing the cache failed");
+        evictions += 1;
+    }
+
+    for thread in compiling {
+        thread.join().expect("a compilation failed while the cache was being cleared");
+    }
+
+    assert!(evictions > 0, "no eviction overlapped the compilations");
+}