Repository navigation
Description
So I really like:
Cache: operational memory used by default solver routines during compute in hot-loops etc for maximizing performance of source code. E.g. storing calculated gradients, hypothesis recipes, or ode-solutions. Good solver will keep thisisbitstype == true. Any non-standard in-memory objects should instead use the "scratch" field.Scratch: Custom in-memory location generally associated with the loading or saving of objects, for holding case-by-case in-memory values that accelerate the compute tasks. "Scratch" is available for custom designs of e.g. factors whereby the developer wishes to manually craft in-place objects to optimize compute operations - e.g. loading lidar blobs from storage into a factor scratch as well as faster rigid transform registration compute techniques. Up to user to be good developer and eventually get.scratchtoisbitstype == true, but doesn't have to be on day one. In contrast, solver default routines should instead use the "cache" field.This means that any time a sampler or residual function runs, the user will have access to the
CalcFactorobject's mutable memory:
calcfactor.scratch: here user/developer does their custom work to make the machine go. Also whatever the user setup on factor load/add.calcfactor.cache: in-place solver stored values such as gradients or whatever else happens per default.Remember the solver design is going down road of always creating a new
CalcFactorobject for each independent thread. I.e.calcfactor.*only lasts as long as the thread/hot-loop used to compute with it.Then there is also the issue of
parching, e.g. load aFactorComputefrom DBDFG or FileDFG such as lidar registration where .laz files representing two point clouds are stored as blobs. Then thepreambleScratchfunction would run once and a user is responsible for something likescratch.pc1 = getBlob(parchedfactor.pointcloud1blobid,...). We have a first example of this, let me see if I can find... yes all theObjectAffordSubcloudstuff: https://github.com/JuliaRobotics/Caesar.jl/blob/9df2df54d4a1bc271afafd5730d35c95af4c67f5/src/objects/ObjectAffordanceSubcloud.jl#L241
Originally posted by @dehann in #1127 (comment)
Activity
Affie commentedon Apr 20, 2026
rebuildFactorCache!is currently coupled tosolverParamsso removing SolvarParams (#1205) influences how caching works.Affie commentedon Apr 20, 2026
Option 1: "Always There" (Implicit Lifecycle)
The cache is automatically populated the moment a factor is created (e.g., via addFactor!) or loaded from disk. It permanently occupies RAM for the entire lifespan of the factor in the graph, regardless of whether it is actively being used in a solve.
Option 2: "Built Just-In-Time" (Explicit Lifecycle)
The factor initializes with an empty cache. The solver is strictly responsible for building the cache during a "warm-up" phase right before inference. This allows the solver to actively manage RAM—building caches only for the active subgraph and discarding them when no longer needed, while the core factor data remains safely in the graph.
Affie commentedon Apr 20, 2026
We need to finalize the verb to build the cache. Currently
rebuildas inrebuildFactorCache!.(WIP) Some other options:
warmupget!function,get!(f::Function, collection, key)orget!(collection, key, default)hydrateCache!inflate!build!dehann commentedon Apr 20, 2026
Caveats:
Formalizing DFG "Data Level 4" -- i.e.
{Variable,Factor}Computevia UXUX 1 - Local Data Wrangling (likely a power user)
UX 2 - Remote Data Wrangling (likely an newbie)
UX 3 - Local Factor Dev (blind customer DX)
UX/DX 4 - Local Factor Dev, using NavAbility API data (customer DX)
UX 5 - Performance DX
UX 6 - Offload Solve
Affie commentedon Apr 21, 2026
I didn't know from just reading the first comment to what
Scratchreferred and after an explanation (meaning it might be the wrong noun) it is the "userCache" in IIF, called as:userCache = preambleCache(dfg, Xi, usrfnc)CalcFactorthe field is just calledcache._createCCWalsouserCacheand then alsodummyCacheCalcFactorNormSqcalls thiscache = ccwl.dummyCacheFrom the current use in IIF,
SolverCacheandUserCacheare also options.The name should reflect: “user cache/preallocs for use in factor residual calculations hot loops“.
Nouns for cahche/preallocs part:
WorkspaceBufferCache- SciML calls similar objects cache, see PreallocationTools.PreallocsUser observation Lifecycle (all optional):
Building the solver cache is separate from the observation and a solver specific feature not influenced by user domain.
Affie commentedon Apr 29, 2026
In a conversation with Dehann using
Stashalso came up.stashis currently used as a verb, but I'm open to changingstashBlobproviders!or usingStashas a noun. However, from the noun alone I'm not sure where it will be used and that kind of says it is not the correct noun.Affie commentedon May 2, 2026
I’m leaning towards
prepareCache(to stay close topreambleCache) orbuildObservationCachefor building the user object to use in “Thread-Local Storage / Preallocated Workspace / Scratchpad Memory / Ephemeral State: build the user cache that is available for residual functions”I also found the original
preambleCacheissue here: JuliaRobotics/IncrementalInference.jl#1462The difference between my suggested observation cache object and the
preambleCacheobject is that the observation cache object should be thread safe andprepareCachewill be called for each thread.Affie commentedon Jul 27, 2026
Similar to the question on a Dict of
FactorDFG.stateby state label (solve key). I have a need for cache to be keyed to enable reuse (depending on where we want the cache to be).