0.5.1: conversion routes say what happens; the observer belongs to its context

ADDING_NEW_FRAMEWORKS.md

@@ -176,6 +176,7 @@ class must define.
 | Parent | The library... | Abstract methods |
 |---|---|---|
 | `HostDevices` | keeps arrays in main memory. | `_wrap_host_array(data)`, `_move_to_host(data)` |
+| `NumpyBufferSharing` | is a `HostDevices` library whose main-memory array and a NumPy array can be one buffer (measure it before declaring it). | those of `HostDevices`, plus `_first_element_address(data)` |
 | `AcceleratorDevices` | can keep arrays on numbered accelerators. | `_accelerator_count()`, `_accelerator_scope(device)`, `_move_to_accelerator(data, device)`, `_upload(data, device)`, `_device_if_accepted(value)` |
 | `CudaLibrary` | is an `AcceleratorDevices` library with its own CUDA streams and a memory cache to release. | those of `AcceleratorDevices`, plus `create_stream()`, `stream_scope(stream)`, `current_device()`, `_free_cached_memory()` |
 | `DLPackImport` | can wrap another library's buffer through DLPack. | `_import_dlpack(payload)` |

CHANGELOG.md

@@ -77,12 +77,112 @@ two different libraries.
 `MemoryType`'s metaclass overrides are annotated with `EnumMeta`'s signatures;
 its `__call__` override is gone (`Enum` already asks `_missing_`).
 
+### Conversion
+
+The mechanism names a consumer can now see, in the order the routes are tried:
+
+| Mechanism | Route class | Meaning |
+|---|---|---|
+| `"stays in place"` | `StaysInPlace` (new) | Same library, same device. Nothing is converted; the input object is returned. No record. |
+| `"device move"` | `DeviceMove` | Same library, another device. |
+| `"dlpack"` | `DLPackExchange` | The target wraps the source's buffer through DLPack. |
+| `"shared host buffer"` | `SharedHostBuffer` (new) | Both arrays are in main memory and the result begins at the same address as the source: no bytes were copied. |
+| `"host copy"` | `HostCopy` | The values were copied through a NumPy array in main memory. |
+
+**The plan and the run agree that nothing is converted.**
+`ConversionRoute.planned(library, device, library, device)` returns
+`StaysInPlace`. In 0.5.0 it returned `DeviceMove`, while running that
+conversion returned the input and recorded nothing, so a caller had to restate
+the rule. `DeviceMove` now applies only when the device differs. The rule is
+stated once, in `StaysInPlace.applies`; `convert_to`, `to_numpy`, `from_numpy`
+and `move_to_device` all ask it. Every route has `writes_a_record`; it is
+`False` for `StaysInPlace` only. A record means data was moved, copied or
+wrapped.
+
+**A host exchange that shares the buffer is no longer called a copy.** On the
+host, torch and NumPy share one buffer in both directions, and JAX hands NumPy
+its own buffer; 0.5.0 recorded these as `"host copy"`. They are now recorded
+as `"shared host buffer"`. A library declares that it can share by inheriting
+`NumpyBufferSharing` (NumPy, torch and JAX do; TensorFlow copies in both
+directions and does not). Whether one array was shared is decided after the
+exchange by comparing where the two arrays begin (`host_address_of`).
+
+What was measured (Python 3.12, NumPy 2.5.4, torch 2.14.1 CPU, JAX 0.11.2,
+TensorFlow 2.21.0 CPU; pointers, `np.shares_memory` and writing to one array
+and reading the other):
+
+| Exchange on the host | Result |
+|---|---|
+| NumPy to torch | shared, for every dtype tried, Fortran order, stepped and offset views, read-only arrays. A view with a negative step is copied (arraybridge makes it contiguous first). |
+| torch to NumPy | shared |
+| JAX to NumPy | shared (the NumPy array is read-only) |
+| NumPy to JAX | copied, unless the buffer is C-contiguous, starts on a 64-byte boundary and keeps its dtype; then shared |
+| NumPy to TensorFlow, TensorFlow to NumPy | copied |
+| an array with no elements | nothing to share: `"host copy"` of 0 bytes |
+
+**What `planned` returns when sharing depends on the array.** A plan sees
+only declarations and devices, never an array. It returns the route that is
+tried first: `SharedHostBuffer` for NumPy, torch and JAX with NumPy on the
+host. The record names what was done. Where the exchange copied (NumPy to JAX
+for an ordinary NumPy allocation, a reversed view to torch) the record's
+mechanism is `"host copy"` and `declined_mechanisms` is
+`("shared host buffer",)`, the way a declined DLPack exchange is already
+reported. The exchange is done once either way.
+
+What a consumer that keyed on the old answers sees:
+
+- `planned(...) is DeviceMove` is no longer true for the same library and
+  device; test `is StaysInPlace` (or `route.writes_a_record`).
+- Rows that were `"host copy"` for NumPy and torch on the host, and for JAX
+  to NumPy on the host, are now `"shared host buffer"`. `"host copy"` rows
+  that follow a declined `SharedHostBuffer` carry it in `declined_mechanisms`,
+  which `ConversionCounters` does not key on.
+- `planned` for NumPy with torch or JAX on the host (either direction) is
+  `SharedHostBuffer`. Between torch and JAX on the host it is still `"dlpack"`.
+  Anything to or from an accelerator, pyclesperanto or TensorFlow is planned
+  as before.
+- The `nbytes` of a `from_numpy` record is the size of the array that was
+  made (it was the size of the NumPy input; they differ when JAX narrows
+  float64 to float32).
+
+`ConversionRoute.taken(data, result, source, target)` is new: the route that
+names what `convert` did, the route itself unless it says otherwise.
+`ArrayLibrary.shares_numpy_buffer_on(device)` and
+`ArrayLibrary.host_address_of(data)` are new, as is
+`NumpyLibrary.registered()`. `HostCopy` and `SharedHostBuffer` share the new
+parent `HostExchange`, which has no mechanism of its own.
+`arraybridge.libraries` no longer imports `ConversionObserver`,
+`ConversionRecord` and `HostCopy` (they were never exported from it; they are
+in `arraybridge.conversion`).
+
+**`ConversionObserver.current` belongs to the context that reads it.** It was
+one attribute for the whole process, and `installed()` put back whatever it
+had seen on entry: two threads that entered and left in the same order left
+the second thread's observer installed for the rest of the process. The hook
+is now a `contextvars.ContextVar`.
+
+- `ConversionObserver.current` reads the observer of the running thread or
+  asyncio task. Assigning to it sets the observer of that context only.
+- `observer.install()` makes it the observer of this context and returns the
+  one it replaced. `with observer.installed():` does so for a block.
+- A new thread starts with the default observer, which ignores conversions.
+  **An observer installed in the main thread no longer sees conversions made
+  in worker threads.** To observe a thread you start, run it in a copy of your
+  context: `Thread(target=contextvars.copy_context().run, args=(work,))`, or
+  `executor.submit(contextvars.copy_context().run, work)`.
+- An asyncio task starts with the observer of the code that created it, and
+  what it installs stays in that task.
+- `current` is read on the class (`ConversionObserver.current`). It is no
+  longer an attribute of an observer instance.
+
 ### Not verified on a GPU
 
 0.5.1 was tested with NumPy on Python 3.10, 3.12 and 3.13 and with the torch
-CPU wheel on 3.12. CuPy, TensorFlow, JAX and pyclesperanto, and torch on CUDA,
-were exercised only through the fakes in `tests/fakes.py`. `TESTING.md` has
-the commands for a GPU machine.
+CPU wheel on 3.12. The host exchanges in the Conversion section were also run
+against the JAX and TensorFlow CPU builds named there. CuPy and pyclesperanto,
+every accelerator path of torch, TensorFlow and JAX, and everything else
+TensorFlow and JAX do, were exercised only through the fakes in
+`tests/fakes.py`. `TESTING.md` has the commands for a GPU machine.
 
 ## 0.5.0
 

README.md

@@ -99,10 +99,13 @@ library does not have.
 
 ## Which route a conversion takes
 
-A conversion tries three mechanisms in order: `"device move"` (same library,
-other device), `"dlpack"` (the target wraps the source's buffer) and
-`"host copy"` (download to NumPy, then upload). `ConversionRoute.planned`
-tells which one will be tried first. It reads only the library classes and the
+A conversion tries five mechanisms in order: `"stays in place"` (same library
+and device: nothing is converted), `"device move"` (same library, other
+device), `"dlpack"` (the target wraps the source's buffer),
+`"shared host buffer"` (both arrays are in main memory and the result is the
+source's own buffer) and `"host copy"` (the values are copied through NumPy).
+`ConversionRoute.planned` tells which one will be tried first; the record of
+a conversion names the one that was used. It reads only the library classes and the
 device values, so it works without the libraries or the hardware:
 
 ```python
@@ -115,11 +118,12 @@ gpu = AcceleratorDevice(0)
 assert ConversionRoute.planned(torch, gpu, cupy, gpu).mechanism == "dlpack"
 assert ConversionRoute.planned(torch, HOST, cupy, gpu).mechanism == "host copy"
 assert ConversionRoute.planned(torch, HOST, torch, gpu).mechanism == "device move"
+assert ConversionRoute.planned(torch, gpu, torch, gpu).mechanism == "stays in place"
 ```
 
 To see what conversions actually ran, install a `ConversionCounters` observer.
 It keeps a count, bytes and seconds per source, target and mechanism. This
-example needs JAX:
+example needs TensorFlow:
 
 ```python
 import numpy as np
@@ -128,16 +132,18 @@ from arraybridge import HOST, ConversionCounters, convert_memory
 
 counters = ConversionCounters()
 with counters.installed():
-    convert_memory(np.zeros((4, 4), dtype=np.float32), "numpy", "jax", HOST)
+    convert_memory(np.zeros((4, 4), dtype=np.float32), "numpy", "tensorflow", HOST)
 
 row = counters.rows()[0]
-assert (row["source_library"], row["target_library"]) == ("numpy", "jax")
+assert (row["source_library"], row["target_library"]) == ("numpy", "tensorflow")
 assert (row["mechanism"], row["count"], row["nbytes"]) == ("host copy", 1, 64)
 print(counters.format_table())
 ```
 
-A call that returns its input unchanged is not counted. With no observer
-installed nothing is timed.
+A conversion that stays in place is not counted. With no observer installed
+nothing is timed. An observer belongs to the thread or asyncio task that
+installed it; a thread you start is observed when you run it in a copy of your
+context (`contextvars.copy_context().run`).
 
 ## Array operations in the array's own library
 

docs/source/api_reference.rst

@@ -138,22 +138,35 @@ Conversion reporting
 
 ``ConversionRoute`` has one subclass per mechanism, tried in this order:
 
+``StaysInPlace`` (``"stays in place"``)
+  Same library and same device. The input is returned and no record is
+  written (``writes_a_record`` is ``False`` for this route only).
+
 ``DeviceMove`` (``"device move"``)
   Source and target are the same library; the array changes device only.
 
 ``DLPackExchange`` (``"dlpack"``)
   The target wraps the source's buffer. Applies when the target imports
   DLPack from the source's device and the source exports from it.
 
+``SharedHostBuffer`` (``"shared host buffer"``)
+  Both arrays are in main memory and the result begins at the address of the
+  source: no bytes are copied. Applies when both libraries are
+  ``NumpyBufferSharing`` (NumPy, torch, JAX) and both devices are the host.
+  Whether one array is shared is known after the exchange; one that was
+  copied is recorded as ``"host copy"`` with ``"shared host buffer"`` under
+  ``declined_mechanisms``.
+
 ``HostCopy`` (``"host copy"``)
-  Download to a NumPy array, then upload to the target. Always applies.
+  The values are copied through a NumPy array. Always applies.
 
 ``ConversionRoute.planned(source_library, source_device, target_library,
 target_device)`` returns the route class that a conversion of that pair tries
 first. It reads only the library classes and the device values: no library is
 imported and no device is touched. The conversion selects its route with the
 same test. A route that applies but cannot carry a particular array is
-skipped and the next one is tried.
+skipped and the next one is tried, so the record of a conversion, not the
+plan, says what was done to that array.
 
 .. code-block:: python
 
@@ -166,19 +179,21 @@ skipped and the next one is tried.
 
    assert ConversionRoute.planned(torch, gpu, cupy, gpu).mechanism == "dlpack"
    assert ConversionRoute.planned(torch, HOST, cupy, gpu).mechanism == "host copy"
-   assert ConversionRoute.planned(numpy_library, HOST, torch, HOST).mechanism == "host copy"
+   assert ConversionRoute.planned(numpy_library, HOST, torch, HOST).mechanism == "shared host buffer"
+   assert ConversionRoute.planned(torch, gpu, torch, gpu).mechanism == "stays in place"
    assert ConversionRoute.planned(torch, HOST, torch, gpu).mechanism == "device move"
 
 ``ConversionObserver``
   Receives one ``ConversionRecord`` per conversion while it is installed.
   Subclass it and define ``record(record)``. ``observer.install()`` makes it
-  the process's observer and returns the one it replaced;
-  ``with observer.installed():`` does so for a block.
-  ``ConversionObserver.current`` is the installed observer. The default
+  the observer of the running thread or asyncio task and returns the one it
+  replaced; ``with observer.installed():`` does so for a block.
+  ``ConversionObserver.current`` is the observer of the context that reads
+  it (a ``contextvars.ContextVar``). A new thread starts with the default;
+  run it with ``contextvars.copy_context().run`` to give it yours. The default
   ignores conversions, and then nothing is timed. ``library.convert_to``
   (which ``convert_memory`` calls), ``library.to_numpy`` and
-  ``library.from_numpy`` report. A call that returns its input unchanged does
-  not.
+  ``library.from_numpy`` report. A conversion that stays in place does not.
 
 ``ConversionRecord``
   Fields: ``source_library``, ``source_device``, ``target_library``,

docs/source/converters.rst

@@ -45,9 +45,11 @@ answers about them.
 Routes
 ------
 
-A conversion tries, in order: a device move when source and target are the
-same library; a DLPack exchange when the target can wrap the source's buffer
-on its device; a copy through a NumPy array in main memory.
+A conversion tries, in order: nothing at all when source and target are the
+same library on the same device; a device move when they are the same library;
+a DLPack exchange when the target can wrap the source's buffer on its device;
+an exchange through NumPy in main memory, which shares the buffer where both
+libraries can (NumPy, torch, JAX) and copies otherwise.
 ``ConversionRoute.planned`` tells which is tried first for a pair, and a
 ``ConversionObserver`` receives what actually ran; both are described in
 :doc:`api_reference`. No pair is promised to be zero-copy.