numpy for Ascend NPU: fork of CuPy, backend-extended, API-compatible

By Qingfeng Xia

API Docs | Developer Docs | Benchmark | Architecture | Issues

English | 中文 Readme

numpy-ascend brings the CuPy ndarray API to Huawei Ascend NPUs. It is a fork of CuPy (v14 lineage, NumPy 2.x) with the CUDA backend replaced by an Ascend backend built on CANN's aclnn operator library, so that existing NumPy/CuPy code and the wider CuPy/SciPy ecosystem run on the NPU.

Unofficial fork. numpy-ascend is a derivative work of CuPy and is not affiliated with, endorsed or sponsored by Preferred Networks, Inc. or the CuPy project. Upstream copyright notices are retained in LICENSE and NOTICE; the name "CuPy" is used only to describe the origin of this work.

Import name: numpy_xpu (preferred) or cupy (compatible)

import numpy_xpu as xp    # public import name of this distribution
import cupy as cp         # kept working: xp is cp

numpy_xpu is a thin alias package (numpy_xpu/__init__.py) over the upstream-derived cupy tree. That tree keeps its original directory name because the Cython/C++ sources and the wider import cupy ecosystem hard-code it; both names resolve to the same module object, so there is no duplicated set of classes and isinstance / except behave identically. Existing code needs no change; new code should prefer import numpy_xpu as xp.

Features

0. complete ArrayAPI, 98% of cupy API, cupyx is not yet completed
1. Default dtype float32 (same as CuPy on GPU) — 10x–200x NPU acceleration for math workloads.
2. float64 and complex128 are supported, with three processing modes (see below) because the NPU has no native double-precision throughput and some aclnn ops reject DOUBLE/COMPLEX128 outright.
3. int64 and int32 have hardware acceleration for add/subtract/multiply/divide.
4. API-compatible with CuPy (GPU) and mostly compatible with NumPy (CPU); works with the CuPy and SciPy ecosystem.

float64 / complex128 processing modes

Selected via the environment variable CUPY_ASCEND_FLOAT64_MODE (read once at first use; legacy switch CUPY_ASCEND_ENABLE_FLOAT64_TO_FLOAT32=1 is equivalent to float32):

Mode	Behavior	Precision	Speed
float32 (default)	Demote float64/complex128 operands to float32/complex64 at the dispatch layer, compute on NPU, cast results back.	loses ~9 decimal digits per operation	fast (NPU)
cpu	Whole-op interception: D2H → NumPy in true double precision → H2D. Works for ufuncs, reductions and most general ops (unsupported ones fail loudly instead of silently).	exact (matches NumPy)	slow (PCIe transfers per op)
off	No demotion, no fallback: ops that cannot run in double precision raise an error.	exact or error	—

export CUPY_ASCEND_FLOAT64_MODE=cpu     # exact double precision, host-computed
export CUPY_ASCEND_FLOAT64_MODE=float32 # default: NPU speed, demoted precision
export CUPY_ASCEND_FLOAT64_MODE=off     # never silently lose precision

Why the default float32 mode can be wrong for you — error accumulation:

• float32 carries a 24-bit mantissa: ~7 significant decimal digits (float64: ~16). Each arithmetic op rounds to the nearest representable value, and the rounding errors accumulate across the computation.
• Reductions are the worst case: summing N values has worst-case relative error ~N·eps — summing 10^8 random float64 samples in float32 mode can drift by 1e-2 relative, versus ~1e-16 in cpu mode. Long chains of elementwise ops (iterative solvers, cumsum, normalization) compound the same way; values above 2^24 (≈1.7e7) lose integer precision entirely.
• complex128 demotes to complex64, so both real and imaginary parts share the same 24-bit mantissa — the relative error applies to magnitude and phase alike.
• Rule of thumb: for ML training/inference and image data, float32 mode is the intended trade-off. For numerical verification, financial/scientific accumulation (large sums, variance over big arrays, eigen/solvers where the residual matters), use cpu mode — it is exact and easy to switch per process; or keep float64 data in NumPy and move only the float32-hot path to the NPU.

Support Matrix

OS: Linux only. Any Linux distribution since 2022 should work provided the C++ runtime (libstdc++, glibc >= 2.27) is new enough; CI is done on aarch64 Linux.

CANN	x86_64	aarch64	Notes
9.2	source	source	not tested; may work, no CI coverage
9.0	binary wheel + source	source	primary test target (Ascend 910B)
8.5	binary wheel + source	source	primary test target (Ascend 910B)
8.2	source build	source build	tested, no CI coverage

Python	Binary wheel	Source build
3.10	yes	yes
3.11	yes	yes
3.12	yes	yes

• Binary wheels: see the Releases page. Wheel naming: numpy_ascend-<ver>+cann<XY>-cp<py>-cp<py>-manylinux_2_34_<arch>.whl for non-PyPI trains (the +cann8.5 / +cann9.0 local version segment, PyTorch-style) and numpy_ascend-<ver>-cp<py>-cp<py>-manylinux_2_34_<arch>.whl for the train published to PyPI (currently CANN 9.0). Pick the wheel matching your interpreter (cp311 = Python 3.11), CPU architecture and CANN release train.
• The wheel does not bundle the CANN SDK: the target machine needs a CANN toolkit of the same release train plus set_env.sh; see docs/ascend/Package.md for the hard runtime prerequisites (libstdc++ version, optional ops-fft, etc.).
• Only one package providing cupy may be installed per environment: never co-install this with upstream cupy / cupy-cudaXX or a different CANN train — they would overwrite each other's files.

Installation

1. From a binary wheel (recommended)

CANN 9.0.x — install from PyPI (the primary release train):

pip install numpy-ascend                # latest release
pip install numpy-ascend==14.0.0a1      # pin a version

CANN 8.5.x — install from the GitHub Release. PyPI rejects local version identifiers, so only the 9.0 train is published there; the 8.5 train ships as release assets. Point pip at the release asset page with --find-links (it accepts an HTML page — GitHub Releases are not a PEP 503 index, so --index-url does not work) and pin the train explicitly:

pip install numpy-ascend==14.0.0a1+cann8.5 \
    --find-links https://github.com/qingfengxia/numpy-ascend/releases/expanded_assets/v14.0.0a1

• Installation Guide - instructions on building from source
• Release Notes
• Projects using CuPy
• Contribution Guide

Learning Materials

• CUDA Python Samples (NVIDIA CUDA Samples)
• NVIDIA Accelerated Computing Hub
• GPU Acceleration in Python using CuPy and Numba (GTC November 2021 Technical Session)
• GPU-Acceleration of Signal Processing Workflows using CuPy and cuSignal (ICASSP'21 Tutorial)[^1]

Each release ships one wheel per (CPython × architecture × CANN train), e.g. for release v14.0.0a1:

CANN on the target machine	x86_64 wheel	aarch64 wheel	Where
9.0.x	numpy_ascend-14.0.0a1-cp311-cp311-manylinux_2_34_x86_64.whl	numpy_ascend-14.0.0a1-cp311-cp311-manylinux_2_34_aarch64.whl	PyPI + Release
8.5.x	numpy_ascend-14.0.0a1+cann8.5-cp311-cp311-manylinux_2_34_x86_64.whl	numpy_ascend-14.0.0a1+cann8.5-cp311-cp311-manylinux_2_34_aarch64.whl	Release only

The non-PyPI trains carry the train as a PEP 440 local version segment (the same scheme as PyTorch's torch==2.5.1+cu121), so it can be pinned exactly. Any release wheel can also be installed by direct URL or local file (download URL pattern: .../releases/download/<tag>/<wheel-file>):

pip install https://github.com/qingfengxia/numpy-ascend/releases/download/v14.0.0a1/numpy_ascend-14.0.0a1+cann8.5-cp311-cp311-manylinux_2_34_x86_64.whl

# or download the wheel from the Releases page first, then install the file:
pip install ./numpy_ascend-14.0.0a1+cann8.5-cp311-cp311-manylinux_2_34_x86_64.whl

source /usr/local/Ascend/ascend-toolkit/set_env.sh
export ASCEND_HOME_PATH=/usr/local/Ascend/ascend-toolkit/latest

python -c "import cupy; print(cupy.__version__, cupy.backends.ascend.check_cann_version())"

Note: when --find-links is in play, always pin the train explicitly — ==14.0.0a1 also matches the +cann8.5 local version, and pip prefers the higher (local) version. 9.0 users should install from PyPI without --find-links.

How to choose the CANN train: it must match the CANN toolkit installed on the target machine (ls /usr/local/Ascend/, or npu-smi info), not the NPU driver version. A +cann8.5 wheel on a CANN 9.0 host (or vice versa) fails at import with an aclnn ABI mismatch — install the matching train (pip uninstall numpy-ascend first, since all trains share one distribution name).

2. From source (new platform / CANN version, or development)

git clone git@github.com:qingfengxia/numpy-ascend.git
cd numpy-ascend && git checkout ascend      # the working branch is ascend, not main
export CUPY_INSTALL_USE_ASCEND=1
python setup.py build_ext --inplace         # incremental in-place build (dev)
python -c "import cupy._core"               # L2/L3 check: link + op registration OK
python -m build --wheel                     # optional wheel (cannX.Y tag auto-detected)
python -m build --sdist                     # optional sdist (one file, CPython/CANN agnostic)

Environment setup (CANN 8.2/8.5/9.0 installation and switching, development on machines without an NPU, porting to new platforms, troubleshooting) is documented in DeveloperNotes.md §1–§3; wheel and sdist packaging strategy is in docs/ascend/Package.md.

3. Verify

pytest tests/ascend -q                     # no NPU needed: registry / composed ops / kernel parsing
pytest tests/cupy_tests -q                 # needs NPU; unsupported dtypes skipped by default
pytest tests/cupy_tests -q --ascend-dtype-filter=off   # full dtype matrix (shows real failures)
python tools/benchmark.py --list           # no NPU needed: print the op x dtype matrix
python tools/benchmark.py --csv result.csv # needs NPU

# minimal numerical check (needs NPU)
python -c "
import numpy as np, cupy as cp
x = np.random.rand(1000).astype(np.float32)
assert np.allclose(cp.asnumpy(cp.asarray(x).sum()), x.sum(), rtol=1e-5); print('ok')
"

Verification levels: L1 Cython generates .cpp → L2 links → L3 import cupy + op-registry audit → L4 numerical results on real hardware. A machine without an NPU can only reach L3; L4 must run on a 910B.

Project Status

API coverage

See Progress.md for the full snapshot:

• Python Array API standard: > 98% covered (2 gaps: i0 — NumPy itself recommends scipy.special.i0 — and nextafter).
• CuPy top-level API: > 98% covered.
• Operator-level facts are auto-generated by tools/scan_ops.py into tools/cst_db.md.

Test results (on Ascend 910B)

Suite	Pass rate
tests/cupy_tests/math_tests (math_test)	100%
full tests/cupy_tests + tests/ascend	90%

Failures concentrate in the documented limitation areas below, not in the math kernels.

Completed

1. Custom kernels (AscendC, and Triton-Ascend in Python).
2. All major CuPy features: custom kernels, profiler, stream/device management — except operator fusion (use a Triton kernel instead).
3. scipy ecosystem support via array_api, see scipy benchmark with numpy-ascend

Limitations

1. uint64 is not supported (int64 is used, with hardware acceleration for add/multiply).
2. Operator fusion.
3. Sparse array/matrix and some cupy.random distributions are supported only partially (low priority).
4. Multi-NPU is not ported/tested yet.

Tools

Tool	Purpose
tools/benchmark.py	CLI benchmark: NumPy (CPU) vs numpy-ascend (NPU) speedup over an op × dtype matrix (unary/binary/reduction/matmul/manipulation/sort). --list prints the matrix without an NPU; --category, --dtype, --repeat, --csv, --strict filter and report. Exit code 2/3 signal import/device errors.
tools/numpy_ascend_migration_helper/	Static migration analyzer for NumPy/CuPy source: reports unsupported APIs/dtypes, semantic differences and CPU-fallback candidates (AST-based, not string matching); --fix applies AUTO_SAFE rewrites, --fail-on-error gates CI. See its README.md.
tools/scan_ops.py	Scans the built backend and generates the operator/coverage database (tools/cst_db.md).

Examples

Python Array API standard

https://github.com/data-apis/array-api

import cupy as cp
import cupy.array_api as cpx   # CuPy's Array API namespace

x_gpu = cpx.asarray([1, 2, 3, 4], device='cuda')  # device slot, NPU here
y_gpu = cpx.reshape(x_gpu, (2, 2))
z_gpu = cpx.matmul(y_gpu, y_gpu)

If you are writing new data-processing code and do not need NumPy/CuPy compatibility, torch_npu's array API (torch._numpy) is an alternative.

Documentation Index

Document	Content
Progress.md	Status snapshot: coverage, operator counts, milestones, remaining gaps
Memory.md	Developer handbook: command cheat-sheet, architecture key points, verification levels, pitfalls
DeveloperNotes.md	Environment setup (incl. no-NPU development), CANN 8.2/8.5/9.0 install, conditional compilation, FFT
docs/ascend/Package.md	Wheel packaging (cann tag), RPATH strategy, runtime hard prerequisites
install/README.md	Build system architecture (features / backends) and op registration flow
docs/ascend/	Custom AscendC kernels, FFT, matmul, code reviews, NumPy/CuPy/PyTorch API diffs
tools/cst_db.md	Auto-generated operator/coverage database (python tools/scan_ops.py)
Roadmap.md · TODO.md	Roadmap and task list

Contributing

Issues and PRs are welcome at https://github.com/qingfengxia/numpy-ascend. For a code change: rebuild with CUPY_INSTALL_USE_ASCEND=1, verify at least to L3 (pytest tests/ascend -q), and state honestly which verification level was reached (see Memory.md). Commit messages use the ASCEND [AI]: prefix on the ascend branch.

License

MIT — see LICENSE. This project forks CuPy; upstream CuPy is MIT-licensed and its copyright notice is preserved in the source tree.

中文文档

简体中文说明见 Readme_ZH.md。

Acknowledgement

CuPy is being developed and maintained by Preferred Networks™ (PFN™), organizational supporters, and community contributors.

Reference

Ryosuke Okuta, Yuya Unno, Daisuke Nishino, Shohei Hido and Crissman Loomis. CuPy: A NumPy-Compatible Library for NVIDIA GPU Calculations. Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS), (2017). [PDF]

@inproceedings{cupy_learningsys2017,
  author       = "Okuta, Ryosuke and Unno, Yuya and Nishino, Daisuke and Hido, Shohei and Loomis, Crissman",
  title        = "CuPy: A NumPy-Compatible Library for NVIDIA GPU Calculations",
  booktitle    = "Proceedings of Workshop on Machine Learning Systems (LearningSys) in The Thirty-first Annual Conference on Neural Information Processing Systems (NIPS)",
  year         = "2017",
  url          = "http://learningsys.org/nips17/assets/papers/paper_16.pdf"
}