Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ it may change without notice: do not depend on it directly. We plan to offer the

## Requirements

- MI355X, ROCm 7.2, Python 3.12, PyTorch for ROCm.
- MI355X or MI350X (gfx950), ROCm 7.2, Python 3.12, PyTorch for ROCm.
- [FlyDSL](https://github.com/ROCm/FlyDSL) >= 0.5.0 and [AITER](https://github.com/ROCm/aiter).

## Install and test
Expand All @@ -29,12 +29,12 @@ python3 benchmarks/<category>/<name>/<benchmark>.py # CUDA graph replay over i

### 1. AMD only

Kernels target AMD Instinct GPUs (today MI355X, gfx950). There is no platform-agnostic API or kernel registry; the
host code picks the variant by shape.
Kernels target AMD Instinct GPUs (today gfx950: MI355X and MI350X). There is no platform-agnostic API or kernel
registry; the host code picks the variant by shape.

### 2. FlyDSL only

No Triton, Gluon, CuteDSL or HIP `.cu`; glue is torch ops or FlyDSL. FlyDSL covers what we would otherwise write in
No Triton, Gluon or HIP (`.hip`, `.cu`); glue is torch ops or FlyDSL. FlyDSL covers what we would otherwise write in
HIP, Triton or Gluon: tile-level ops plus direct buffer loads, LDS DMA, MFMA and inline asm. One language gives one
set of tools, tests and recorded lessons. We accept longer and more complex kernel code for that.

Expand All @@ -44,22 +44,22 @@ Learned from [TokenSpeed kernels](https://github.com/lightseekorg/tokenspeed/blo
whose PyTorch references are the ground truth their kernels are tested against. One reference function per op, in the
kernel package's `reference.py`; tests and benchmarks import it and never write their own. Its docstring states what
"correct" means: where each rounding and quantization happens, the scale layout, the accumulate dtype, and the tolerance
the tests use (cosine ~1 and deterministic, not bitwise). Test shapes come from the served regime, including CUDA graph
padding rows.
the tests use (bit for bit where the kernel matches the reference exactly, otherwise cosine ~1; two runs always match
bit for bit). Test shapes are the served ones, including the padding rows of CUDA graph batch sizes.

### 4. Every variant compiles before serving

All variants a server can hit (every shape tier and config) compile during warm-up or CUDA graph capture. An entry
registered with `amd_kernels.warm_up.entry` warms itself up on its first call of each mode, compiling in worker
processes ([`warm_up.py`](amd_kernels/warm_up.py)); after a caller's warm-ups, a compile is reported with its call site
([`compile_watch.py`](amd_kernels/compile_watch.py)). A test walks
the batch sizes after a warm-up and asserts that nothing compiles.
([`compile_watch.py`](amd_kernels/compile_watch.py)). A test walks the batch sizes after a warm-up and asserts that
nothing compiles.

### 5. One kernel per served shape is allowed

A kernel may be written for exactly the shapes it serves, and a new shape may get its own copy instead of another
branch in a shared kernel. The fastest instruction order and scheduling differ from shape to shape (tile, pipeline
depth, how MFMAs interleave with loads, epilogue, grid), so a copy per shape lets an agent push each one to its limit
depth, how MFMAs interleave with loads, epilogue, grid), so a copy per shape lets each one be pushed to its limit
without regressing the others. We accept the duplicated code for that; a review then touches one shape.

### 6. Names carry the computation and the shape
Expand All @@ -81,6 +81,7 @@ We accept the reduce step that an atomic would skip for that.
amd_kernels/<category>/<name>/ kernels, reference.py, README.md
tests/<category>/<name>/ tests against reference.py
benchmarks/<category>/<name>/ graph-replay benchmarks
tools/ CI (Slurm + enroot) and profiling scripts
```

Writing rules for code and docs: [AGENTS.md](AGENTS.md).
Expand Down