Skip to content

feat(qy): QY ops backend with FA and longctx MoE/RoPE kernels (2026.11.12) - #1038

Open
Ceng23333 wants to merge 8 commits into
InfiniTensor:masterfrom
Ceng23333:feat/qy-backend-20261112
Open

Ceng23333 wants to merge 8 commits into
InfiniTensor:masterfrom
Ceng23333:feat/qy-backend-20261112

Conversation

@Ceng23333

@Ceng23333 Ceng23333 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Denglin QY native CUDA ops backend (add/gemm M0 path + MoE/RoPE/FA kernels) rebased onto current master.
  • Torch instantiate.h / stream_guard.h wire WITH_QY C10 specialization.
  • Gate Device::Type::kQy behind WITH_QY so non-QY unit jobs still compile against InfiniRT master (no kQy enum).

Motivation / related

Type of change

  • feat
  • fix (clang-format; kQy CI gate; QY stream_guard include)

Platforms affected

  • QY (Denglin)
  • NVIDIA/Iluvatar/etc.: compile surface only via WITH_QY gate (no functional change when QY off)

Validation

  • Host Denglin QY, DENGLIN_DEVICES=3.
  • Build against RT ci: add clang-format checking GitHub Action #54 head 047340f45efa; pytest tests/test_add.py tests/test_gemm.py --devices qy → 1633 passed, 6 skipped.
  • See PR comments: QY required smoke (PR head) (+ detailed).
  • Logs: nov_milestone/logs/prhead-verify-20261113/ops-pytest-add-gemm.log

Checklist

  • clang-format-21 clean (CI green)
  • Conventional Commits title/branch (feat/qy-backend-20261112)
  • Full non-QY unit matrix: some jobs fail on self-hosted git proxy 127.0.0.1:11312 connection refused (infra, not kQy compile)

Stack PR index (merge order)

  1. InfiniOps: feat(qy): QY ops backend with FA and longctx MoE/RoPE kernels (2026.11.12) #1038 — 71e85162ab96
  2. InfiniRT: feat(qy): enable QY CUDA runtime device backend (2026.11.12) InfiniRT#54 — 047340f45efa
  3. InfiniCCL: feat(qy): enable InfiniCCL NCCL backend for Denglin QY (2026.11.12) InfiniCCL#90 — 691c3092c4c9
  4. InfiniCore: chore(qy): pin InfiniRT/Ops/CCL for QY MoE bf16-warpnk (2026.11.12) InfiniCore#1583 — 0b69ee9002bd
  5. InfiniLM: feat(qy): MiniCPM5 FA/longctx/cudagraph + MoE bf16/warpnk surpass (2026.11.12) InfiniLM#618 — 36beae5376f4 → base refactor/adopt-modern-infini-stack

ITW pin: https://github.com/Ceng23333/InfiniTensorWorktree/tree/v2026.11.12-qy-moe-bf16-warpnk

Ceng23333 and others added 4 commits November 12, 2026 17:26
Enable Denglin/QY native and torch paths, linked flash-attn varlen/kvcache
providers via *_no_sinks, and reshape_and_cache_flash for paged caching.
Promote mul_scalar, silu_and_mul, moe_sum/align/topk_sigmoid, and rope/rope_pack kernels from the longctx adapting lane onto feat/qy-backend.
Co-authored-by: Cursor <cursoragent@cursor.com>
Upstream moved device instantiation to instantiate.h macros; add kQy
alongside the rebased QY backend so torch providers link under WITH_QY.

Co-authored-by: Cursor <cursoragent@cursor.com>
bench and others added 2 commits November 13, 2026 10:56
Format failing clang-format-21 paths and wrap Device::Type::kQy behind
WITH_QY so unit jobs building against InfiniRT master still compile.
WITHOUT this, WITH_QY torch ops fail to instantiate StreamGuard.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Ceng23333

Copy link
Copy Markdown
Contributor Author

QY required smoke (PR head)

  • Head SHA: 71e85162ab96192d7fda620e1c80e21f02fadc3f
  • Host: Denglin QY, DENGLIN_DEVICES=3
  • Command: pytest tests/test_add.py tests/test_gemm.py --devices qy (m0 ops config build against InfiniRT PR head 047340f45efa)
  • Result: PASS (1633 passed, 6 skipped)
  • Log: nov_milestone/logs/prhead-verify-20261113/ops-pytest-add-gemm.log
1633 passed, 6 skipped, 1 warning in 15.64s

@Ceng23333

Copy link
Copy Markdown
Contributor Author

QY required smoke (PR head) — detailed

  • Head SHA: 71e85162ab96192d7fda620e1c80e21f02fadc3f
  • Host: Denglin QY, DENGLIN_DEVICES=3
  • Build: m0 ops config (add,gemm) vs InfiniRT PR head 047340f45efa
  • Command: pytest tests/test_add.py tests/test_gemm.py --devices qy
  • Overall: PASS
  • Log: nov_milestone/logs/prhead-verify-20261113/ops-pytest-add-gemm.log
Suite Device Passed Skipped Failed Wall time
test_add + test_gemm qy 1633 6 0 15.64s

Also on this tip: clang-format-21 CI green; kQy gated behind WITH_QY for non-QY unit compile; stream_guard.h includes QY C10. Remaining unit failures on nvidia/ascend are infra git proxy 127.0.0.1:11312 connection refused (not kQy enum).

Co-authored-by: Cursor <cursoragent@cursor.com>
@Ceng23333

Copy link
Copy Markdown
Contributor Author

CI note (infra re-run follow-up)

Tip: 67a6b3c10fc7 (ci: retrigger unit jobs after runner proxy flake) after prior tip 71e85162ab96.

What we did

  1. Waited out prior CI run 43356761569 / shadow 43356761509 — ci / unit / iluvatar ended cancelled; shadow iluvatar failure; nvidia/ascend already red.
  2. Re-ran failed jobs on both runs via rerun-failed-jobs.
  3. First pass after re-run still died at checkout on runner proxy http://127.0.0.1:11312 (Connection refused, git exit 128) — hosts included Nvidia-Server-005, Ascend-910C-Server-001, shadow Nvidia-Server-002 / Ascend-910C-Server-002.
  4. One empty-commit retrigger → new CI 43376178404 / shadow 43376178438.

Current classification (not a kQy / compile defect)

Check Runner (latest tip) Result
clang-format / ruff / docs — green
ci / unit / metax Metax-Server-006 green (others still running)
ci / unit / nvidia Nvidia-Server-006 red — actions/checkout → proxy :11312 Connection refused (exit 128); never reaches build
ci / unit / ascend Ascend-910C-Server-000 red — same
shadow nvidia / ascend Nvidia-Server-001 / Ascend-910C-Server-002 red — same
ci / unit / iluvatar (+ shadow) Iluvatar-Server-006 / 000 still in_progress at comment time (past checkout)

Required format is green. Remaining nvidia/ascend unit reds are runner proxy checkout infra (127.0.0.1:11312), not a QY/kQy/Device::Type failure. No further Ops code commits planned for this flake; needs org runner/proxy attention.

Comment thread CMakeLists.txt Resolved
@baominghelly

Copy link
Copy Markdown
Collaborator

Comment thread src/native/cuda/qy/mul_scalar_kernel.cu Resolved
Comment thread src/native/cuda/qy/rope_pack_kernel.cu Resolved
@baominghelly

Copy link
Copy Markdown
Collaborator

cuda/qy下面不放具体算子的文件,请全部迁移到ops文件夹下面对应的算子文件夹下面。

@baominghelly

Copy link
Copy Markdown
Collaborator

我发现include/下面的launcher都没有被调用,可以等用到的时候再加;如果加的话,也是加在调用算子的文件夹下面

Wire qy into scripts/dev/build.sh; honor -DQY_ROOT without wipe+FORCE;
move flat cuda/qy op kernels under ops/; drop unused include/ launchers;
fix mul_scalar half/bf16 alpha overflow and rope_pack FP32 vector copy.

Squash of Contents-API micro-commits after 67a6b3c (build.sh through include cleanup).
@Ceng23333

Copy link
Copy Markdown
Contributor Author

已补充(tip fae5eedf26de33c9c0437ec915b26138ad34f730):scripts/dev/build.sh 已加入 qy 到 usage、case 分支,并传入 -DWITH_QY。

@Ceng23333

Copy link
Copy Markdown
Contributor Author

已按建议迁移(tip fae5eedf26de33c9c0437ec915b26138ad34f730):具体算子内核已全部移到 src/native/cuda/qy/ops/<算子>/;cuda/qy 根目录仅保留设备基础设施(blas/runtime/caster 等)。

@Ceng23333

Copy link
Copy Markdown
Contributor Author

已删除(tip fae5eedf26de33c9c0437ec915b26138ad34f730):include/ 下未使用的 launcher 头文件已移除;后续若需要再放到调用算子目录下。

@Ceng23333

Copy link
Copy Markdown
Contributor Author

评审意见处理说明(tip fae5eedf26de33c9c0437ec915b26138ad34f730)

已处理 baominghelly 全部评审项:

  1. build.sh:已加入 qy(usage / case / -DWITH_QY)
  2. QY_ROOT:CMake 尊重 -DQY_ROOT / cache,不再 wipe+FORCE
  3. mul_scalar:half/bf16 在 float 中乘再转回
  4. rope_pack:向量步进改为 8/sizeof(T)
  5. 目录:算子文件迁至 ops/<算子>/
  6. include launcher:未使用头文件已删除

本机验证:QY 镜像下 InfiniRT+InfiniOps(m0 add/gemm)编译通过,import infini.ops 成功;因离线网络未能安装 pytest,完整 test_add/test_gemm 套件未在本轮重跑。

@Ceng23333
Ceng23333 force-pushed the feat/qy-backend-20261112 branch from fae5eed to f5ca75b Compare November 18, 2026 01:17

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants