[leejet]

Summary

Reduce intermediate tensor writes and separate dispatches in MOSS-Image's Vulkan execution:

• Extend weighted RMSNorm + RoPE fusion to write F16 output directly, ported from ggml #23.
• Fuse 3072-wide weighted RMSNorm, modulation and optional shift, ported from ggml #24.
• Update ggml to 135ce1e0, including native SiTU-to-F16 staging for Q8 matrix multiplication from ggml #25.

The normalization and RoPE fusions use sd.cpp's ggml extension mechanism. SiTU staging remains in ggml and reuses its matrix multiplication implementation.

Thanks to @mirek190 for the original optimizations.

Related Issue / Discussion

N/A

Additional Information

Hardware: NVIDIA GeForce RTX 4090 24 GB, NVIDIA driver 596.36.

DiT files:

Model	DiT file
MOSS-Image SFT	MOSS-Image-SFT-Q8_0.gguf

Sampling throughput in it/s; higher is better:

Model	Resolution	sd.cpp before	sd.cpp after
MOSS-Image SFT	1024×1024	1.403	1.473

The measured mean increased by 4.99%.

Configuration:

• Vulkan backend, batch 1, CFG 3, Euler, discrete scheduler, seed 42.
• 20 steps; arithmetic mean of printed per-step it/s after excluding the first two steps, then averaged across three interleaved runs per version. No additional outliers removed.
• Same Qwen3.5-9B-Q4_K_M.gguf text encoder, flux2_ae.safetensors VAE and tokenizer_qwen3.5.json.
• Prompt: a lovely cat holding a sign says 'moss-image.cpp'.

Checklist

• I have read and confirmed this PR follows the contribution guidelines.

Summary by CodeRabbit

• New Features
  • Added Vulkan acceleration for RMS normalization with supported scaling and affine operations, improving performance on compatible workloads.
  • Expanded Vulkan handling of rotary-position outputs converted to half precision, including an optional scaling step.

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

This change adds a Vulkan RMS-affine kernel and registers it with the backend. It also updates Vulkan RoPE fusion to match direct and scaled F16-copy paths, with shader output scaling enabled only for the scaled path.

Changes: Vulkan fusions

Layer / File(s)	Summary
RMS-affine kernel and integration cmake/backend_extensions.h.in, src/core/vulkan/shaders/rms_modulate.comp, src/core/vulkan/rms_affine.cpp, src/core/vulkan/CMakeLists.txt, CMakeLists.txt, ggml, src/core/layer_split_partition.cpp, src/core/vulkan/shaders/modulate.comp	Adds an RMS-affine Vulkan shader and kernel that matches supported RMS-normalization and affine patterns, prepares workspace, obtains Vulkan buffer information, dispatches the shader, and copies workspace output when needed. Adds the kernel declaration, build entries, and backend registration. The ggml reference changes, and the partition lambda and modulation shader declarations are reformatted.
RoPE fusion output matching src/core/vulkan/rope.cpp, src/core/vulkan/shaders/rms_rope_apply.comp, src/core/vulkan/shaders/modulate_convrot.comp	RoPE fusion now matches direct F16 copies and zero-parameter scale followed by F16 copy. The cast shader applies output scaling only for the scaled-copy path. ConvRot shader declarations are reformatted.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Refactor

Sequence Diagram(s)

sequenceDiagram
  participant Graph as ggml graph
  participant Kernel as rms_affine kernel
  participant Vulkan as Vulkan service
  participant Shader as rms_modulate shader
  Graph->>Kernel: Match RMS-affine pattern
  Kernel->>Vulkan: Get device and buffer information
  Kernel->>Vulkan: Dispatch sd.rms_modulate.vulkan
  Vulkan->>Shader: Execute RMS normalization and modulation
  Shader-->>Vulkan: Write output
  Vulkan-->>Kernel: Return dispatch result

Loading

Possibly related PRs

• leejet/stable-diffusion.cpp#2253: Introduced the backend-extension registration mechanism and Vulkan extension header extended here.
• leejet/stable-diffusion.cpp#2258: Adds modulation nodes and Vulkan ConvRot fusion that consumes them; this change adds a Vulkan RMS-affine path for related modulation data.
• leejet/stable-diffusion.cpp#2265: Adds CUDA RMS-affine fusion for normalization, affine operations, and modulation; this change implements a Vulkan variant.

Suggested reviewers: cumal

Merge Risk: Minimal · up to 93e39

No actionable issue remains identified in the Vulkan fusion changes; they are mergeable after normal checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title clearly summarizes the main changes: Vulkan normalization acceleration and SiTU staging for MOSS-Image.
Description check		The description includes all required sections. It explains the changes, records related discussions as N/A, provides hardware and performance verification details, and confirms the contribution check…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description includes all required sections. It explains the changes, records related discussions as N/A, provides hardware and performance verification details, and confirms the contribution checklist.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 14 functions across 3 files. (8 skipped: 8 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate docstrings

• Commit to this branch
• Create a new PR

Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

Recent review info

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Repository UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 8e5872ad-0ebf-4715-846f-18c439e924ab

Commits

Reviewing files that changed from the base of the PR and between 3e92bf9 and 93e39d7.

Files selected for processing: 11

• CMakeLists.txt
• cmake/backend_extensions.h.in
• ggml
• src/core/layer_split_partition.cpp
• src/core/vulkan/CMakeLists.txt
• src/core/vulkan/rms_affine.cpp
• src/core/vulkan/rope.cpp
• src/core/vulkan/shaders/modulate.comp
• src/core/vulkan/shaders/modulate_convrot.comp
• src/core/vulkan/shaders/rms_modulate.comp
• src/core/vulkan/shaders/rms_rope_apply.comp

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.