Skip to content

feat(mediaplayer): copy the video into a RenderTexture - #1116

Merged
dooly123 merged 1 commit into
BasisVR:experimental-branch-major-changesfrom
towneh:feat/mediaplayer-texture-output
Oct 31, 2026
Merged

dooly123 merged 1 commit into
BasisVR:experimental-branch-major-changesfrom
towneh:feat/mediaplayer-texture-output

Conversation

@towneh

@towneh towneh commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

This adds BasisVideoTextureOutput, which copies a media player's video into a RenderTexture for anything that samples a texture rather than drawing a screen.

BasisVideoMaterialOutput binds the video to a renderer's material and BasisVideoDisplay to a RawImage, but neither gives you a RenderTexture asset you can hand to something else, such as a CustomRenderTexture's input, a lighting system's colour source or a material shared by many renderers. This component fills that gap:

  • It copies only when the player presents a new frame, so a paused player costs nothing.
  • The copy fills the target whatever its size, handles the platform's frame origin, and takes the same flip options as the material output (the flip logic is now shared through BasisVideoOutputMath.ApplyFlips).
  • While there's no video, the target holds ClearColor (black by default). A target assigned, re-created or released while the output runs is filled straight away, with the current frame or ClearColor.

It's added to the scene and prop content police lists and the Cilbox prop whitelist, beside the material output, so worlds and props can use it. The README, TESTING.md and the player's class doc cover it.

Required checks

All boxes below must be ticked before this PR can merge. If a check is genuinely N/A, tick it anyway and explain under Notes.

  • Tested — I built and ran this locally. The change works in the editor and (where relevant) in a built player.
  • Transform access is combined and limited — In hot paths, transform reads/writes go through TransformAccessArray or are otherwise batched. I have not added per-frame transform.position / transform.rotation / transform.localPosition calls inside loops. Whenever I need both position and rotation, I use the combined APIs — SetPositionAndRotation / SetLocalPositionAndRotation for writes, GetPositionAndRotation / GetLocalPositionAndRotation for reads — instead of two separate property accesses; the combined call does one local-to-world matrix traversal instead of two.
  • Addressables used for asset/memory loading — Any new asset loads go through Addressables. No new Resources.Load, no direct asset references that pull large content into memory on scene load.
  • No new GetComponent / AddComponent where avoidable — Where unavoidable, the result is cached on a field, and any GetComponent<T> is replaced with TryGetComponent<T>(out var x) — bare GetComponent will be denied. TryGetComponent is the modern API (Unity 2019.2+) and skips the Editor-only GC allocation GetComponent causes when a component is missing: Unity wraps the null return in a managed "fake null" object so its overloaded == operator can still detect destroyed C++ objects, and constructing that wrapper allocates; TryGetComponent returns a bool plus out parameter and never builds the wrapper. None of these calls run inside Update, LateUpdate, FixedUpdate, jobs, or other per-frame code paths.
  • Per-frame work is scheduled through BasisEventDriver — Any new per-frame work hooks into BasisEventDriver rather than adding standalone Update / LateUpdate / FixedUpdate callbacks on a MonoBehaviour.
  • Anything added to BasisEventDriver is bulletproof, or guarded by try/catch — BasisEventDriver runs the single per-frame tick that drives the whole framework (network apply, local player sim, blendshapes, JigglePhysics, nameplates, and more) as one sequential chain. An unhandled exception anywhere in that chain aborts the rest of the tick, so every step after the throwing one is silently skipped for that frame. New work added to the driver must either be guaranteed not to throw, or be wrapped in a try/catch that contains the failure and surfaces it through BasisDebug — logged once / rate-limited, never every frame (see the existing HVRBasisBuiltInAddresses.Simulate() guard for the pattern). Expect this to be scrutinized closely in review.
  • Considered jobification — I asked whether this work can be moved to a Unity Job (Burst-compiled where possible). If it can, it is. If it cannot, the reason is in Notes.
  • No needless { get; set; } properties or access lockdowns — Public fields are fine; Basis is meant to be read and modified freely, so don't wall things off private/internal without a real reason. Don't wrap a field in { get; set; } when the accessors do nothing — property accessors have a real performance cost vs direct field access, and the lead maintainer prefers plain fields (or a method / setter-only property when only the setter needs logic) over a noop-getter pair. For .Instance singletons, callers reassigning Type.Instance is allowed; if that would break your code, log a warning or throw — don't block the assignment. Locking down access is not your call.
  • Camera access goes through BasisLocalCameraDriver — Code that needs the local camera (transform, projection, rig data, etc.) pulls it from BasisLocalCameraDriver rather than looking one up itself. Don't roll a separate camera discovery path.
  • Logging uses BasisDebug — All new logging calls go through BasisDebug.Log / BasisDebug.LogWarning / BasisDebug.LogError (with an appropriate LogTag) instead of UnityEngine.Debug.Log / Debug.LogWarning / Debug.LogError. BasisDebug routes through Basis's tagged, color-coded logger and respects the project-wide LoggingDisabled toggle so logging can be killed at runtime; bare Debug.Log calls bypass that and will be denied.
  • No scene-wide discovery for dependencies — New code is architected so it does not need FindObjectOfType / FindObjectsOfType / GameObject.Find / FindGameObjectsWithTag to locate what it depends on. References are wired in — registered through an existing manager/driver, injected at init, or passed in by the caller — rather than discovered by scanning the scene at runtime. If a scene scan is genuinely unavoidable, justify it under Notes.
  • No allocations in hot paths — Per-frame code (Update / LateUpdate / FixedUpdate, simulation loops, jobs, anything called once per frame or more) does not allocate. No new on reference types, no LINQ, no string concatenation/interpolation, no boxing, no foreach over interface-typed collections. Allocate once at init and reuse the buffer.
  • No debugging in hot paths — No log calls of any kind on per-frame paths, including BasisDebug. Hot-path logging floods the console and incurs cost on every frame regardless of whether the message is filtered out downstream. If a hot-path log is needed while iterating, gate it behind #if UNITY_EDITOR and remove (or leave gated) before merge.
  • Hot-path collection access is optimized — Cache .Count (lists) / .Length (arrays) into a local int before the loop instead of re-reading the property each iteration. Prefer T[] (with a separate length int when the array is over-sized) over List<T> where the data is hot — Unity's mono BCL doesn't expose CollectionsMarshal.AsSpan(List<T>), so a list can't be fed into Span<T> / unsafe paths cleanly. Where the perf justifies it, drop into Span<T> / ref locals / Unsafe.As / unsafe pointer code to skip bounds checks and copies, and call out the invariants you're relying on under Notes so reviewers can sanity-check them.

Testing details

Tick the platforms you actually tested on. Leave the rest unticked — these are informational and do not block merge.

  • Windows
  • Linux
  • Android
  • iOS
  • macOS

Input / control mode coverage:

  • Tested in VR (note headset under Notes)
  • Tested in desktop / non-VR mode
  • Tested with phone controls (mobile touch input)
  • N/A — change does not touch player/XR/input code

Where applicable, confirm these flows still work after your changes:

  • Hot-switching (desktop ↔ VR mode swap at runtime)
  • Avatar swapping
  • Server swapping (joining / leaving / changing servers)
  • N/A — change does not touch any of the above

Notes

Managed C# plus the two content police assets and docs, with no plugin restage.

The per-frame work runs in the player's existing tick as an IBasisMediaTickConsumer at the Output stage, after the poll has queued the frame's present, rather than in its own Update. It doesn't allocate or log there. GetComponentInParent runs only in Reset and OnEnable, when the Player field is empty, as the material output already does.

In a Linear project the copy and the clear set GL.sRGBWrite from the target's sRGB flag and restore it after, as Graphics.Blit's docs recommend, and ClearColor is converted to linear since GL.Clear takes it as stored.

Tested in the Editor on 6000.7.0b6, on Direct3D 11 and Direct3D 12, with an on-demand H.264 MP4 into a 320x180 sRGB target in a Linear project: a grey ClearColor reads back exactly before the first frame, the picture is upright while playing, a pause holds the same frame, a new RenderTexture assigned while paused holds that frame at once, and the clear colour returns after Stop. I've also wired it into a world as a VRSL-URP light manager's sampling texture, and the colour-sampled fixtures follow the video.

BasisVideoTextureOutput copies the player's video into a RenderTexture each
frame the player presents one, for anything that samples a texture rather than
drawing a screen: a CustomRenderTexture's input, a lighting system's colour
source, a material many renderers share. BasisVideoMaterialOutput only binds to
a renderer's material, and BasisVideoDisplay to a RawImage.

It runs in the player's tick at the Output stage, after the poll has queued the
frame's present, and copies only when the presented frame count moves, so a
paused player costs nothing. The copy fills the target and restores the active
RenderTexture; the platform's frame origin is handled, and the material
output's flips apply through BasisVideoOutputMath.ApplyFlips, which both
outputs now share. While there is no video the target holds ClearColor,
converted to linear in a Linear project since GL.Clear takes it as stored. The
copy and the clear both set GL.sRGBWrite from the target's sRGB flag and
restore it after, so state left by earlier rendering doesn't change what is
written. A target assigned, re-created or released while the output runs is
filled at once, with the current frame or ClearColor.

The component is on the scene and prop content police lists and the Cilbox
prop whitelist, beside BasisVideoMaterialOutput, so worlds and props can use
it. The player's class doc, the README and TESTING.md cover it; a
CustomRenderTexture target must not have a material, since its update would
draw over the copy.

Tested in the Editor on Direct3D 11 and Direct3D 12 with a public on-demand
video into a 320x180 sRGB target in a Linear project: a grey ClearColor reads
back exactly before the first frame, the picture is upright while playing, a
pause holds the same frame, a new RenderTexture assigned while paused holds
that frame at once, and the clear colour returns after Stop. Also wired into a
world as a VRSL-URP light manager's sampling texture, where the colour-sampled
fixtures follow the video. Not yet run on Android.
@towneh
towneh requested a review from dooly123 October 30, 2026 19:15
@towneh towneh added the enhancement New feature or request label Oct 30, 2026
@dooly123
dooly123 merged commit d508874 into BasisVR:experimental-branch-major-changes Oct 31, 2026
13 of 15 checks passed
@towneh
towneh deleted the feat/mediaplayer-texture-output branch October 31, 2026 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants