Repository navigation
feat(telemetry): measure every tick and tick section per phase - #317
Open
TheMeinerLP wants to merge 1 commit into
Open
TheMeinerLP wants to merge 1 commit into
TheMeinerLP wants to merge 1 commit into
Conversation
Record cygnus.tick.duration, cygnus.tick.section.duration and cygnus.tick.slow as OpenTelemetry metrics for every tick, labelled with the running phase.
5 of 8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Slow tick spans only cover the tail. This records OpenTelemetry metrics for every server tick and every measured section, per phase, so per-phase and sub-tick panels can use Mimir.
Depends on #316. This branch is based on it (it reuses the extended
TickSections). Merge #316 first, then rebase this onto main.Change.
TickMetrics, built once inCygnusfromGlobalOpenTelemetry(meternet.onelitefeather.cygnus) and injected intoSlowTickTracer, like the tracer.cygnus.tick.duration(histogram,ms,cygnus.phase.name) for every tick.cygnus.tick.section.duration(histogram,ms,cygnus.tick.section.name+cygnus.phase.name) for each section in each tick it ran.cygnus.tick.slow(counter,{tick},cygnus.phase.name) for ticks at or above the threshold.ms, nots, because the budget, threshold and span attributes are in ms; documented.RoundTracer.phaseLabel()is a volatile precomputed label (lobby|waiting|gamephase|restart|none). Attributes are cached per phase and per section/phase, values recorded as primitives.docs/telemetry.md: metrics section with the Prometheus names after OTLP translation (cygnus_tick_duration_milliseconds_bucket,cygnus_tick_slow_total, ...), checked against the existingjvm_*series in Mimir, and a note for thecygnus-tracesdashboard.Knock-on effects.
Types of changes
Checklist
Further comments
Test plan.
TickMetricsTest(in-memorySdkMeterProvider+InMemoryMetricReader, not registered globally): every tick recorded, right phase, sections per phase and only when run, no leak into the next tick, slow counter only at or above the threshold, unit and buckets, cachedAttributesreused (same instance)../gradlew :game:test :common:testis green.