Skip to content

feat(telemetry): measure every tick and tick section per phase - #317

Open
TheMeinerLP wants to merge 1 commit into
fix/telemetry-tick-sectionsfrom
feat/telemetry-tick-metrics
Open

TheMeinerLP wants to merge 1 commit into
fix/telemetry-tick-sectionsfrom
feat/telemetry-tick-metrics

Conversation

@TheMeinerLP

Copy link
Copy Markdown
Contributor

Proposed changes

Slow tick spans only cover the tail. This records OpenTelemetry metrics for every server tick and every measured section, per phase, so per-phase and sub-tick panels can use Mimir.

Depends on #316. This branch is based on it (it reuses the extended TickSections). Merge #316 first, then rebase this onto main.

Change.

  • New TickMetrics, built once in Cygnus from GlobalOpenTelemetry (meter net.onelitefeather.cygnus) and injected into SlowTickTracer, like the tracer.
  • cygnus.tick.duration (histogram, ms, cygnus.phase.name) for every tick.
  • cygnus.tick.section.duration (histogram, ms, cygnus.tick.section.name + cygnus.phase.name) for each section in each tick it ran.
  • cygnus.tick.slow (counter, {tick}, cygnus.phase.name) for ticks at or above the threshold.
  • Explicit buckets 0.1 to 200 ms. Unit is ms, not s, because the budget, threshold and span attributes are in ms; documented.
  • RoundTracer.phaseLabel() is a volatile precomputed label (lobby|waiting|gamephase|restart|none). Attributes are cached per phase and per section/phase, values recorded as primitives.
  • docs/telemetry.md: metrics section with the Prometheus names after OTLP translation (cygnus_tick_duration_milliseconds_bucket, cygnus_tick_slow_total, ...), checked against the existing jvm_* series in Mimir, and a note for the cygnus-traces dashboard.

Knock-on effects.

  • The agent exports these over OTLP with its other metrics; without it they are no-ops.
  • Series count is bounded: 5 phases x 15 sections for sections, 5 for ticks and the counter.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING.md
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Further comments

Test plan. TickMetricsTest (in-memory SdkMeterProvider + InMemoryMetricReader, not registered globally): every tick recorded, right phase, sections per phase and only when run, no leak into the next tick, slow counter only at or above the threshold, unit and buckets, cached Attributes reused (same instance).

  • ./gradlew :game:test :common:test is green.

Record cygnus.tick.duration, cygnus.tick.section.duration and cygnus.tick.slow as OpenTelemetry metrics for every tick, labelled with the running phase.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant