Skip to content

Split the dashboard's Environments table and history by service, with docs and /status links and more status detail, and fit every page on a phone - #160

Merged
gaurav merged 30 commits into
mainfrom
environments-table
Nov 11, 2026
Merged

gaurav merged 30 commits into
mainfrom
environments-table

Conversation

@gaurav

@gaurav gaurav commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

The dashboard's Environments table mixed NodeNorm and NameRes, and its one Babel version row, like the cards above it and the History page, was NodeNorm's. The two services are loaded separately and can be on different releases: today test has NodeNorm on 2025sep1 and NameRes on 2026jul22, and the site didn't show it. This PR gives each service its own section, with its own Babel version, Biolink model, version and backend, a link to each environment's Swagger UI, its Status values linked to the full /status, and a few more /status fields worth watching.

A card shows one unlabelled Babel release when both services report the same one, or when only NodeNorm reports one; otherwise it labels each service's (NN/NR), so NameRes's release is never read as the environment's while NodeNorm is down. The history now records NameRes's Babel release alongside NodeNorm's, and its list of changes names the service each version belongs to. Closes #161.

Every page now also fits a phone and suits dark mode, and two existing dashboard bugs are fixed: History announced ci-es, dropped weeks ago, as "no longer reported" on every run, and an environment reporting nothing was shaded as an odd one out.

The calls worth overruling

  • Docs links follow the configured backend. Redis and Solr serve Swagger UI at /docs; the ES services serve it at their root and answer /docs with a 404 (checked on 2026-11-11). The generator builds both URLs from targets.ini, so the website links only to checked-in URLs. Backend rows show the configured value; NameRes /status has none.
  • NameRes's index stats go into one index key, wherever they arrive: under solr on v1.5 (prod), at the top level on v1.7 and NameRes ES, so the old report had no document count for four of six targets. history.jsonl's solr_docs/solr_size fill in from the first run after this merges; a comment in build_history_line says a null-to-number step there is missing data, not a change.
  • NodeNorm and NameRes fields are keyed on the service, not on the response's shape. NameRes v1.7's top-level version is its index's counter (68618), not a release, and an error body need not carry the fields that would give that away. So version is kept for NodeNorm only, and index is read for NameRes only.
  • Neither Version row is shaded for disagreement: Redis and ES NodeNorm, like Solr and ES NameRes, are separate codebases.
  • History keeps babel_version as NodeNorm's and adds nameres_babel_version, since a rename would blank the Babel version of every earlier run in history.jsonl. A field an older run didn't record isn't a change.
  • Rows no environment reports are hidden, but a cluster reporting zero nodes isn't: its cell says "no nodes", shaded red.
  • In the Environments table only, ci-nameres-es sits beside ci, whose NodeNorm it shares. Every other view keeps it after prod, where Put ci-nameres-es after prod on the dashboard, and test NameRes /synonyms alongside /reverse_lookup #157 put it so it isn't read as a promotion. The parent comes from a new SideBranchOf = ci in targets.ini, not a map in the website; it is stated rather than derived from the shared URL, because two sections sharing a URL don't say which is the branch. A unit test fails if two sections share a service URL and neither names the other. Until the dashboard regenerates, the published report.json has no side_branch_of, and ci-nameres-es sorts last there too.
  • On a phone, the Results table scrolls sideways inside its card and its header stops sticking. theme.css asked for dropping columns instead, but six environment columns alone are about 410px. Below 768px only; the filter bar still sticks; see Re-evaluate the Results table on phones: it scrolls sideways and loses its sticky header #166.

What's here

  • generate_report.py additionally whitelists the service's version, the index's lastModified/startTime, and a summary of NodeNorm ES's Elasticsearch nodes: how many, their distinct versions (ordered as numbers, newest five kept), and the fullest disk. It also keeps the Babel releases a NodeNorm ES from 1.2.0 can serve (babel.available_versions, an object or a list, each name validated as a release, newest 50 kept; any other shape logs a warning rather than hiding the row). The table shows the newest three, the rest in a tooltip. It also times each /status request. All of it goes through the existing sanitize/_trimmed_* helpers.
  • STATUS_SECTIONS in reportData.js replaces STATUS_ROWS; a row may now give a danger shading and a title tooltip. StatusMatrix.vue renders a section per service, then the test counts. Shaded cells use theme-aware .cell-differs/.cell-broken, since Bootstrap's .table-warning/.table-danger were near-white in dark mode.

What it deliberately doesn't do

Verification

On 018b4c7, the unit suite (290-odd tests) and the website tests (60-odd) pass and npm run build succeeds; the generator run against live /status gave ci-nameres-es its parent, listed ci's two served releases and prod's ES nodes, and gave no NodeNorm an index. On 06c5ad5, with the deployed report.json in Chromium, no page is wider than the viewport at 360, 390, 768 or 1280px, and the Results header sticks from 768px. On bbdfb0d, a site built from live /status, a short live run and the deployed history showed test's NN/NR labels, ci-nameres-es's unlabelled release, prod's ES nodes and disk, and no false History changes; a zero-node cluster and a twenty-release list were checked with a substituted report.

Nothing is blocking.

🤖 Generated with Claude Code

gaurav and others added 3 commits November 9, 2026 17:36
The report kept one /status shape for both services and nothing to link
to. The generator now also whitelists the service's own version, the Solr
index's lastModified and startTime, NameRes ES's top-level numDocs and size
(as 'index', since it isn't Solr), and a summary of NodeNorm ES's
Elasticsearch nodes: how many, their version, and the fullest disk. It
records how long each /status took to answer.

Each target section also gains the configured backend for each service and
its docs and /status URLs, built from targets.ini. The Elasticsearch
services serve Swagger UI at their root and answer /docs with a 404, so
their docs link is the base URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NameRes v1.7 moved its Solr core's stats from under 'solr' to the top level
of /status, so test, ci, dev and exp were reporting no document count, and
the history line's solr_docs/solr_size had been null for them. The
top-level block also carries the index's own 'version' counter (68618),
which the previous commit would have taken for the service's version.

The index stats now go into one 'index' key whether they came from 'solr'
(prod, NameRes v1.5), the top level (v1.7) or NameRes ES; 'version' is kept
only when there is no top-level numDocs; and the history line reads 'index'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NodeNorm and NameRes are loaded separately and can be on different Babel
releases: test had NodeNorm on 2025sep1 and NameRes on 2026jul22, but the
table's one Babel row, and the environment cards, read only NodeNorm.

Each service now has its own section: a header row of docs and /status
links per environment, then status, Babel version, Biolink model, version,
backend, and its own index and health rows (Redis records and memory,
Elasticsearch nodes and disk, NameRes documents, index size and dates,
latency, /status response time). A row no environment has a value for is
hidden. The test counts become a third section. A card shows both Babel
versions, a line each, when the two services disagree.

NodeNorm's Version row is not shaded for disagreement: Redis 2.x and ES 1.x
are separate codebases, and comparing them shaded every environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gaurav and others added 7 commits November 9, 2026 18:00
NodeNorm ES from 1.2.0 can serve several Babel releases, chosen per
request, and lists them under babel.available_versions in /status: on
2026-11-09 ci served 2025sep1 and 2026jul22 while test served only
2025sep1. The generator keeps the names that are real Babel releases,
oldest first, and the NodeNorm section gains a 'Babel releases served' row,
hidden while no environment reports one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The environment card fell back to NameRes's release, unlabelled, when
NodeNorm reported none (NodeNorm down, say), so it read as the whole
environment's release. Now a single unlabelled version appears only when
both services report the same one; otherwise each reported release is
prefixed NN or NR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NameRes Solr and NameRes ES are separate codebases with unrelated version
numbers, the same reason NodeNorm's Version row is already not compared.
Comparing them would mark ci-nameres-es amber every day.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes to what trim_status() and fetch_status() keep from a /status:

- babel.available_versions was cut to its first 50 keys before being
  validated and sorted, so on a long or oldest-first list the current
  releases could be the ones dropped. Every key is now checked, then the
  newest 50 kept.
- The Elasticsearch summary read the version of the first node only, and
  counted at most 50 nodes, so a cluster mid rolling upgrade looked
  uniform. It now counts every node and reports up to five distinct
  versions ("versions", a list, replacing "version").
- The top-level "version" was kept whenever the response had no numDocs,
  so a NameRes error body without one would pass the Solr index's
  version counter through as the service's release. trim_status() now
  takes the service and keeps "version" for NodeNorm only.
- response_ms used requests' elapsed, which stops when the headers
  arrive. It is now timed with time.monotonic() around the request and
  the JSON parse, which is what "/status time" claims to show.
- fetch_status() took the base URL and rebuilt the /status URL with a
  dummy backend; it now takes the URL build_report() already computed,
  so the fetched and linked URLs cannot drift apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The backend defaulted to redis or solr even when targets.ini gives that
service no URL, so the always-shown Backend row named "solr" beside a
red NoURL on test. It is now None there, and the empty-row filter can
hide it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_history_line() recorded only NodeNorm's babel_version, so the
history diff never saw NameRes move to a new release on its own, which
is the case the split Environments table exists to show. It now records
nameres_babel_version too, and History.vue diffs it. A field the
previous run did not record is not reported as a change, so the first
run after this lands does not list every environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The history now records nameres_babel_version; the table shows it beside
NodeNorm's on any run where the two disagree. Runs recorded before the
field existed render as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gaurav gaurav changed the title Split the dashboard's Environments table into NodeNorm and NameRes sections, with docs and /status links and more status detail Split the dashboard's Environments table and history by service, with docs and /status links and more status detail Nov 9, 2026
gaurav and others added 5 commits November 10, 2026 13:45
A node mid-restart can report "version": null. sanitize() keeps None as
None, and sorting a set holding None and strings raised TypeError, which
fetch_status caught and turned into {"error": "TypeError"} for the whole
NodeNorm status: the card lost its Babel version, records and every
other row over one field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generator records {nodes: 0} for a cluster whose nodes have not
joined, but the Elasticsearch row's `!es?.nodes` read 0 as missing data
and showed an em dash. With only one ES environment, the row-hiding
rule then removed the whole row, so the most important failure state
vanished. The cell now says "no nodes" and is shaded red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generator keeps up to 50 release names and the Environments table's
cells don't wrap, so a NodeNorm ES serving twenty releases stretched the
table far past the viewport. The cell now shows the newest three and
"+N older"; a new `title` option on STATUS_SECTIONS rows puts the full
list in the cell's tooltip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Older NameRes Solr releases, and a NameRes that timed out, report no
babel_version, so every such card had changed from "2026sep24" to
"NN 2026sep24" though nothing differed. Only NodeNorm's release alone is
unlabelled again, as the card showed before NameRes's was read; the
tooltip says NameRes reported none. NameRes's release alone keeps its
"NR" label, so with NodeNorm down it is still not read as the
environment's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fetch_status checked `"error" not in status`, which trim_status only
sets for a body that is not a JSON object; test that directly.

Runs before this branch recorded solr_docs and solr_size as null for
NameRes v1.7+ environments, which moved the index to the top level of
/status. The first run after the deploy fills them in. Nothing compares
them across runs today, but a comment now warns that a null followed by
a number there is missing data, not a change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gaurav and others added 4 commits November 10, 2026 17:03
The History page's change list walks every environment in the whole
history file, so one dropped weeks ago (ci-es) was missing from both of
the runs being compared and was announced as "no longer reported" every
day. A target absent from both runs is now no change at all.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the Environments table's compared rows, the majority is found among
reported values, but an em dash was still shaded amber whenever the
others disagreed, and left plain when they agreed. With NameRes's own
Babel and Biolink rows, NameRes ES (which reports neither) was shaded
on every day prod lagged behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At 390px the whole page scrolled sideways: the Environments table
reached 904px (711px before NodeNorm and NameRes got a section each),
the history table 733px, and the navbar's links ran to 484px. The two
tables now scroll sideways inside their cards, and the navbar wraps its
page links onto a second line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The results table is 700-odd pixels wide at 390px: six environment
columns of outcome badges come to about 410px alone, and the Category
column already goes below lg. Dropping columns, which theme.css asked
for, can't get it onto a phone, and wrapping it in .table-responsive
would silently break its sticky header, as that comment warns. So below
md the card scrolls the table sideways and the header stops sticking;
from md up nothing changes. The filter bar is outside the card and still
sticks.

Two other things widened the page: the sources dropdown, which w-auto
sized to its longest option, a 580px URL; and the six outcome buttons,
about 365px, which a btn-group never wraps. The dropdowns are capped at
the bar's width and the outcome buttons wrap. The environment headers
no longer break at each hyphen, which tripled the header's height.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gaurav gaurav changed the title Split the dashboard's Environments table and history by service, with docs and /status links and more status detail Split the dashboard's Environments table and history by service, with docs and /status links and more status detail, and fit every page on a phone Nov 10, 2026
gaurav and others added 6 commits November 10, 2026 17:55
ci-nameres-es is ci's NodeNorm against the NameRes ES under validation.
At the end of the table its NodeNorm cells were far from the ci ones
they match, and its NameRes ones far from the Solr NameRes they are
meant to replace. A new SIDE_BRANCHES map in deploymentOrder.js names
the target each side branch comes from, and sortBesideParents() moves it
there for that table only.

Every other table keeps the pipeline order, where ci-nameres-es stays
after prod: they are read column against left neighbour, and between ci
and test it would make every Solr-vs-ES difference look like a ci→test
promotion (see #157).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Environments table shows a whitelisted part of each /status
response; clicking a service's Status value now opens the whole thing,
error responses included. With /status linked there, each service's
header row carries just an "API docs" link, to the Swagger UI the
generator located from targets.ini.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bootstrap's .table-warning and .table-danger, and .table-light on the
results table's section headings, are fixed light colours: in dark mode
they were near-white blocks in an otherwise muted table. The shaded
cells now use new .cell-differs and .cell-broken classes built on the
warning and danger subtle/emphasis pairs, which flip with the theme, as
theme.css asks; light mode looks the same as before. The "no results"
badge uses the same subtle pair, and the results headings use
.table-group, like the Environments table's section rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Docs" read as the written API documentation each service keeps in its
repository (NodeNormalization's documentation/API.md, say), which this
does not link to: it opens the service's own Swagger UI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t their root

The comment said they "answer the API itself at the root", which contradicted
service_links(), and a review of #160 took the "API" link for ES services to be
broken. On 2026-11-11 the roots of nodenorm-es and namelookup-es returned Swagger
UI and /docs returned 404, so the link was right and the comment was wrong.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sorted() compared them as strings, so 8.10.0 came before 8.9.0, and the cut to
five kept the lexically smallest versions rather than the newest ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gaurav and others added 5 commits November 11, 2026 01:19
trim_status read the list of Babel releases a NodeNorm ES serves only when it
was an object, and dropped any other shape without a word. That hides the
"Babel releases served" row, which looks exactly like a NodeNorm serving only
its default release. A plain list of names is now read the same way as the
object NodeNorm ES 1.2.0 sends; any other shape, or a list in which no name is
a Babel release, logs a warning in the dashboard workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The index block was chosen by the response's shape ("numDocs" in it), so a
NodeNorm body carrying a numDocs would have been given a NameRes index. Gate it
on the service as well, as "version" already is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…site

deploymentOrder.js kept SIDE_BRANCHES (ci-nameres-es → ci) by hand: a third
place to register a target, after targets.ini and DEPLOYMENT_ORDER, and unlike
DEPLOYMENT_ORDER nothing warned when a new side branch was missing from it. It
would just have sat at the far end of the Environments table.

The parent is now SideBranchOf in the branch's targets.ini section, beside the
shared URL that makes it a side branch. generate_report checks it names another
deployed target and emits it as each target's side_branch_of, and
sortBesideParents takes a lookup instead of the map. test_targets_ini fails when
two sections share a service URL and neither names the other.

Deriving the parent from the shared URL alone was considered and not done: two
sections sharing a URL do not say which of them is the branch.

Until the dashboard regenerates, the published report.json has no
side_branch_of, and ci-nameres-es sorts last, as it does in every other table.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The diff since the previous run labelled NodeNorm's release "babel version",
beside "nameres babel version", so a NodeNorm promotion read as the whole
environment changing release. The labels are now "NodeNorm Babel version",
"NameRes Babel version" and "NameRes version".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The card template called babelVersion(target) twice, once for its tooltip and
once for its lines, building the same object twice on every render. A computed
map gives both from one result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gaurav
gaurav marked this pull request as ready for review November 11, 2026 07:23
@gaurav
gaurav merged commit fe26db5 into main Nov 11, 2026
2 checks passed
@gaurav
gaurav deleted the environments-table branch November 11, 2026 07:23
gaurav added a commit that referenced this pull request Nov 11, 2026
main has moved on 260 commits since this branch was cut, including #159
(CLAUDE.md renamed to AGENTS.md) and #160.

Conflicts:
- CLAUDE.md: kept deleted, as on main. This branch's edits to it (ruff in
  place of black, the -m unit note, babel_validation imports) are carried
  into AGENTS.md in the commits after this merge.
- pyproject.toml: this branch's ruff dependency and settings, pythonpath and
  marker comment; main's pyyaml. black stays dropped.
- Ten source and test files: main's side. This branch changed them only by the
  src. prefix rename, ruff's lint fixes and the ruff format sweep, all
  mechanical, so they are re-applied over the whole tree in the commits after
  this one rather than hand-merged here. main already had
  deselected_by_markexpr, which this branch also used.

tests/unit/test_gsheet_parsing.py stubbed requests.get inside
google_sheet_test_cases, which main replaced with fetch_sheet_csv(): the
fixture now stubs that instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Record NodeNorm's and NameRes's Babel versions separately in the dashboard history

1 participant