Skip to content

feat: expose ARS querying on the MCP and CLI surfaces - #66

Open
SkyeAv wants to merge 21 commits into
mainfrom
query-ars-mcp
Open

SkyeAv wants to merge 21 commits into
mainfrom
query-ars-mcp

Conversation

@SkyeAv

@SkyeAv SkyeAv commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Third of three stacked PRs for ARS querying. Puts @gloriachin's ARS finder flow in front of agents on the MCP and CLI surfaces. It also adds query_ars, which runs the whole flow as an MCP background task (SEP-1686).

Stacked on #65.

What changed

  • Five shared tools in TCT/interfaces/tools.py, available on both MCP and CLI (tct ARS-neighborhood-finder, tct submit-ars-query, and so on):
    • ARS_neighborhood_finder and ARS_pathfinder: the blocking finders from Guangrong's branch, including json_file passthrough of uploaded TRAPI queries. They return resolved inputs plus the top 20 ranked rows, or the full merged message for uploads.
    • submit_ars_query(node, neighbor_categories, predicates) returns a pk immediately. get_ars_status(pk) reports parent status, the merged pk, and per-ARA status. get_ars_results(pk, top_n=20) returns ranked rows, or the full merged TRAPI message with top_n=0.
  • query_ars, MCP-only, in TCT/interfaces/ars_task.py, registered with task=TaskConfig(mode="optional"). Task-aware clients get a task ID immediately and poll; task-unaware clients get an ordinary blocking call (SEP-1686 graceful degradation). It reports progress in two steps (ARS finished, merged answer fetched). The polling wait and merged-message fetch run in a worker thread.
  • query_ars lives in its own module because mcp.py uses PEP 563 string annotations, and FastMCP could not resolve the Progress dependency through them. Plain calls failed until the move.
  • None of the ARS submitters carries readOnlyHint: submit_ars_query, ARS_neighborhood_finder, ARS_pathfinder, and query_ars each create an ARS submission. Status and results lookups stay read-only.
  • TCT/server.py re-exports the six new names. The shared registry grows from 16 to 21 tools; MCP lists 22 with query_ars.
  • FastMCP 2.14.7 test-transport quirk: with the in-memory transport, once a server instance has run a task call, a second client session on that instance fails. The same session keeps working, and a real HTTP server handled task and plain calls across five separate sessions, so production is unaffected. The in-process tests reload the adapter to get a fresh server per session.

Stats (for audit)

  • 19 commits, 23 files, +2731/-14.
  • Offline suite: 171 passed, 7 skipped (157 before this PR; +14 in tests/test_ars_tools.py).
  • New offline coverage in tests/test_ars_tools.py: tool shaping (TRAPI construction, predicate forwarding, ranked rows, ready flags, json_file passthrough), plus the MCP surface through an in-process FastMCP client (submit/poll/fetch, task flow, graceful degradation, task metadata).
  • Contract tests extended with the new names, signatures, input schemas, and annotation sets. The unstructured-output contract still holds for all 22 tools.
  • Live tests: 3 to 6. Added the submit, poll, fetch flow; query_ars as a task; and query_ars through a task-unaware client.

Verification

  • uv run pytest: 171 passed, 7 skipped.
  • Live run 1: TCT_LIVE_ARS=1 uv run pytest tests/test_ars_live.py -v: 6 passed in 846.56 s.
  • Live run 2, same command: 6 passed in 760.28 s.
  • Performance figures for the underlying client are in feat: ARS neighborhood finder and pathfinder #65.

@Santoshkumarpuppala

Copy link
Copy Markdown

8fe83ee takes submit_ars_query out of the read-only set because "submitting a query creates ARS state", but three tools that stay in _READ_ONLY_TOOLS (TCT/interfaces/mcp.py:30-37) make the same submission. ARS_neighborhood_finder and ARS_pathfinder call submit_ARS (TCT/Query_ARS.py:701, :767), which does the POST {ars}/submit at :294, and query_ars calls it too (TCT/interfaces/ars_task.py:58) while hardcoding readOnlyHint: True (:92). tests/test_server.py:189-196 pins that split.

The MCP spec defines readOnlyHint as "the tool does not modify its environment". It's a hint, and the spec says clients should never base tool use decisions on annotations from untrusted servers, but a client that skips confirmation for read-only tools from a server it does trust would prompt for submit_ars_query and not for the other three. So either those three drop the hint alongside submit_ars_query (for query_ars that's ars_task.py:92, not the set), or the comment at mcp.py:34 and the test should say why their submission differs from submit_ars_query's. Read at 8fe83ee, not run.

@SkyeAv

SkyeAv commented Oct 5, 2026 •

Copy link
Copy Markdown
Member Author

@Santoshkumarpuppala Good catch, all four ARS submitters now carry readOnlyHint: False on query-ars-mcp (commit 94eefb7), thanks for the detailed review.

@kennethbruskiewicz kennethbruskiewicz left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The module reads clearly, the error types are useful, and gating the live tests behind TCT_LIVE_ARS keeps CI from loading the ARS.

Two things look like bugs to me, and I've left inline comments on both. get_ars_results can report ready: true with zero results while the merge is still being saved, and ARS_pathfinder accepts predicates and attribute_constraints without using them. The other inline comments are smaller.

On landing it: the base branch is query-ars and #65 was closed without merging, so this needs retargeting to main. Now that #64 has landed it also conflicts with main in TCT/interfaces/mcp.py and tests/test_server.py. It will overlap with #68 in tools.py and mcp.py as well, and I'm happy to coordinate on the order.

One question on the tool surface. The PR adds six tools, and three of them run a one-hop ARS query in different ways: ARS_neighborhood_finder, the submit_ars_query / get_ars_status / get_ars_results trio, and query_ars. That gives an agent three choices for the same task. Would it be enough to expose the trio plus query_ars, and keep the two blocking finders from Guangrong's branch as Python-only functions? Their ARS_* names also differ from the snake_case of the other tools and become ARS-neighborhood-finder on the CLI.

I read the code at 94eefb7 and ran the offline tests and the two reproductions noted inline. I did not run the live ARS tests.

Comment thread TCT/interfaces/tools.py Outdated
of retrying this.
"""
status = _get_ARS_status(pk)
if not status.is_terminal:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module docstring describes the race in NCATSTranslator/Relay#621, where the parent reads Done a few seconds before the merge agent has saved, and wait_for_ARS waits for both. This function checks only status.is_terminal. In that window it fetches the parent's own payload and returns ready: true with result_count: 0.

I reproduced it with a trace where the parent is Done, the ars-ars-agent child is Running and merged_version is empty. The result was {'ready': True, 'result_count': 0, 'merged_pk': None}.

The docstring tells callers to stop polling once ready is true, so an agent would report no answers for a query that has them. Returning ready: false until status.merged_ready, with the same grace period wait_for_ARS uses for queries that never produce a merge, would close it. get_ARS_result in Query_ARS.py (line 491) has the same check.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 790fbdb. ARSStatus gained a merge_pending property: parent terminal, no merged version, and the ars-ars-agent child still non-terminal. get_ars_results returns ready: false in that window, and get_ARS_result raises ARSPendingError instead of fetching the parent payload. Since get_ars_results is a stateless one-shot poll I used the deterministic merge-child check rather than a wall-clock grace period; a Done parent whose merge child is terminal (or absent) still falls back to the parent payload as the final answer. Added regressions reproducing the exact trace from your repro.

Comment thread TCT/Query_ARS.py Outdated
intermediate_categories: CategoryList | None = None,
predicates: list[str] | None = None,
json_file: dict[str, Any] | None = None,
attribute_constraints: list[dict[str, Any]] | None = None,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

predicates (line 718) and attribute_constraints are in the signature and are never passed to format_query_json_forARS_pathfinder. A caller who sets them gets an unfiltered query with no warning. They could be wired through or dropped.

start and end are also required positionally when json_file is given, which ARS_neighborhood_finder avoids by making node optional.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped both in 790fbdb - build_query_graph accepts neither and no caller passed them. start/end now default to None and are only required when json_file is absent, matching ARS_neighborhood_finder.

Comment thread TCT/interfaces/ars_task.py Outdated
object_categories=shared_tools._normalize_categories(neighbor_categories),
predicates=predicates,
)
pk = ars.submit_ARS(query)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_resolve_nodes (line 52) and submit_ARS run directly in the coroutine. Both make HTTP calls with timeouts of up to 120 seconds, and they block the event loop for other MCP requests while they run. Only the wait and the fetch go through asyncio.to_thread. Moving these two calls there as well would match the rest of the function.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both now go through asyncio.to_thread in 790fbdb, matching the wait and fetch. Added a test asserting resolution and submission run off the event loop thread.

Comment thread TCT/Query_ARS.py
# ---------------------------------------------------------------------------
# submit / poll / fetch
# ---------------------------------------------------------------------------
def submit_ARS(query_json: dict[str, Any], *, url: str | None = None) -> str:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Queries built by the format helpers carry "submitter": "TCT". A json_file query is submitted unchanged, so it may carry none. There was a request in #engineering-committee on 2026-10-22 that agent-driven requests include a submitter so they can be traced. Setting it when it's absent would cover uploaded queries too.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

submit_ARS now annotates submitter: "TCT" when absent, on a copy so the caller's dict is not mutated and an explicit submitter is preserved (790fbdb).

Comment thread TCT/translator_metakg.py
import requests
CI_URL_PREFIX = "https://multiomics.ci.transltr.io/"
DEV_URL_PREFIX = "https://multiomics.rtx.ai:9990/"
TEST_URL_PREFIX = "https://multiomics.test.transltr.io/"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a third fallback, from CI to dev to the test environment, for five CATRAX KPs. With the environment set to CI, the provider list can then hold test-environment query URLs without the caller knowing.

A run I recorded on 2026-11-16 showed the same effect from the existing fallbacks. With environment="ci", one question reached production, dev and RENCI hosts. I'd like to raise the fallback policy as its own ticket. For this PR, a warning that names the environment actually used would help.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in f7daf2a: a warning naming the environment actually used and its query URL prefix, on both the dev and test fallback paths for the five CATRAX KPs. Agreed on raising the fallback policy as its own ticket.

Comment thread tests/test_query_ars.py
# --------------------------------------------------------------------------- #
# query construction
# --------------------------------------------------------------------------- #
def test_one_hop_input_matches_notebook_query(monkeypatch):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test, and test_ars_neighborhood_finder_shapes_finder_result and test_ars_pathfinder_shapes_finder_result in tests/test_ars_tools.py, call Name Resolver and Node Normalizer. All three failed on my machine while it was offline. The other ARS tests are mocked or gated behind TCT_LIVE_ARS. Mocking _resolve_nodes in these three would make the offline suite fully offline.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three now mock TCT.Query_ARS._resolve_nodes - the test_ars_tools patches targeted tools._resolve_nodes, which the finder tools never consult because resolution happens inside Query_ARS. The ARS suites pass under a blackhole HTTP proxy.

gloriachin and others added 21 commits November 17, 2026 13:17
The node_normalizer CI hostname returns 404; resolve every environment to
the production URL unless an explicit override is set.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
Carry the status code, URL, and response into the raised requests.HTTPError
so callers see what failed instead of a bare exception.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
When the CI and dev multiomics URLs 404, try the test deployment before
warning, so the Plover API registry still loads in restricted environments.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
Reshape TCT.Query_ARS around Guangrong Qin's feat/ars-query-support module:
shared requests.Session transport, deadline-based merge-aware polling
(Relay#621), frozen ARSStatus with an O(1) child index, and an ARSResult that
extends FinderResult so both finders behave like the other TCT finders.
format_query_json_forARS_pathfinder reuses the shared pathfinder query-graph
builder.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
Co-authored-by: Frankie Hodges <hodgesf@oregonstate.edu>
Co-authored-by: Frankie Hodges <129905251+hodgesf@users.noreply.github.com>
Re-executed Test_ARS.ipynb end to end (neighborhood, raw submit/poll flow,
and the new pathfinder cells) against the CI ARS.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
… URL change

Guangrong's nodenorm fix was written against an older base that predated the
constant; keep his endpoint change but preserve the constant.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
Ported from Guangrong Qin's feat/ars-query-support tool layer, reshaped onto
the ARSResult API: ranked summary rows for built queries, the full merged
message for uploaded json_file queries.

Co-authored-by: Guangrong Qin <2994757+gloriachin@users.noreply.github.com>
Async wrapper over the ARS finder flow with per-poll progress reporting.
Task-aware clients get a task ID immediately; task-unaware clients get a
plain blocking call per SEP-1686 graceful degradation.
ARS_neighborhood_finder, ARS_pathfinder, submit_ars_query, get_ars_status,
get_ars_results, and the query_ars background task join the server namespace.
FastMCP resolves the Progress dependency from real annotations, and PEP 563
string annotations in the adapter module defeat that resolution, which broke
task-unaware clients. The task tool now lives in a module without the future
import; the adapter registers it after the shared tools.
Tool shaping for the finder pair and the submit/status/results trio, plus the
MCP surface: SEP-1686 task flow (task ID, status, ranked results) and graceful
degradation for task-unaware clients.
Covers the sync trio (submit, poll, fetch) and query_ars through an
in-process MCP client both as a background task and as a blocking call.
The finder and submit tools now return plain status strings, and
submit_ars_query is marked non-read-only: submitting a query creates ARS
state, so readOnlyHint was wrong. The polling and fetch tools keep it.
ARS_neighborhood_finder, ARS_pathfinder, and query_ars each create an
ARS submission through submit_ARS, so readOnlyHint is now False for
them as it already was for submit_ars_query. Removes the inert
query_ars entry from _READ_ONLY_TOOLS and updates the annotation pins.
- add ARSStatus.merge_pending for the NCATSTranslator/Relay#621 window
  where the parent reads Done before the merge agent has saved
- get_ars_results reports ready: false while merge_pending;
  get_ARS_result raises ARSPendingError instead of fetching the parent
- drop unused predicates/attribute_constraints from ARS_pathfinder and
  make start/end optional when json_file is given
- submit_ARS annotates a missing submitter as TCT on a copy, per the
  2026-10-22 #engineering-committee tracing request
- run node resolution and submission off the event loop in query_ars
- mock Query_ARS._resolve_nodes in the three tests that still reached
  Name Resolver / Node Normalizer; ARS suites now pass fully offline
- regression tests for each behavior above
Print a warning whenever a CATRAX KP is registered from the dev or test
environment because the higher-priority one did not serve its meta
knowledge graph, so callers can see which query URLs are in play.
@SkyeAv
SkyeAv removed this pull request from stack #67 November 17, 2026 21:20
@SkyeAv
SkyeAv changed the base branch from query-ars to main November 17, 2026 21:20
@SkyeAv

SkyeAv commented Nov 17, 2026

Copy link
Copy Markdown
Member Author

Thanks for the thorough review. All six inline comments are addressed in 790fbdb and f7daf2a, with regressions for each; replies inline.

The branch is rebased onto main (the fastmcp commits dropped as already upstream via #64) and the PR is retargeted to main; the offline suite is green apart from the pre-existing network-dependent nameres/nodenorm/node_annotator tests. The ARS test files now pass with the network fully blocked.

On the tool surface question: I'll leave that call to Guangrong - she explicitly asked for an ARS neighborhood finder and pathfinder implementation separate from the rest of the MCP tooling, which is why ARS_neighborhood_finder and ARS_pathfinder are exposed as tools here. If the two of you agree on trio + query_ars only (and/or snake_case renames), that's a small follow-up. Also happy to coordinate the ordering with #68.

@SkyeAv

SkyeAv commented Nov 17, 2026

Copy link
Copy Markdown
Member Author

Note on the red Tests workflow: all 14 failures are the live-service tests (tests/test_nodenorm.py, tests/test_nameres.py, tests/test_node_annotator.py), not anything in this PR. Node Normalizer appears to be down - https://nodenorm.transltr.io/status, the root, and get_normalized_nodes all return 404 right now (also on nodenorm.test.transltr.io), and nameres.transltr.io/status is not connecting. The node_annotator failures are upstream annotation drift ('clinical_approval' missing from the live payload).

The same 14 tests fail identically on the pre-review base commit 94eefb7 and on a local run of this branch, and every offline/mocked test passes (164 passed, 7 skipped). Will re-run CI once the services are back; no code changes needed for this.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants