Skip to content

Add mcpscore to LLM & MCP Testing - #138

Open
a-akimov wants to merge 1 commit into
cleder:mainfrom
a-akimov:patch-1
Open

a-akimov wants to merge 1 commit into
cleder:mainfrom
a-akimov:patch-1

Conversation

@a-akimov

@a-akimov a-akimov commented Oct 13, 2026 •

Copy link
Copy Markdown

Adds mcpscore, an open-source auditor for MCP servers, to the LLM & MCP Testing section (alphabetical, between mcp-server-fuzzer and OrcaReplay).

It connects to a running server, runs deterministic protocol, tool-schema and security-posture checks, and reports a 0-100 score with a suggested fix for each failed check. Available on PyPI (pip install mcpscore), MIT licensed.

Disclosure: I'm the author.

Summary by Sourcery

Add mcpscore to the LLM and MCP testing resource collection.

New Features:

  • Add mcpscore to the LLM and MCP testing resources list, highlighting its MCP server auditing and scoring capabilities.

Documentation:

  • Document mcpscore as a deterministic MCP server auditor with CLI, GitHub Action, and SARIF support.

@semanticdiff-com

Copy link
Copy Markdown

Review changes with  SemanticDiff

@sourcery-ai

sourcery-ai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

The README now lists mcpscore between mcp-server-fuzzer and OrcaReplay as an open-source MCP server auditor that deterministically evaluates protocol conformance, tool/schema quality, and security posture, producing scored results and suggested fixes.

File-Level Changes

Change Details Files
Add mcpscore to the LLM & MCP Testing resource list in alphabetical order.
  • Link to the mcpscore GitHub repository.
  • Describe its 0–100 MCP server auditing score, deterministic checks, and remediation guidance.
  • Mention CLI and GitHub Action usage, SARIF output, and MIT licensing.
README.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@qodo-code-review

qodo-code-review Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

Grey Divider

Sorry, something went wrong

We weren't able to complete the code review on our side. Please try again manually by commenting /agentic_review on this PR.

Grey Divider

Qodo Logo

@qodo-code-review

Copy link
Copy Markdown

PR Summary by Qodo

List mcpscore in LLM & MCP Testing resources

📝 Documentation 🕐 Less than 5 minutes

Grey Divider

AI Description

• Adds mcpscore to the LLM & MCP Testing resource list.
• Describes its MCP server audit scores, suggested fixes, CLI and GitHub Action support.
High-Level Assessment

A README entry is the appropriate approach for adding a resource to this curated list; no architectural alternative is warranted.

Files changed (1) +1 / -0

Documentation (1) +1 / -0
README.mdAdd mcpscore to the LLM & MCP Testing list +1/-0

Add mcpscore to the LLM & MCP Testing list

• Adds a link to mcpscore between mcp-server-fuzzer and OrcaReplay. The entry describes its MCP server audit scope, 0–100 score, suggested fixes, CLI and GitHub Action support, and MIT license.

README.md

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

@mergify

mergify Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@llamapreview llamapreview Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LlamaPReview — No blocking issues found

This Markdown-only PR adds the requested mcpscore entry to the LLM & MCP Testing list, rewriting no existing content and closing the stated objective; there is no executable, configuration, or contract surface to regress.

Review details and evidence

Material unknowns

  • Whether the linked project surface is real as described — the repository at mcp-box/mcpscore is live, public, and MIT-licensed, and it provides the CLI / GitHub Action / SARIF behavior described in the blurb. Deep stated this does not decide the objective (the entry was added as specified) and is not a request-changes reason; it would only matter if maintainers additionally require external-link liveness verification for curated-list entries.
    • Check: No pre-merge action is required by Deep; the optional check is to open the linked repository and package-registry metadata if the maintainers want external-link liveness verified.

LlamaPReview checks

  • Reviewed changed regions in README.md.
  • Read the complete PR-head file README.md.

LlamaPReview is open source. Explore the code behind this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant