protosig

Verify that a class's methods actually match a Protocol's declared signatures — not just that members with the right names exist.

from typing import Protocol, runtime_checkable

@runtime_checkable
class Sender(Protocol):
    def send(self, msg: str, *, retries: int = 3) -> None: ...

class MyHandler:
    def send(self) -> None: ...

isinstance(MyHandler(), Sender)   # True
MyHandler().send("hi", retries=1) # TypeError: send() got an unexpected keyword argument 'retries'

That's the whole bug. isinstance() said yes.

The problem

@runtime_checkable on a Protocol lets isinstance() check it at runtime, which is what makes plugin systems, dependency injection, and duck-typed handler registries type-annotatable at all. But the check it performs is shallow: for each protocol member, it asks "does an attribute with this name exist?" — nothing about its signature. A class whose send takes zero arguments satisfies a protocol whose send takes a required string and a keyword-only int, because both classes have "an attribute called send". You find out it doesn't actually work the first time it's called, as a TypeError at whatever call site happens to exercise that code path — which, for a rarely-hit plugin or error handler, might be production.

This isn't a quirk of the standard library specifically:

• beartype and typeguard's runtime-checkable Protocol support are both built on the same isinstance() mechanism — a @beartype- or @typechecked-decorated function that accepts a Protocol-typed parameter lets the same broken MyHandler through. (typeguard 4.6 does go slightly further than plain isinstance() — it catches gross arity mismatches, like a required positional parameter missing outright — but it does not check parameter or return types, so a send(self, msg: int) for a protocol declaring send(self, msg: str) still passes. See examples/verify_the_gap.py.)
• Pydantic, validating a field typed as a Protocol (arbitrary_types_allowed=True), does the same isinstance() check and no more.

None of them close this gap, because none of them compare signatures. mypy and pyright do catch this — at the call site, statically, when the concrete type is known at that point in the code. What they cannot see is a class assembled at runtime and handed back through a Protocol boundary: a plugin loaded by name, a handler registered from a decorator, a class built by a factory. protosig is for that case — a runtime check for the classes a type checker never got to look at together with their protocol.

Verify the gap yourself:

python examples/verify_the_gap.py

Real output (Python 3.14.7, beartype 0.22.9, typeguard 4.6.0, pydantic 2.13.5 — beartype/typeguard/pydantic are not protosig dependencies, the example just skips a section if one isn't installed):

=== Part 1: the standard library ===
isinstance(h, Sender) = True
h.send('hello', retries=1) -> TypeError: MyHandler.send() got an unexpected keyword argument 'retries'

protosig.conforms(MyHandler, Sender) = False
MyHandler vs Sender: does not conform
  [x] send: candidate is missing positional parameter 'msg' (position 0)
  [x] send: candidate is missing keyword-only parameter 'retries'

=== Part 2a: beartype ===
beartype let it through; failed only at the real call: MyHandler.send() got an unexpected keyword argument 'retries'

=== Part 2b: typeguard ===
typeguard catches the arity mismatch: TypeCheckError: __main__.MyHandler is not compatible with the Sender protocol because its 'send' method has too few positional arguments
typeguard.check_type on a str->int mismatch: PASSED (the real gap)
protosig catches it: conforms(WrongType, Sender) = False

=== Part 2c: pydantic ===
pydantic accepted MyHandler as a Sender
holder.sender.send('hello', retries=1) -> TypeError: MyHandler.send() got an unexpected keyword argument 'retries'

Install

pip install protosig

Python >= 3.11. No runtime dependencies.

Use

from typing import Protocol
from protosig import conforms, check_conformance

class Sender(Protocol):
    def send(self, msg: str, *, retries: int = 3) -> None: ...

class MyHandler:
    def send(self) -> None: ...

conforms(MyHandler, Sender)  # False

if not conforms(MyHandler, Sender):
    raise TypeError(check_conformance(MyHandler, Sender).report())

MyHandler vs Sender: does not conform
  [x] send: candidate is missing positional parameter 'msg' (position 0)
  [x] send: candidate is missing keyword-only parameter 'retries'

@runtime_checkable is not required on the Protocol — protosig never calls isinstance() internally, so it also works on Protocols nobody remembered to mark runtime-checkable.

Fail at import time instead of call time

from protosig import implements

@implements(Sender)
class MyHandler:
    def send(self) -> None: ...  # raises TypeError immediately, at class definition

This is the main intended use: decorate plugin/handler classes so a signature mismatch is a stack trace at import time, pointing at the class, instead of a TypeError three layers deep in whatever called .send().

The structured result

result = check_conformance(MyHandler, Sender)

result.ok             # bool — False if any error-severity issue exists
result.issues         # tuple[Incompatibility, ...], every member checked
result.errors         # only the error-severity ones
result.unknowns       # types protosig couldn't verify either way
result.checked_members
result.report()       # the human-readable string shown above

Each Incompatibility has .member, .reason, and .severity ("error" or "unknown").

What "compatible types" means here

Parameters are checked contravariantly and return types covariantly: an implementation may accept a wider parameter type than the protocol promises and return a narrower one than the protocol requires — never the reverse. Reversing this is the classic mistake (it's tempting to check both the same way) and would make protosig actively wrong, not just incomplete, so it's covered by tests in both directions (tests/test_variance.py).

Beyond direction, "compatible" is deliberately a small, documented subset — not a reimplementation of a type checker:

Case	Handled
Identity (X vs X)	Yes
typing.Any on either side	Yes — universal match
object as the wider side	Yes
None / Optional[X] / X | Y	Yes — recurses into each union member
Plain-class subclassing (issubclass)	Yes
Parameterized generics (list[int] vs list[str], dict[str, int], ...)	No — only exact origin+args identity matches; anything else is unknown
TypeVar, ParamSpec, Literal, nested Callable[...]	No — unknown
Forward references / strings that fail to resolve	No — unknown, named explicitly in the report

Unknown is a real outcome, not a fallback for "probably fine." When protosig cannot determine compatibility, it says so as a distinct severity="unknown" entry rather than silently passing the member or silently failing it. conforms() / ConformanceResult.ok treat unknowns as non-blocking (protosig won't claim a false positive failure for something it genuinely can't check) — call .unknowns or read .report() if you want to see them. This is the honest boundary: full variance checking against arbitrary generics is a type checker's job.

What else is checked

• Positional-only vs. positional-or-keyword vs. keyword-only parameter kinds, matched by position (positional) or name (keyword) — a protocol parameter named without a leading underscore must be callable by that keyword on the candidate too, mirroring the convention mypy uses for callback protocols.
• Defaults: a candidate may not turn a protocol's optional parameter into a required one; a candidate may add extra parameters only if every extra one has a default (or is absorbed by *args/**kwargs).
• *args / **kwargs absorbing parameters the candidate doesn't name explicitly.
• @property, @classmethod, @staticmethod, and plain data attributes, each compared against the protocol member of the matching kind. A kind mismatch (protocol declares a property, candidate defines a method, or vice versa) is an error; where the candidate's type can't be determined without an instance (a data attribute typed only inside __init__, for example), it's reported as unknown.
• async def vs. def — an async candidate for a sync protocol member returns a coroutine object instead of the declared value, and vice versa; both directions are checked and both are errors.

Where this stops

• Data attributes are checked covariantly, not invariantly. A mutable attribute exposed for both read and write is, strictly, invariant — but treating it that way produced far more false positives than it caught real bugs, so protosig checks attribute compatibility the same way it checks a return type. Document this if you rely on write-narrowing through a Protocol attribute.
• Property setters aren't modeled. Only the getter's return type is compared; a mismatched @x.setter parameter type isn't checked.
• Generic containers are not unified. list[int] vs. list[object] is reported as unknown, not as the (correct, but variance-dependent) answer a real type checker would give.
• This is a runtime check, not a substitute for mypy or pyright. Static type checkers catch this class of bug at build time, wherever they can see the concrete class and the Protocol together. protosig exists for the cases they cannot see: a plugin loaded by name, a handler assembled by a factory, anything wired together dynamically where the type checker never gets to compare the two.
• No instantiation. protosig works from the class object alone (inspect.getattr_static, so property getters are never invoked) — it cannot see attributes that only exist after __init__ runs unless they're annotated on the class.

Develop

python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
.venv/bin/python -m pytest -q
.venv/bin/python -m mypy src --strict

License

MIT