Skip to content

Do not mutate typing.Protocol classes - #591

Open
GiulioDER wants to merge 1 commit into
boxed:mainfrom
GiulioDER:skip-protocol-classes
Open

GiulioDER wants to merge 1 commit into
boxed:mainfrom
GiulioDER:skip-protocol-classes

Conversation

@GiulioDER

@GiulioDER GiulioDER commented Oct 3, 2026 •

Copy link
Copy Markdown

Fixes #590.

Mutating a method of a Protocol adds the generated mutant methods (xǁSourceǁread__mutmut_orig, xǁSourceǁread__mutmut_1) as members of the protocol. Classes that implement it structurally then no longer conform, so the type checker reports an error wherever one is used as the protocol. An error in an original method body crashes filter_mutants_with_type_checker with "Could not find mutant for type error"; an error inside a mutant function discards that unrelated mutant as caught by the type checker.

This skips any class that lists typing.Protocol or typing_extensions.Protocol among its bases, plain or subscripted (Protocol[T]), decorated with @runtime_checkable or not.

How the base is recognised. Matching the name Protocol alone would also skip asyncio.Protocol and Twisted's Protocol, which are ordinary base classes with real methods. libcst's QualifiedNameProvider resolves this correctly but made create_mutations about 65% slower on the larger files in this repository. So the visitor records how typing, typing_extensions and their Protocol are imported (aliases, star imports and imports under if TYPE_CHECKING: included) and resolves the bases against that. A subclass of a protocol that does not list Protocol itself is a regular class and is still mutated.

Trade-offs and limitations, so you can decide whether any of them should change:

  • Method bodies of protocols are no longer mutated at all, including default implementations that explicit subclasses inherit, because mutating them changes the protocol's members in the same way.
  • Imports are recorded per module, not per scope, so a from typing import Protocol inside a function also counts at module level. I have not seen this in real code.
  • A Protocol re-exported through a project module (from mypkg.compat import Protocol) or imported relatively is not recognised, so such a protocol is still mutated as before.
  • As with any change to the mutation rules, an existing mutants/ directory keeps the old mutated copy of a source file until that file changes, so mutants/ needs to be removed to pick up the fix.

Verification (Linux, Python 3.10 and 3.12):

  • New tests: on main, the 13 protocol cases fail on their assertions and the 4 negative cases (an explicit subclass, asyncio.Protocol, Twisted's Protocol, a relative import) pass. I also mutated the new code ten ways (each import form, the subscript unwrap, the relative import check, matching by name only); each mutation fails at least one case.
  • Full suite: 469 passed on 3.10 and 470 on 3.12, against 453 on main; the difference is the new tests. pre-commit (ruff, ruff format, mypy) passes.
  • The minimal reproduction from Mutating a typing.Protocol stub adds generated members to the Protocol and breaks type_check_command #590: released 3.8.0 stops with the "Could not find mutant" exception; with this branch it completes all 7 mutants (3 killed, 1 survived, 3 caught by the type checker).
  • The project where I first hit this: with mutmut's own filter_mutants_with_type_checker, 3.8.0 raises on the 140 type errors caused by the rewritten protocol; with this branch they are gone and the filter completes.
  • Differential check: I mutated 704 files (the standard library, installed packages and my project, 268k lines) with main and with this branch. 694 produce byte-identical output, and every mutant removed in the other 10 lies inside one of the 105 typing protocols found by QualifiedNameProvider, which I used as an independent oracle.
  • create_mutations timing on src/mutmut/__main__.py and tests/mutation/test_mutation.py is within 3% of main.

Mutating a method of a Protocol adds the generated mutant methods
(xǁFooǁbar__mutmut_orig, xǁFooǁbar__mutmut_1, ...) as members of the
protocol. Classes that implement it structurally then no longer conform,
so the type checker reports an error wherever one is used as the
protocol. An error in an original method body crashes the type check
filter with "Could not find mutant for type error"; an error inside a
mutant function discards that unrelated mutant as caught by the type
checker. Method stubs are never executed either, so mutants of their
default arguments could not be killed.

Skip any class that lists typing.Protocol or typing_extensions.Protocol
among its bases, plain or subscripted. The base is resolved through the
module's imports (aliases and star imports included), so other classes
named Protocol, such as asyncio.Protocol or Twisted's, are still
mutated, and so is a subclass of a protocol that does not list Protocol
itself.

Fixes boxed#590
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Mutating a typing.Protocol stub adds generated members to the Protocol and breaks type_check_command

1 participant