Repository navigation
Support smallest and largest subunit mode for RLEs #665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
theferrit32
wants to merge
10
commits into
main
Choose a base branch
from
kf/637-rle-subunit-mode
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
1b147a3
feat(normalize): add RleSubunitMode to select RLE repeat subunit length
theferrit32 c5f071c
test(normalize): cover RLE subunit mode selection and round-tripping
theferrit32 7446db9
test(normalize): round-trip corpus RLE alleles through denormalize
theferrit32 e40e6ca
test(normalize): clarify RLE subunit mode cases and assert why modes …
theferrit32 da7c064
test(normalize): use REST proxy for RLE mode cases
theferrit32 fa669fc
fix(normalize): validate RLE subunit mode
theferrit32 d45494c
Merge remote-tracking branch 'origin/main' into kf/637-rle-subunit-mode
theferrit32 2246997
refactor(normalize): add DEFAULT_RLE_SUBUNIT_MODE and clean-up docstr…
theferrit32 e467202
test: parametrize RLE subunit mode tests and cover translator and SPD…
theferrit32 acef36b
refactor(normalize): yield factors lazily in either RLE subunit mode
theferrit32 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||
|---|---|---|---|---|
|
|
@@ -6,7 +6,8 @@ | |||
|
|
||||
| import itertools | ||||
| import logging | ||||
| from enum import IntEnum | ||||
| from collections.abc import Iterator | ||||
| from enum import Enum, IntEnum | ||||
| from typing import NamedTuple | ||||
|
|
||||
| from bioutils.normalize import NormalizationMode | ||||
|
|
@@ -20,6 +21,26 @@ | |||
| _logger = logging.getLogger(__name__) | ||||
|
|
||||
|
|
||||
| class RleSubunitMode(str, Enum): | ||||
| """Define which repeat subunit length to select for a reference-derived ambiguous | ||||
| insertion normalized to a ``ReferenceLengthExpression``. More than one factor of | ||||
| the seed length may be circularly expandable to recreate the alternate sequence; | ||||
| this selects which of them becomes the ``repeatSubunitLength``. The options are: | ||||
| LARGEST - Select the greatest valid factor (VRS 2.0.x normalization) | ||||
| SMALLEST - Select the smallest valid factor (VRS 2.1 normalization) | ||||
|
|
||||
| ``repeatSubunitLength`` is part of the ``ReferenceLengthExpression`` digest, so the | ||||
| two modes may produce different identifiers for the same input Allele. | ||||
| """ | ||||
|
|
||||
| LARGEST = "largest" | ||||
| SMALLEST = "smallest" | ||||
|
|
||||
|
|
||||
| # VRS 2.0.x normalization selects the greatest valid factor | ||||
| DEFAULT_RLE_SUBUNIT_MODE = RleSubunitMode.LARGEST | ||||
|
|
||||
|
|
||||
| class PosType(IntEnum): | ||||
| """Define the kind of position on a location""" | ||||
|
|
||||
|
|
@@ -85,7 +106,10 @@ def _get_new_allele_location_pos( | |||
|
|
||||
|
|
||||
| def _normalize_allele( | ||||
| input_allele: models.Allele, data_proxy: _DataProxy, rle_seq_limit: int = 50 | ||||
| input_allele: models.Allele, | ||||
| data_proxy: _DataProxy, | ||||
| rle_seq_limit: int = 50, | ||||
| rle_subunit_mode: RleSubunitMode = DEFAULT_RLE_SUBUNIT_MODE, | ||||
| ): | ||||
| """Normalize Allele using "fully-justified" normalization adapted from NCBI's | ||||
| VOCA. Fully-justified normalization expands such ambiguous representation over the | ||||
|
|
@@ -111,7 +135,12 @@ def _normalize_allele( | |||
| of the `sequence`. | ||||
| To exclude `sequence` from the response, set to 0. | ||||
| For no limit, set to `None`. | ||||
| :param rle_subunit_mode: Which valid factor of the seed length to select as the | ||||
| `repeatSubunitLength` for a reference-derived ambiguous insertion. Defaults to | ||||
| `DEFAULT_RLE_SUBUNIT_MODE`. See `RleSubunitMode`. | ||||
| """ | ||||
| rle_subunit_mode = RleSubunitMode(rle_subunit_mode) | ||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this is necessary if we're requiring an enum member to be passed
Suggested change
|
||||
|
|
||||
| # Algorithm applies to LiteralSequenceExpression alleles only; other states are returned unchanged | ||||
| if not isinstance(input_allele.state, models.LiteralSequenceExpression): | ||||
| _logger.warning( | ||||
|
|
@@ -204,7 +233,7 @@ def _normalize_allele( | |||
|
|
||||
| # 5.c | ||||
| if len_extended_alt > len_extended_ref: | ||||
| factors = _factor_gen(seed_length) | ||||
| factors = _factor_gen(seed_length, rle_subunit_mode) | ||||
| for cycle_length in factors: | ||||
| if cycle_length > len_extended_ref: | ||||
| continue | ||||
|
|
@@ -308,17 +337,32 @@ def denormalize_reference_length_expression( | |||
| return alt | ||||
|
|
||||
|
|
||||
| def _factor_gen(n: int): | ||||
| """Yield all factors of an integer `n`, in descending order""" | ||||
| lower_factors = [] | ||||
| def _factor_gen( | ||||
| n: int, mode: RleSubunitMode = DEFAULT_RLE_SUBUNIT_MODE | ||||
| ) -> Iterator[int]: | ||||
| """Yield all factors of an integer `n` | ||||
|
|
||||
| :param n: The integer to factor | ||||
| :param mode: `RleSubunitMode.LARGEST` to yield factors in descending order, | ||||
| `RleSubunitMode.SMALLEST` to yield them in ascending order. The first valid | ||||
| factor found by the caller is the one selected, so the order determines whether | ||||
| the largest or smallest valid factor wins. | ||||
| """ | ||||
| paired_factors = [] | ||||
| i = 1 | ||||
| while i * i <= n: | ||||
| if n % i == 0: | ||||
| yield n // i | ||||
| if n // i != i: | ||||
| lower_factors.append(i) | ||||
| # i <= sqrt(n) <= n // i. Yield the factor that is already in the requested | ||||
| # order, and save its pair to yield in reverse once the loop finishes. | ||||
| if mode == RleSubunitMode.SMALLEST: | ||||
| factor, pair = i, n // i | ||||
| else: | ||||
| factor, pair = n // i, i | ||||
| yield factor | ||||
| if pair != factor: | ||||
| paired_factors.append(pair) | ||||
| i += 1 | ||||
| yield from reversed(lower_factors) | ||||
| yield from reversed(paired_factors) | ||||
|
|
||||
|
|
||||
| def _define_rle_allele( | ||||
|
|
@@ -371,6 +415,9 @@ def normalize(vo, data_proxy: _DataProxy | None = None, **kwargs): | |||
| :param data_proxy: GA4GH sequence dataproxy instance, if needed | ||||
| :keyword rle_seq_limit: If RLE is set as the new state, set the limit for the length | ||||
| of the `sequence`. To exclude `state.sequence`, set to 0. | ||||
| :keyword rle_subunit_mode: Which valid factor of the seed length to select as the | ||||
| `repeatSubunitLength` for a reference-derived ambiguous insertion. Defaults to | ||||
| `DEFAULT_RLE_SUBUNIT_MODE`. See `RleSubunitMode`. | ||||
| :return: normalized object, or unmodified input object if the normalization algorithm | ||||
| does not provide normalization steps for the given type. | ||||
| :raise TypeError: if given object isn't a pydantic.BaseModel | ||||
|
|
||||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the
2.1branch, we'll only allow smallest, right?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@korikuzma my plan was to just change the default to smallest just in case someone wanted to generate a hash based on using the largest subunit. But we could remove support for largest entirely too. I have no strong opinion since it's not too many lines to keep around.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO I think we should only allow smallest in
2.1branch so that people don't create invalid digests