Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With OMNeT++ 6.5.0,
tests/fingerprint/fingerprinttest.pyreports a failing fingerprint asERROR (should be PASS): otherinstead ofFAILED (should be PASS): some fingerprint mismatch ....The mismatch itself is hidden. Passing rows are not affected.
On 6.5.0 this misreported 33 rows, for example lte/multicell
Interference-AllOn,-NoDownlinkand
-NoUplink, nr/bgTrafficCBR-DL, lte/demoVoIP_DL-ULand nr/mec/multiMecHostMultiMec.Cause
(line numbers on dev 764db6f,
runSimulationintests/fingerprint/fingerprinttest.py)runProgrammerges stderr into stdout.<!>.Fingerprintlines set the pass/fail data,Simulation time limit reachedis ignored (line 410), and every other<!>line falls into thefinal
else(line 412), which setswasError = True.unused-entry-checkoption (defaultwarning). It prints<!> Warning: N configuration entries had no effect in this run ..., so any run with anunused ini entry lands in that
else.wasErrortrue, theif/elifchain at lines 430-440 takes the first branch and skipsisFingerprintFail.result.isFingerprintOKstaysNone, andrunTestraisesException("other")(line 515).Fix
The warning line is added to the ignored cases, next to
Simulation time limit reached. Thepattern is anchored to the exact message of
EnvirBase::formatUnusedEntriesReport()(
N configuration entry/entries had no effect ...).I chose this over treating all
<!> Warning:lines as non-errors. The script has nospecial handling for any warning today: every
<!>line that is not a fingerprint line or the timelimit line counts as an error, and that strictness is the intent of the test (a simulation that
prints a warning is not a clean run). A blanket rule would also hide other warnings, for example
the obsolete-option warning. Only the one message that 6.5.0 prints by default, and that says
nothing about the simulation result, is exempted. Only the first line of the multi-line report
starts with
<!>, so one pattern covers it.Verification
Full fingerprint suite (
python3 fingerprinttest.py -t 8, release build) on macOS, OMNeT++ 6.5.0,INET 4.7.0, before (script from dev) and after (script with the fix), same binaries:
The 33 rows move from ERROR to FAILED with their real mismatch, and no passing row changes.
For example
Interference-AllOn: beforeERROR: other, afterFAILED: some fingerprint mismatch: actual '4eb8-526d/sz', expected: '4e98-526d/sz'.The FAILED rows are fingerprint differences on 6.5.0/INET 4.7.0 and are not part of this change.
The Simu5G build used for testing needed the macOS
intval_tcommit from PR #1 (3a50c1d4, appliedonly on a local test branch). This PR is independent of it and contains only the script change.