Repository navigation
Adding GUI System Tests - #249
MikeSullivan7 wants to merge 24 commits into
Conversation
StephenNneji
left a comment
StephenNneji
left a comment
There was a problem hiding this comment.
Thanks for this, please see comments
Also please update the unit testing section in contributing.md appropriately
bd5e4d5 to
14f84e2
Compare
…m-tests and --run-unit-tests
… project in testing
… due to matlab runner not exiting properly
e1e04c9 to
2dc1c80
Compare
StephenNneji
left a comment
StephenNneji
left a comment
There was a problem hiding this comment.
Apologies for the long wait for review, still a few issues especially on Linux
| self.process.kill() | ||
| self.process = None | ||
| self.clear_queues_and_events() | ||
| self.matlab_helper.close_event.set() |
There was a problem hiding this comment.
matlab helper is already stopped in the start_app function, if you are trying to stop matlab heleper for the tests it might be better to add it to the test cleanup function
| sys.excepthook = sys.__excepthook__ | ||
| logger = logging.getLogger("rascal2") | ||
| logger.handlers.clear() | ||
| QTimer.singleShot(SHORT_DELAY, lambda: self._click_messagebox(["Discard", "Don't Save"])) |
There was a problem hiding this comment.
This doesn't work on Linux because the button name is different, maybe try using the button role https://doc.qt.io/qt-6/qmessagebox.html#ButtonRole-enum
or you can clean the undo stack to prevent the msgbox from showing
| ("dream", 9.67234, 2000), | ||
| ] | ||
| ) | ||
| def test_run_slow(self, procedure_name, expected_chi, test_duration): |
There was a problem hiding this comment.
I have a weird issue on my Linux machine where the slow test doesn't complete for some reason. It gets stuck on 99% for dream and finish is never printed for NS. Please can you investigate on a Vm or IDAaas
|
|
||
| def setUp(self) -> None: | ||
| setup_logging() | ||
| self.start_processes_old = os.getenv("START_PROCESSES") |
There was a problem hiding this comment.
Is the START_PROCESS env variable still needed?
This branch has not been deployed
2 participants
This PR introduces a new form of GUI System Test which creates an instance of RasCal2, performs a sequence of events (clicks, typing, etc) and tests the validity of the output. This is more useful than isolated unit tests as it tests theinteraction between different parts of the program and more closely resembles a realistic user workflow.
System tests inherit the
GuiSystemBaseclass, which handles thesetUpand tearDownof theself.main_window` instance, simplifying how the tests are written.Unit tests and System tests are differentiated in pytest. We can run only unit tests via:
pytest testsorpytest tests --run-unit-testsSystem tests are run by default but can be skipped via:
pytests tests --skip_system_testsTests which take too long are marked with a "skip" mark, to run these, use the flag:
pytests tests --run_slow_testsIf we want to include screenshot/eyes tests in the future, a flag can be created for them to differentiate screenshot tests from unit and system tests.