Adapted skills:check and skills:install and added reusable tests

[jana-selva]

Fixes #963

Checklist

Note: If any of the items in the checklist are not relevant to your PR, just check the box.

For any Pull Request

Is the following correct:

• the title of the Pull Request?
• the title of the corresponding issue?
• there are no other open Pull Requests for the same update/change?
• that the issue which this Pull Request fixes ("Fixes...") is mentioned?

When Changes Were Made

Did you:

• update the changelog?
• update the cookiecutter-template?
• update the implementation?
• check coverage and add tests: unit tests and, if relevant, integration tests?
• update the User Guide & other documentation?
• resolve any failing CI criteria (incl. Sonar quality gate)?

When Preparing a Release

Have you:

• thought about version number (major, minor, patch)?
• checked Exasol packages for updates and resolved open vulnerabilities, if easily possible?

[sonarqubecloud]

❌ The last analysis has failed.

See analysis details on SonarQube Cloud

[ArBridgeman]

These kinds of tests are require lots of maintenance, if the code is changed.

[sonarqubecloud]

Quality Gate failed

Failed conditions
75.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

[ArBridgeman]

Quality Gate failed

Failed conditions 75.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

Ah, please add a test for project:check

[ArBridgeman]

This will actually load each time the fixture is called. If we wanted to do it once, we should set the scope.

Suggested change

	@pytest.fixture
	@pytest.fixture(scope="module")

[ArBridgeman]

Thanks! In general, this looks much cleaner.

I'll let you decide, but realized today that I think long-term it'd be better if we introduced a Pydantic model that we validate against. This way, we can share the model in the documentation (via the API) and link it where appropriate. Then, we can just validate against it in the tests. Below is just a mock up for it - I didn't evaluate it extensively or iterate on the names much

NonBlankString = Annotated[str, Field(pattern=r"\S")]


class ExpectedResponse(BaseModel):
    model_config = ConfigDict(extra="forbid")

    must_include: list[NonBlankString] = Field(min_length=1)
    must_not_include: list[NonBlankString] = Field(min_length=1)


class EvalCase(BaseModel):
    model_config = ConfigDict(extra="forbid")

    id: NonBlankString
    category: NonBlankString
    prompt: NonBlankString
    expected: ExpectedResponse


class PackagedSkillEvalCases(BaseModel):
    model_config = ConfigDict(extra="forbid")

    version: Literal[1]
    skill: NonBlankString
    cases: list[EvalCase] = Field(min_length=1)