mcp: add lab (runtime) queries - #311
Open
bhcopeland wants to merge 7 commits into
Open
Conversation
nuclearcat
added a commit
to nuclearcat/kci-dev
that referenced
this pull request
Aug 31, 2026
Each CI matrix job installs one Python version, but invoking tox without an environment selector makes it evaluate every configured environment. The unavailable interpreters are then reported alongside the environment that actually ran:
py312: FAIL
py313: skipped because could not find python interpreter with spec(s): py313
py313: SKIP
py314: skipped because could not find python interpreter with spec(s): py314
py314: SKIP
evaluation failed :(
This output is visible in the Python 3.12 job from PR kernelci#311:
https://github.com/kernelci/kci-dev/actions/runs/33051625550/job/98448291347?pr=311
Add an explicit tox environment to each matrix entry and pass it to tox with -e. This preserves coverage for Python 3.12, 3.13, and 3.14 while ensuring every job runs only against the interpreter installed for it.
Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
bhcopeland
force-pushed
the
mcp-lab-queries
branch
3 times, most recently
from
September 3, 2026 07:30
4e88fed to
65af198
Compare
The MCP query tools take status as a free-form string and hand it to StatusFilter, which only ever compares against lowercase 'pass', 'fail' and 'inconclusive'. Anything else matched nothing and returned an empty page with no error, so a caller could not tell a rejected value from a genuine absence of results. That includes the uppercase 'FAIL' the dashboard itself reports, which made echoing an observed status back as a filter look like a clean "no failures" answer. Normalise the value and reject anything outside the set the filter understands. The CLI is unaffected, since click.Choice already constrains it there. Also list 'all' in the tool docstrings, which the filter accepts but none of them mentioned. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
limit and offset reach a plain list slice, so Python's negative index handling quietly reinterpreted them: a negative limit returned nearly the whole result set, defeating the pagination that exists to keep responses small, and a negative offset returned an empty page while still reporting the full total. list_nodes passed both straight to the Maestro API instead. Reject negative values in both paths. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
Filters are split on '=' and passed to requests as query parameter pairs. A filter string without '=' produced a one-element tuple that failed deep in the request layer, where the generic handler turned it into "Maestro nodes request failed" with no mention of the filter that caused it. Check the syntax before the request and name the offending filter. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
The issue fetchers interpolate the id into their endpoint, so an empty id turns "issue/<id>" into "issue/", which is the collection endpoint dashboard_fetch_issue_list already uses. The request then succeeded and returned every known issue in place of the one that was asked for. Over MCP that surfaced as a 150KB result reported as success. On the command line, "kci-dev results issue --id ''" printed an issue with every field None and a dashboard link to /issue/None, then exited 0. Guard the three issue fetchers, which is the point the CLI and the library client share. The builds and tests siblings requested "issue//builds", which 404s, so those failed already but without saying why. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
Add list_labs, backed by the dashboard /metrics/ endpoint, returning the labs reporting to KernelCI with their build, boot and test counts for the last N days. Those names feed a new lab filter on list_builds, list_boots and list_tests, matching the top-level 'lab' field on boots and tests and 'misc.lab'/'misc.runtime' on builds. The dashboard has no server-side lab filter, so the filter narrows the fetched page rather than the request. Keep the 'labs' key in the compact get_summary response, which already aggregates per lab, and document data.runtime and data.platform in the list_nodes docstring. Reported-by: Yogesh Lal <yogesh.lal@oss.qualcomm.com> Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com> Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
days was passed through unchecked to a metrics aggregation that gets slower as its window grows. Measured against production: 3 days answers in under 10s cold, 7 days takes around 56s, and 14 and 30 days always exceed the 60s read timeout and come back as a bare "Dashboard metrics request failed" after a full minute. Cap the window at 7 days, so a caller cannot ask for one that cannot succeed. An unknown lab returned matched=0, which is indistinguishable from a lab that ran nothing on that commit. Return the labs the fetched entries actually report, so a mistyped name shows up without a second call. Validating up front is not possible: lab names are an open set and the only source is the metrics call itself. Drop 'maestro' as the example lab for builds. Every build from that origin reports it as misc.lab, so filtering by it matches everything; the runtime cluster is the value that discriminates. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
bhcopeland
force-pushed
the
mcp-lab-queries
branch
from
September 3, 2026 10:35
65af198 to
40600b1
Compare
Member
|
list_labs does not reliably discover labs accepted by list_builds Suggested fix:
|
Member
|
labs_present becomes misleading when combined with status For example, when lava-collabora has only passing tests:
returns no results and omits lava-collabora from labs_present. This still makes a valid lab with no failures indistinguishable from a misspelled lab, which is precisely what the hint intends to solve. Capture the complete entry set before applying the status filter, or return separately named status-scoped and unfiltered lab lists. A regression test combining a valid lab, status="fail" and only passing entries should be added. |
labs_present was collected after the status filter, so a real lab with no entries in the requested status was left out of it: asking for status=fail on a lab whose tests all passed returned no results and no mention of the lab, which is exactly the mistyped-name case the hint exists to rule out. Collect it from the commit's entries before any status filter, so it answers "does this lab appear here at all" rather than "does it appear with this status". Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
Member
Author
|
Thanks @aliceinwire. I have fixed up, and also added the regression test you stated. However, on the /labs/ endpoint. Agreed in principle, but it isn't deployed yet, i.e. /labs/ 404'ing. I suggest we push this as a follow up PR when this goes through if you are okay with that? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is also based on #310 so that needs to be merged first.
Adds list_labs and a lab filter to the list tools, so "how is lab X doing" no longer means listing a whole tree and filtering client-side. list_labs is capped at 7 days because wider windows cannot succeed.
The feature commit is Denys's work, carried here with his authorship. The follow-up commit bounds list_labs and makes an unmatched lab self-explanatory; it also adjusts one docstring from the first commit, deliberately kept separate so the correction is attributed to me rather than folded into his commit.