Add RFC9457 support problem details handling

[luisgcoding]

This PR adds support for RFC 9457 (Problem Details for HTTP APIs), enabling pip to parse and display structured error responses from package indexes that implement this standard.

What Changed

• Added ProblemDetails data class to parse RFC 9457 responses
• Created HTTPProblemDetailsError exception for structured HTTP errors
• Modified raise_for_status() to detect and handle Problem Details responses
• Falls back to existing error handling for non-RFC 9457 responses

Implementation Details

When pip receives an HTTP error response with Content-Type: application/problem+json, it:

1. Parses the response as RFC 9457 Problem Details
2. Raises HTTPProblemDetailsError with structured error information
3. Displays human-readable error messages with status, title, and detail fields

Non-RFC 9457 responses continue to use the existing NetworkConnectionError behavior.

Based on this discussion in the forum, this implementation could potentially lead to a PEP proposal. This PR is intended to start the conversation with the pip maintainers about adopting RFC 9457 support in the packaging ecosystem.

[luisgcoding]

pre-commit.ci autofix

[luisgcoding]

pre-commit.ci autofix

[ichard26]

Hello! Thanks again for following up! I regret to inform you that we currently have limited capacity to review contributions, so it may take some time until we are able to review your PR. We would appreciate your patience.

I take it as there are no publicly available indices that provide RFC 9457 error responses?

[notatallshaw]

Also, it wasn't clear to me from the DPO discussion that is ready for implementation, should this not be a packaging PEP first and gain community consensuses? @pfmoore

[pfmoore]

I agree, I think we should probably wait until this is a standard before implementing it here. Or at least until PyPI implements it (there's a possibility that PyPI won't wait for a standard, and if something is available in PyPI there's a decent justification for us implementing it anyway - but that would be on a case by case basis).

[notatallshaw]

@luisgcoding thanks for the PR but I'm marking it as draft for now. I guess either:

• PyPI updates it's current non-standard message to said RFC standard (but not Python packaging standard), and we update update pip's logic to reflect that
• Or, a Python packaging PEP is created and accepted and we can implement that

[luisgcoding]

Hello! Thanks again for following up! I regret to inform you that we currently have limited capacity to review contributions, so it may take some time until we are able to review your PR. We would appreciate your patience.

I take it as there are no publicly available indices that provide RFC 9457 error responses?

Thank you for the information @ichard26 . You are right as far as I know there are no public registries that support RFC9457 responses yet.

[luisgcoding]

@luisgcoding thanks for the PR but I'm marking it as draft for now. I guess either:

• PyPI updates it's current non-standard message to said RFC standard (but not Python packaging standard), and we update update pip's logic to reflect that
• Or, a Python packaging PEP is created and accepted and we can implement that

Thanks for the feedback @notatallshaw I understand the concern about implementing this before there's concrete adoption.

A few questions to help me determine the best path forward:

1. Would it be appropriate to propose this to the PyPI maintainers first? If so, what's the best channel for that discussion?
2. Do you know what is the best way to start the conversation about creating a PEP for this?
3. Are there any other considerations I should be aware of when deciding between these approaches?

Happy to keep this as a draft and coordinate with the appropriate stakeholders before moving forward.

[pfmoore]

I would strongly suggest working towards a standard. Even if PyPI implement this without a standard, I'd be cautious about implementing it in pip (my comment above wasn't intended to endorse the idea of us implementing this without a standard, just to note that it was possible).

There's already a discussion on discuss.python.org, so the process to start working towards a PEP is to continue the discussion there, and offer to write a PEP 🙂

[luisgcoding]

I would strongly suggest working towards a standard. Even if PyPI implement this without a standard, I'd be cautious about implementing it in pip (my comment above wasn't intended to endorse the idea of us implementing this without a standard, just to note that it was possible).

There's already a discussion on discuss.python.org, so the process to start working towards a PEP is to continue the discussion there, and offer to write a PEP 🙂

@pfmoore thank you for all the insightful feedback and guidance so far.

Following the suggestion to move forward via the PEP process, I wanted to let you know that I’ve posted in the relevant forum this comment to get the conversation started.

If possible, could any of the project maintainers provide further suggestions or point me to best practices as I begin outlining this PEP? I want to make sure I proceed in accordance with community expectations and coordinate in the most effective way.

Thanks again for your help and for clarifying the recommended next steps.

[ichard26]

@luisgcoding it may be worth reaching out to @woodruffw who was looking to standardize RFC 9457 in a different context (I already forgot what exactly). Perhaps you two can pair up on a PEP?

[woodruffw]

Thanks for the ping! Yeah, PEP 807 (for trusted publishing) and PEP 694 (for Upload 2.0, not my PEP though) both stipulate RFC 9457 for error responses, and IMO it'd be great/beneficial to the ecosystem as a whole if all of the index response interfaces were to unify on a single error reporting scheme 🙂. So I'd be happy to discuss / coordinate on a standards approach here.

(The nuance with PEP 503/691 is that they don't define the error response codes/structure at all at the moment, so conforming indices can technically use anything, including RFC 9457. That's what pyx does and I believe a few others, with the hope that clients progressively enhance when they receive an error response with a problem details content type.)

[woodruffw]

For some more context, I opened this DPO thread a while back to discuss the idea but didn't get much feedback/insight from the larger community on it:

https://discuss.python.org/t/does-it-make-sense-to-standardize-index-error-responses/104839

[luisgcoding]

@woodruffw Thank you so much for the context and for offering to coordinate on this! I really appreciate the insight about PEP 807 and PEP 694 already mandating RFC 9457, that makes a strong case for consistency across the entire packaging ecosystem.

I'd love to collaborate on a PEP for this. A few thoughts on next steps:

1. From what I understand, the change proposed here would be on standardizing error responses for PEP 503/691.
2. I'm happy to take the lead on drafting, but I'd greatly value your input given your experience and knowledge of the ecosystem.

Would you be interested in being a co-author or reviewer? Also, any guidance on the PEP process or sponsors would be incredibly helpful.

Thanks again for your willingness to help move this forward

[woodruffw]

Would you be interested in being a co-author or reviewer? Also, any guidance on the PEP process or sponsors would be incredibly helpful.

I'm happy to review a PEP/offer guidance on the PEP process (with the caveat being that I'm not a PEP-Delegate or anything like that, just someone who has written PEPs before and is interested in this one 🙂)

I agree with your steps forward! I think a good concrete thing would be to start a pre-PEP thread on DPO to get buy-in + others' perspectives on the implications of standardizing errors atop the existing index standards.