Overall, 2024 was a slower year for urllib3 in terms of development and funding compared to previous years. The year's highlights were increased financial support from multiple sources, developments for HTTP/2 and WASM support, and successful sponsored work from LaunchDarkly.

Past updates:

• Review of 2019 for urllib3
• urllib3 in 2020
• urllib3 raised $15,000 in 2021
• urllib3 in 2022
• urllib3 is fundraising for HTTP/2 support (2023)

Funding

urllib3 received $30,213 US dollars in financial support in 2024 from all sources and distributed $21,400 to contributors and maintainers. This is a substantial increase in financial support compared to 2023, when the project only received $17,830, 87% of which came from a single source (Tidelift).

Our primary supporter continues to be Tidelift, who provided $18,000 to core maintainers Seth, Quentin, and Illia.

Microsoft was the second largest funding source, generously donating $5,000 through the Microsoft FOSS Fund. Thanks to Microsoft for supporting their open source dependencies.

We distributed $1,600 to community contributors through our bounty program, slightly less than last year, despite our open bounties for HTTP/2 support. At the time of publication there are $3,300 worth of bounties in our issue tracker, so if you’d like to be paid to contribute to open source please take a look!

Our Open Collective started the year with around $12,000 USD and ended the year with nearly $20,000. This reflects both our fundraising for HTTP/2 and our lack of ability to pay contributors and maintainers to work on HTTP/2 issues.

While Open Collective makes it easy to receive and spend money while maintaining transparency, it is also costly. In 2024, we paid $1,304 in Open Collective fees and $233 in processing fees. For example, when LaunchDarkly donated $2,500, we paid more than $300 in fees.

Now, let’s examine the two main themes we identified for 2024: HTTP/2 and WebAssembly support.

Backwards compatible HTTP/2

Last year, we started fundraising for HTTP/2 support. Unfortunately, this has not been a complete success yet. Thanks to contributions from companies and individuals and reduced spending from fewer contributions, we grew our Open Collective balance from $12,000 to $20,000. However, we are still far from the $50,000 that is needed to pay someone full-time for a few months and make major progress in that area. Despite this, we still made some progress on our HTTP/2 milestone.

Testing-wise, we started 2024 by finishing the transition from Tornado to Hypercorn and Quart. Tornado served us well during all those years, but it does not support HTTP/2 and likely never will. We also added an http_version pytest fixture to make it easy to test HTTP/1.1 and HTTP/2 in the same test where possible, unfortunately, many of our tests inspect bytes on the wire to make assertions, so will be more difficult to port to using HTTP/2.

Another area of focus has been the HTTP/2 availability probing via ALPN and caching availability per origin and HTTP/2 connection pooling. For HTTP/2 to be more efficient than HTTP/1.1, sockets need to be reused for multiple concurrent HTTP requests (called “streams”) at the same time. The trouble is making this new model fit into urllib3’s existing API in a backwards compatible way so that applications don’t need to change their code (backwards compatibility over everything!)

LaunchDarkly sponsored bug fixes

In January 2023, a LaunchDarkly employee opened an issue complete with a reproducing script showing how their code using urllib3 could hang while reading a response, even if another thread tried to close that same response. Despite a few back-and-forths, nothing moved until November 2024, where LaunchDarkly expressed interest in sponsoring a fix of this issue, after a customer of theirs was affected.

Quentin and Illia collaborated on a fix to add support for socket.shutdown() to properly stop all reads on that socket, even ongoing ones. The fix was involved for a few reasons:

• The urllib3 response did not have a direct reference to the underlying socket, it was only stored in the internals of the http.client standard library module

• Writing a test for previously hanging code is tricky: it has to be realistic enough to actually test the fix, but should fail quickly in case of a regression, as a hanging test is time-consuming to debug.

• Our initial fix failed with the pyOpenSSL backend, thankfully Illia noticed this during the code review, and using TLS in the test helped confirm that fix.

Ultimately, fixing and releasing that issue took Quentin and Illia 12 hours, or $1800 at $150/hour. Andrey Petrov, the urllib3 author, helped to coordinate this collaboration. Without LaunchDarkly sponsoring this issue, we would likely never have prioritized it against the many other open issues in our bug tracker.

We are really grateful for the sponsorship, and would be happy to continue working with LaunchDarkly or other organizations in the future! We have a long track record of doing paid short-and-medium-term work on urllib3, such as with Stripe, CERT Governmental Luxembourg, and GitCoin Grants. Get in contact with sethmichaellarson@gmail.com to explore sponsored bug fixes or contributions.

WebAssembly (WASM)

When PyScript was first announced during the PyCon US 2023 keynote, Seth was sitting near the front row and was blown away. After the keynote Seth attended the hosted open space about PyScript and attempted to implement urllib3 using JavaScript APIs like fetch() or XMLHttpRequest() unfortunately to no avail. At the time, it wasn’t possible to synchronously wait for an async operation like the two previously mentioned APIs, this would be required to add WASM support to urllib3.

Fast-forward to today and thanks to the work of Joe Marshall, urllib3 v2.2.0 added support for being run in the browser (Emscripten) runtime. This means that many popular libraries like Requests will work out-of-the-box when used from Python in a browser environment. For urllib3 v2.3.0 Joe also added support for Node.js.

Sunsetting Python 2 and urllib3 1.x

Five years after Python 2's end-of-life, less than 2% of urllib3 1.26.x downloads are still done with Python 2. We feel this is the right time to drop it from the 1.x branch.

Additionally, we are dropping support for the 1.x branch of urllib3. urllib3 2.x is the best version of urllib3 and is widely supported by the larger Python ecosystem. That said, urllib3 1.x still sees significant download numbers, mainly because the botocore package still requires urllib3 1.26.x for Python 3.9 and earlier. Indeed, urllib3 2.x dropped support for OpenSSL 1.0.2, which reached end-of-life on January 1, 2020 (the same day as Python 2!). However, Amazon Linux 2 still supports that version of OpenSSL and will only reach end-of-life on June 30th, 2026. If your organization would benefit from the continued support of urllib3 1.x, please contact sethmichaellarson@gmail.com to discuss sponsorship or contribution opportunities.

Looking forward

We continue to improve and maintain this critical part of the Python ecosystem. Having funds to pay for maintainer time has allowed us to tackle difficult and tricky issues that no one was excited to do. The quality and reliability of urllib3 is the result of this kind of polish, something that everyone benefits from. We will continue this work, and our main focus for the coming year will continue to be HTTP/2.

If your employer relies on Python, please consider asking them for a small budget to help fund our work. Either reach out to us directly, or help fund us via https://opencollective.com/urllib3.

This post was written by Seth M. Larson, Quentin Pradet, Illia Volochii, with feedback from Andrey Petrov.

I'm on Mastodon!