Repolode
Submit
ross

ross/requests-futures

A small add-on for Python's requests library that sends HTTP requests asynchronously using concurrent.futures.

View on GitHub
Stars
2,222
Forks
158
Open issues
1
Last commit
16 days ago

What is it?

requests-futures adds an asynchronous interface to the popular Python requests library. Instead of blocking on each call, FuturesSession returns Future objects, so requests run in the background and you call result() when you need the response. The API otherwise matches requests.Session, with exceptions raised from result() rather than at the call site.

By default the session uses a ThreadPoolExecutor with 8 workers, and a ProcessPoolExecutor can be supplied instead for process-based concurrency. Closing the session as a context manager cancels any queued requests that have not started yet. Extra data can be attached to futures, and the library provides hooks as a replacement for the deprecated background_callback.

The default executor shares one requests.Session across worker threads, and Session is not thread-safe, so mutating shared cookies or headers across in-flight requests needs care. Requires Python 3.10 or newer.

Key features

  • 01Returns Future objects from the familiar requests.Session API.
  • 02Defaults to a ThreadPoolExecutor with 8 workers, and supports ProcessPoolExecutor.
  • 03Cancels queued requests that have not started when the session closes.
  • 04Lets you attach custom data to futures for tracking which request is which.
  • 05Supports hooks as the replacement for the deprecated background_callback.

Use cases

  • Fetching many independent URLs in parallel from a Python script.
  • Adding background HTTP calls to an existing requests-based codebase with minimal changes.
  • Running large batches of requests where per-request memory use calls for a process pool.

Quick start

  1. 1

    Install the package with pip

    pip install requests-futures
  2. 2

    Create a FuturesSession and send requests; call result() on each future to get the response

Things to know

  • Requires Python 3.10 or newer.
  • The default executor shares one requests.Session across threads, which is not thread-safe.
  • The license is not clearly specified in the project metadata.

Topics

#python#http#asynchronous#concurrency#requests#futures#threading

FAQ

What is requests-futures?
A small add-on for Python's requests library that sends HTTP requests asynchronously using concurrent.futures. requests-futures adds an asynchronous interface to the popular Python requests library. Instead of blocking on each call, FuturesSession returns Future objects, so requests run in the background and you call result() when you need the response. The API otherwise matches requests.Session, with exceptions raised from result() rather than at the call site.
Is requests-futures free to use? What license does it have?
requests-futures is open source under the Other license. Check the license terms before using it in commercial products.
Is requests-futures still maintained?
Actively maintained. The last commit to requests-futures was on Oct 5, 2026.
What can I use requests-futures for?
Fetching many independent URLs in parallel from a Python script. Adding background HTTP calls to an existing requests-based codebase with minimal changes. Running large batches of requests where per-request memory use calls for a process pool.
How do I install requests-futures?
Install the package with pip: pip install requests-futures Create a FuturesSession and send requests; call result() on each future to get the response
Where can I get requests-futures?
The source code and installation instructions are on GitHub: https://github.com/ross/requests-futures

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects