Repolode
Submit
cloudpipe

cloudpipe/cloudpickle

Extends Python's pickle to serialize lambdas, interactively defined functions and classes, for shipping code to remote workers.

View on GitHub
Stars
1,947
Forks
212
Open issues
124
Last commit
last month

What is it?

cloudpickle is a Python serialization library that handles constructs the standard pickle module cannot. It can serialize lambda functions and functions or classes defined in the __main__ module, such as in a script, a shell or a Jupyter notebook.

The main use case is cluster computing, where Python code is sent over the network to run on remote hosts, often close to the data. Instead of pickling functions by reference to their module, cloudpickle can serialize them by value, and since version 2.0.0 developers can explicitly register modules for by-value serialization so workers do not need the module installed.

Serialized data can only be exchanged between the same Python version, and long-term storage is not supported. Because unpickling can run arbitrary code, data should only be loaded from trusted sources.

Key features

  • 01Serializes lambda functions and interactively defined functions and classes.
  • 02Supports pickling functions and classes by value rather than by reference.
  • 03Lets developers register modules for by-value serialization via register_pickle_by_value.
  • 04Works as a drop-in extension of the standard pickle interface with dumps and loads.

Use cases

  • Sending Python functions to remote workers in distributed computing frameworks.
  • Passing lambdas or notebook-defined functions to a parallel processing job.
  • Developing modules whose code must be updated on workers without reinstalling them.

Quick start

  1. 1

    Install cloudpickle from PyPI

    pip install cloudpickle

Things to know

  • Only sends objects between the exact same Python version.
  • Not supported for long-term object storage.
  • Only load pickle data from trusted sources, since unpickling can run arbitrary code.
  • Serialization by value for importable modules is experimental.

Topics

#python#serialization#pickle#distributed-computing#cluster-computing

FAQ

What is cloudpickle?
Extends Python's pickle to serialize lambdas, interactively defined functions and classes, for shipping code to remote workers. cloudpickle is a Python serialization library that handles constructs the standard pickle module cannot. It can serialize lambda functions and functions or classes defined in the __main__ module, such as in a script, a shell or a Jupyter notebook.
Is cloudpickle free to use? What license does it have?
cloudpickle is open source under the Other license. Check the license terms before using it in commercial products.
Is cloudpickle still maintained?
Updated within 6 months. The last commit to cloudpickle was on Sep 10, 2026.
What can I use cloudpickle for?
Sending Python functions to remote workers in distributed computing frameworks. Passing lambdas or notebook-defined functions to a parallel processing job. Developing modules whose code must be updated on workers without reinstalling them.
How do I install cloudpickle?
Install cloudpickle from PyPI: pip install cloudpickle
Where can I get cloudpickle?
The source code and installation instructions are on GitHub: https://github.com/cloudpipe/cloudpickle

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects