| 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265 |
- Metadata-Version: 2.4
- Name: msgpack
- Version: 1.1.2
- Summary: MessagePack serializer
- Author-email: Inada Naoki <songofacandy@gmail.com>
- License-Expression: Apache-2.0
- Project-URL: Homepage, https://msgpack.org/
- Project-URL: Documentation, https://msgpack-python.readthedocs.io/
- Project-URL: Repository, https://github.com/msgpack/msgpack-python/
- Project-URL: Tracker, https://github.com/msgpack/msgpack-python/issues
- Project-URL: Changelog, https://github.com/msgpack/msgpack-python/blob/main/ChangeLog.rst
- Keywords: msgpack,messagepack,serializer,serialization,binary
- Classifier: Development Status :: 5 - Production/Stable
- Classifier: Operating System :: OS Independent
- Classifier: Topic :: File Formats
- Classifier: Intended Audience :: Developers
- Classifier: Programming Language :: Python :: Implementation :: CPython
- Classifier: Programming Language :: Python :: Implementation :: PyPy
- Requires-Python: >=3.9
- Description-Content-Type: text/markdown
- License-File: COPYING
- Dynamic: license-file
- # MessagePack for Python
- [](https://github.com/msgpack/msgpack-python/actions/workflows/wheel.yml)
- [](https://msgpack-python.readthedocs.io/en/latest/?badge=latest)
- ## What is this?
- [MessagePack](https://msgpack.org/) is an efficient binary serialization format.
- It lets you exchange data among multiple languages like JSON.
- But it's faster and smaller.
- This package provides CPython bindings for reading and writing MessagePack data.
- ## Install
- ```
- $ pip install msgpack
- ```
- ### Pure Python implementation
- The extension module in msgpack (`msgpack._cmsgpack`) does not support PyPy.
- But msgpack provides a pure Python implementation (`msgpack.fallback`) for PyPy.
- ### Windows
- If you can't use a binary distribution, you need to install Visual Studio
- or the Windows SDK on Windows.
- Without the extension, the pure Python implementation on CPython runs slowly.
- ## How to use
- ### One-shot pack & unpack
- Use `packb` for packing and `unpackb` for unpacking.
- msgpack provides `dumps` and `loads` as aliases for compatibility with
- `json` and `pickle`.
- `pack` and `dump` pack to a file-like object.
- `unpack` and `load` unpack from a file-like object.
- ```pycon
- >>> import msgpack
- >>> msgpack.packb([1, 2, 3])
- '\x93\x01\x02\x03'
- >>> msgpack.unpackb(_)
- [1, 2, 3]
- ```
- Read the docstring for options.
- ### Streaming unpacking
- `Unpacker` is a "streaming unpacker". It unpacks multiple objects from one
- stream (or from bytes provided through its `feed` method).
- ```py
- import msgpack
- from io import BytesIO
- buf = BytesIO()
- for i in range(100):
- buf.write(msgpack.packb(i))
- buf.seek(0)
- unpacker = msgpack.Unpacker(buf)
- for unpacked in unpacker:
- print(unpacked)
- ```
- ### Packing/unpacking of custom data types
- It is also possible to pack/unpack custom data types. Here is an example for
- `datetime.datetime`.
- ```py
- import datetime
- import msgpack
- useful_dict = {
- "id": 1,
- "created": datetime.datetime.now(),
- }
- def decode_datetime(obj):
- if '__datetime__' in obj:
- obj = datetime.datetime.strptime(obj["as_str"], "%Y%m%dT%H:%M:%S.%f")
- return obj
- def encode_datetime(obj):
- if isinstance(obj, datetime.datetime):
- return {'__datetime__': True, 'as_str': obj.strftime("%Y%m%dT%H:%M:%S.%f")}
- return obj
- packed_dict = msgpack.packb(useful_dict, default=encode_datetime)
- this_dict_again = msgpack.unpackb(packed_dict, object_hook=decode_datetime)
- ```
- `Unpacker`'s `object_hook` callback receives a dict; the
- `object_pairs_hook` callback may instead be used to receive a list of
- key-value pairs.
- NOTE: msgpack can encode datetime with tzinfo into standard ext type for now.
- See `datetime` option in `Packer` docstring.
- ### Extended types
- It is also possible to pack/unpack custom data types using the **ext** type.
- ```pycon
- >>> import msgpack
- >>> import array
- >>> def default(obj):
- ... if isinstance(obj, array.array) and obj.typecode == 'd':
- ... return msgpack.ExtType(42, obj.tostring())
- ... raise TypeError("Unknown type: %r" % (obj,))
- ...
- >>> def ext_hook(code, data):
- ... if code == 42:
- ... a = array.array('d')
- ... a.fromstring(data)
- ... return a
- ... return ExtType(code, data)
- ...
- >>> data = array.array('d', [1.2, 3.4])
- >>> packed = msgpack.packb(data, default=default)
- >>> unpacked = msgpack.unpackb(packed, ext_hook=ext_hook)
- >>> data == unpacked
- True
- ```
- ### Advanced unpacking control
- As an alternative to iteration, `Unpacker` objects provide `unpack`,
- `skip`, `read_array_header`, and `read_map_header` methods. The former two
- read an entire message from the stream, respectively deserializing and returning
- the result, or ignoring it. The latter two methods return the number of elements
- in the upcoming container, so that each element in an array, or key-value pair
- in a map, can be unpacked or skipped individually.
- ## Notes
- ### String and binary types in the old MessagePack spec
- Early versions of msgpack didn't distinguish string and binary types.
- The type for representing both string and binary types was named **raw**.
- You can pack into and unpack from this old spec using `use_bin_type=False`
- and `raw=True` options.
- ```pycon
- >>> import msgpack
- >>> msgpack.unpackb(msgpack.packb([b'spam', 'eggs'], use_bin_type=False), raw=True)
- [b'spam', b'eggs']
- >>> msgpack.unpackb(msgpack.packb([b'spam', 'eggs'], use_bin_type=True), raw=False)
- [b'spam', 'eggs']
- ```
- ### ext type
- To use the **ext** type, pass a `msgpack.ExtType` object to the packer.
- ```pycon
- >>> import msgpack
- >>> packed = msgpack.packb(msgpack.ExtType(42, b'xyzzy'))
- >>> msgpack.unpackb(packed)
- ExtType(code=42, data='xyzzy')
- ```
- You can use it with `default` and `ext_hook`. See below.
- ### Security
- When unpacking data received from an unreliable source, msgpack provides
- two security options.
- `max_buffer_size` (default: `100*1024*1024`) limits the internal buffer size.
- It is also used to limit preallocated list sizes.
- `strict_map_key` (default: `True`) limits the type of map keys to bytes and str.
- While the MessagePack spec doesn't limit map key types,
- there is a risk of a hash DoS.
- If you need to support other types for map keys, use `strict_map_key=False`.
- ### Performance tips
- CPython's GC starts when the number of allocated objects grows.
- This means unpacking may trigger unnecessary GC.
- You can use `gc.disable()` when unpacking a large message.
- A list is the default sequence type in Python.
- However, a tuple is lighter than a list.
- You can use `use_list=False` while unpacking when performance is important.
- ## Major breaking changes in the history
- ### msgpack 0.5
- The package name on PyPI was changed from `msgpack-python` to `msgpack` in 0.5.
- When upgrading from msgpack-0.4 or earlier, do `pip uninstall msgpack-python` before
- `pip install -U msgpack`.
- ### msgpack 1.0
- * Python 2 support
- * The extension module no longer supports Python 2.
- The pure Python implementation (`msgpack.fallback`) is used for Python 2.
-
- * msgpack 1.0.6 drops official support of Python 2.7, as pip and
- GitHub Action "setup-python" no longer supports Python 2.7.
- * Packer
- * Packer uses `use_bin_type=True` by default.
- Bytes are encoded in the bin type in MessagePack.
- * The `encoding` option is removed. UTF-8 is always used.
- * Unpacker
- * Unpacker uses `raw=False` by default. It assumes str values are valid UTF-8 strings
- and decodes them to Python str (Unicode) objects.
- * `encoding` option is removed. You can use `raw=True` to support old format (e.g. unpack into bytes, not str).
- * The default value of `max_buffer_size` is changed from 0 to 100 MiB to avoid DoS attacks.
- You need to pass `max_buffer_size=0` if you have large but safe data.
- * The default value of `strict_map_key` is changed to True to avoid hash DoS.
- You need to pass `strict_map_key=False` if you have data that contain map keys
- whose type is neither bytes nor str.
|