METADATA 7.9 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265
  1. Metadata-Version: 2.4
  2. Name: msgpack
  3. Version: 1.1.2
  4. Summary: MessagePack serializer
  5. Author-email: Inada Naoki <songofacandy@gmail.com>
  6. License-Expression: Apache-2.0
  7. Project-URL: Homepage, https://msgpack.org/
  8. Project-URL: Documentation, https://msgpack-python.readthedocs.io/
  9. Project-URL: Repository, https://github.com/msgpack/msgpack-python/
  10. Project-URL: Tracker, https://github.com/msgpack/msgpack-python/issues
  11. Project-URL: Changelog, https://github.com/msgpack/msgpack-python/blob/main/ChangeLog.rst
  12. Keywords: msgpack,messagepack,serializer,serialization,binary
  13. Classifier: Development Status :: 5 - Production/Stable
  14. Classifier: Operating System :: OS Independent
  15. Classifier: Topic :: File Formats
  16. Classifier: Intended Audience :: Developers
  17. Classifier: Programming Language :: Python :: Implementation :: CPython
  18. Classifier: Programming Language :: Python :: Implementation :: PyPy
  19. Requires-Python: >=3.9
  20. Description-Content-Type: text/markdown
  21. License-File: COPYING
  22. Dynamic: license-file
  23. # MessagePack for Python
  24. [![Build Status](https://github.com/msgpack/msgpack-python/actions/workflows/wheel.yml/badge.svg)](https://github.com/msgpack/msgpack-python/actions/workflows/wheel.yml)
  25. [![Documentation Status](https://readthedocs.org/projects/msgpack-python/badge/?version=latest)](https://msgpack-python.readthedocs.io/en/latest/?badge=latest)
  26. ## What is this?
  27. [MessagePack](https://msgpack.org/) is an efficient binary serialization format.
  28. It lets you exchange data among multiple languages like JSON.
  29. But it's faster and smaller.
  30. This package provides CPython bindings for reading and writing MessagePack data.
  31. ## Install
  32. ```
  33. $ pip install msgpack
  34. ```
  35. ### Pure Python implementation
  36. The extension module in msgpack (`msgpack._cmsgpack`) does not support PyPy.
  37. But msgpack provides a pure Python implementation (`msgpack.fallback`) for PyPy.
  38. ### Windows
  39. If you can't use a binary distribution, you need to install Visual Studio
  40. or the Windows SDK on Windows.
  41. Without the extension, the pure Python implementation on CPython runs slowly.
  42. ## How to use
  43. ### One-shot pack & unpack
  44. Use `packb` for packing and `unpackb` for unpacking.
  45. msgpack provides `dumps` and `loads` as aliases for compatibility with
  46. `json` and `pickle`.
  47. `pack` and `dump` pack to a file-like object.
  48. `unpack` and `load` unpack from a file-like object.
  49. ```pycon
  50. >>> import msgpack
  51. >>> msgpack.packb([1, 2, 3])
  52. '\x93\x01\x02\x03'
  53. >>> msgpack.unpackb(_)
  54. [1, 2, 3]
  55. ```
  56. Read the docstring for options.
  57. ### Streaming unpacking
  58. `Unpacker` is a "streaming unpacker". It unpacks multiple objects from one
  59. stream (or from bytes provided through its `feed` method).
  60. ```py
  61. import msgpack
  62. from io import BytesIO
  63. buf = BytesIO()
  64. for i in range(100):
  65. buf.write(msgpack.packb(i))
  66. buf.seek(0)
  67. unpacker = msgpack.Unpacker(buf)
  68. for unpacked in unpacker:
  69. print(unpacked)
  70. ```
  71. ### Packing/unpacking of custom data types
  72. It is also possible to pack/unpack custom data types. Here is an example for
  73. `datetime.datetime`.
  74. ```py
  75. import datetime
  76. import msgpack
  77. useful_dict = {
  78. "id": 1,
  79. "created": datetime.datetime.now(),
  80. }
  81. def decode_datetime(obj):
  82. if '__datetime__' in obj:
  83. obj = datetime.datetime.strptime(obj["as_str"], "%Y%m%dT%H:%M:%S.%f")
  84. return obj
  85. def encode_datetime(obj):
  86. if isinstance(obj, datetime.datetime):
  87. return {'__datetime__': True, 'as_str': obj.strftime("%Y%m%dT%H:%M:%S.%f")}
  88. return obj
  89. packed_dict = msgpack.packb(useful_dict, default=encode_datetime)
  90. this_dict_again = msgpack.unpackb(packed_dict, object_hook=decode_datetime)
  91. ```
  92. `Unpacker`'s `object_hook` callback receives a dict; the
  93. `object_pairs_hook` callback may instead be used to receive a list of
  94. key-value pairs.
  95. NOTE: msgpack can encode datetime with tzinfo into standard ext type for now.
  96. See `datetime` option in `Packer` docstring.
  97. ### Extended types
  98. It is also possible to pack/unpack custom data types using the **ext** type.
  99. ```pycon
  100. >>> import msgpack
  101. >>> import array
  102. >>> def default(obj):
  103. ... if isinstance(obj, array.array) and obj.typecode == 'd':
  104. ... return msgpack.ExtType(42, obj.tostring())
  105. ... raise TypeError("Unknown type: %r" % (obj,))
  106. ...
  107. >>> def ext_hook(code, data):
  108. ... if code == 42:
  109. ... a = array.array('d')
  110. ... a.fromstring(data)
  111. ... return a
  112. ... return ExtType(code, data)
  113. ...
  114. >>> data = array.array('d', [1.2, 3.4])
  115. >>> packed = msgpack.packb(data, default=default)
  116. >>> unpacked = msgpack.unpackb(packed, ext_hook=ext_hook)
  117. >>> data == unpacked
  118. True
  119. ```
  120. ### Advanced unpacking control
  121. As an alternative to iteration, `Unpacker` objects provide `unpack`,
  122. `skip`, `read_array_header`, and `read_map_header` methods. The former two
  123. read an entire message from the stream, respectively deserializing and returning
  124. the result, or ignoring it. The latter two methods return the number of elements
  125. in the upcoming container, so that each element in an array, or key-value pair
  126. in a map, can be unpacked or skipped individually.
  127. ## Notes
  128. ### String and binary types in the old MessagePack spec
  129. Early versions of msgpack didn't distinguish string and binary types.
  130. The type for representing both string and binary types was named **raw**.
  131. You can pack into and unpack from this old spec using `use_bin_type=False`
  132. and `raw=True` options.
  133. ```pycon
  134. >>> import msgpack
  135. >>> msgpack.unpackb(msgpack.packb([b'spam', 'eggs'], use_bin_type=False), raw=True)
  136. [b'spam', b'eggs']
  137. >>> msgpack.unpackb(msgpack.packb([b'spam', 'eggs'], use_bin_type=True), raw=False)
  138. [b'spam', 'eggs']
  139. ```
  140. ### ext type
  141. To use the **ext** type, pass a `msgpack.ExtType` object to the packer.
  142. ```pycon
  143. >>> import msgpack
  144. >>> packed = msgpack.packb(msgpack.ExtType(42, b'xyzzy'))
  145. >>> msgpack.unpackb(packed)
  146. ExtType(code=42, data='xyzzy')
  147. ```
  148. You can use it with `default` and `ext_hook`. See below.
  149. ### Security
  150. When unpacking data received from an unreliable source, msgpack provides
  151. two security options.
  152. `max_buffer_size` (default: `100*1024*1024`) limits the internal buffer size.
  153. It is also used to limit preallocated list sizes.
  154. `strict_map_key` (default: `True`) limits the type of map keys to bytes and str.
  155. While the MessagePack spec doesn't limit map key types,
  156. there is a risk of a hash DoS.
  157. If you need to support other types for map keys, use `strict_map_key=False`.
  158. ### Performance tips
  159. CPython's GC starts when the number of allocated objects grows.
  160. This means unpacking may trigger unnecessary GC.
  161. You can use `gc.disable()` when unpacking a large message.
  162. A list is the default sequence type in Python.
  163. However, a tuple is lighter than a list.
  164. You can use `use_list=False` while unpacking when performance is important.
  165. ## Major breaking changes in the history
  166. ### msgpack 0.5
  167. The package name on PyPI was changed from `msgpack-python` to `msgpack` in 0.5.
  168. When upgrading from msgpack-0.4 or earlier, do `pip uninstall msgpack-python` before
  169. `pip install -U msgpack`.
  170. ### msgpack 1.0
  171. * Python 2 support
  172. * The extension module no longer supports Python 2.
  173. The pure Python implementation (`msgpack.fallback`) is used for Python 2.
  174. * msgpack 1.0.6 drops official support of Python 2.7, as pip and
  175. GitHub Action "setup-python" no longer supports Python 2.7.
  176. * Packer
  177. * Packer uses `use_bin_type=True` by default.
  178. Bytes are encoded in the bin type in MessagePack.
  179. * The `encoding` option is removed. UTF-8 is always used.
  180. * Unpacker
  181. * Unpacker uses `raw=False` by default. It assumes str values are valid UTF-8 strings
  182. and decodes them to Python str (Unicode) objects.
  183. * `encoding` option is removed. You can use `raw=True` to support old format (e.g. unpack into bytes, not str).
  184. * The default value of `max_buffer_size` is changed from 0 to 100 MiB to avoid DoS attacks.
  185. You need to pass `max_buffer_size=0` if you have large but safe data.
  186. * The default value of `strict_map_key` is changed to True to avoid hash DoS.
  187. You need to pass `strict_map_key=False` if you have data that contain map keys
  188. whose type is neither bytes nor str.