optree v0.13.1

optree v0.13.1

What's New

• CI workflow improvements and minor fixes for typing annotation.

Added

• Upload coverage / JUnit results / core dumps in CI workflows by @XuehaiPan in #170 and #172.
• Add more info to tree_flatten_one_level by @XuehaiPan in #168.
• Improve typing support for generic PyTree[T] and registry lookup / register functions by @XuehaiPan in #160 and #166.

Changed

• Move include directory include/{ => optree}/*.h by @XuehaiPan in #167.

Fixed

• Improve typing support for optree.dataclasses.dataclass and optree.dataclasses.field by @manulari in #165.

Full Changelog: v0.13.0...v0.13.1

New Contributors

@manulari made their first contribution in #165.

optree v0.13.0

optree v0.13.0

What's New

• Add Python 3.13 and Python 3.13t (free-threading build) support.
• Add dataclasses integration.
• Significantly refactor and improve CXX build support.

Added

• Add Python 3.13t support by @XuehaiPan in #137.
• Expose Python implementation for C utilities for namedtuple and PyStructSequence by @XuehaiPan in #157.
• Add dataclasses integration by @XuehaiPan in #142.
• Add Python 3.13 support by @XuehaiPan in #156.
• Respect cmake variable pybind11_DIR by @XuehaiPan in #155.
• Add tests with PyDebug enabled in CI by @XuehaiPan in #150.

Changed

• Split implementation files and add more inline / constexpr / noexcept qualifiers by @XuehaiPan in #159.
• Use cmake's FindPython module by @XuehaiPan in #151.

Fixed

• Fix potential segmentation fault for structseq_fields cache support by @XuehaiPan in #150.

Full Changelog: v0.12.1...v0.13.0

optree v0.12.1

optree v0.12.1

Fixed

• Fix warning regression during import when launch with strict warning filters by @XuehaiPan in #149.

Full Changelog: v0.12.0...v0.12.1

optree v0.12.0

optree v0.12.0

Added

• Add context manager to temporarily set the dictionary sorting mode by @XuehaiPan in #147.
• Add PyPy support by @XuehaiPan in #145.
• Add 32-bit wheels for Linux and Windows by @XuehaiPan in #141.
• Add Linux ppc64le and s390x wheels by @XuehaiPan in #138.
• Add accessor APIs tree_flatten_with_accessor and PyTreeSpec.accessors by @XuehaiPan in #108.
• Add submodule optree.functools by @XuehaiPan in #134.

Changed

• Use stable tag instead of 2.12.0 for pybind11 version by @XuehaiPan in #146.
• Refactor the raw import statement in setup.py with importlib utilities by @XuehaiPan in #135.
• Update minimal version of typing-extensions to 4.5.0 for typing_extensions.deprecated by @XuehaiPan in #134.
• Update string representation for OrderedDict by @XuehaiPan in #133.

Fixed

• Fix gc for self-referential case by implementing tp_traverse by @XuehaiPan in #144.
• Fix potential segmentation fault for pickling support by @XuehaiPan in #143.
• Update CI runner image for Python 3.7 on macOS by @XuehaiPan in #135.

Removed

• Deprecate key path APIs by @XuehaiPan in #108.
• Deprecate optree.Partial and replace with optree.functools.partial by @XuehaiPan in #134.

Full Changelog: v0.11.0...v0.12.0

optree v0.11.0

optree v0.11.0

Added

• Add function is_namedtuple_instance and is_structseq_instance and result caches by @XuehaiPan in #121.
• Add tree_iter function by @XuehaiPan in #130.
• Add API to unregister node type in the registry by @XuehaiPan in #124.
• Add tree map functions with transposed outputs tree_transpose_map and tree_transpose_map_with_path by @XuehaiPan in #127.
• Add static constructors to create PyTreeSpec instances by @XuehaiPan in #120.
• Cache intermediate str objects in PyObject_GetAttr calls by @XuehaiPan in #106 and #109.
• Install clang-format and clang-tidy from PyPI by @XuehaiPan in #107.
• Also check _make and _asdict in function is_namedtuple_class by @XuehaiPan in #105.

Changed

• Set recursion limit to 1000 for all platforms by @XuehaiPan in #121.
• Allow types to be registered in both the global namespace and custom namespaces by @XuehaiPan in #124.
• Set treespec_is_leaf as strict by default by @XuehaiPan in #120.
• Reorder functions for better code correspondence between C++ and Python by @XuehaiPan in #117.
• Standardize py::handle and py::object usage in function signature by @XuehaiPan in #115.
• Reorder cases for namedtuple and PyStructSequence types by @XuehaiPan in #111.
• Use __bases__ rather than __base__ in function is_structseq_class by @XuehaiPan in #104.

Fixed

• Fix potential segmentation fault when modifying treespec.entries() by @XuehaiPan in #116.

Full Changelog: v0.10.0...v0.11.0

optree v0.10.0

optree v0.10.0

Added

• Add tree_ravel function for JAX/NumPy/PyTorch array/tensor tree manipulation by @XuehaiPan in #100.
• Expose node kind enum for PyTreeSpec by @XuehaiPan in #98.
• Expose function tree_flatten_one_level by @XuehaiPan in #101.
• Add tree broadcast functions broadcast_common, tree_broadcast_common, tree_broadcast_map, and tree_broadcast_map_with_path by @XuehaiPan in #87.
• Add function tree_is_leaf and add is_leaf argument to function all_leaves by @XuehaiPan in #93.
• Add methods PyTreeSpec.entry and PyTreeSpec.child by @XuehaiPan in #88.
• Add Python 3.12 support by @XuehaiPan in #90.
• Allow passing third-party dependency version from environment variable by @XuehaiPan in #80.

Changed

• Set recursion limit to 2000 for all platforms by @XuehaiPan in #97.
• Make PyTreeSpec.is_prefix to be consistent with PyTreeSpec.flatten_up_to by @XuehaiPan in #94.
• Decrease the MAX_RECURSION_DEPTH to 2000 on Windows by @XuehaiPan in #85.
• Bump abseil-cpp version to 20230802.1 by @XuehaiPan in #80.

Fixed

• Memorize ongoing repr / hash calls to resolve infinite recursion under self-referential case by @XuehaiPan and @JieRen98 in #82.

Removed

• Remove dependence on abseil-cpp by @XuehaiPan in #85.

Full Changelog: v0.9.2...v0.10.0

optree v0.9.2

Patch Release [0.9.2] - 2023-09-18

Changed

• Bump pybind11 version to 2.11.1 and add initial Python 3.12 support by @XuehaiPan in #78.
• Bump abseil-cpp version to 20230802.0 by @XuehaiPan in #79.

Fixed

• Fix empty paths when flatten with custom is_leaf function by @XuehaiPan in #76.

Full Changelog: v0.9.1...v0.9.2

optree v0.9.1

Patch Release [0.9.1] - 2023-05-23

Changed

• Use py::type::handle_of(obj) rather than deprecated obj.get_type() by @XuehaiPan in #49.
• Bump abseil-cpp version to 20230125.3 by @XuehaiPan in #57.

Fixed

• Add @runtime_checkable decorator for CustomTreeNode protocol class by @XuehaiPan in #56.

Full Changelog: v0.9.0...v0.9.1

optree v0.9.0

What's New

Now optree preserves dict / defaultdict key order in the output of tree_unflatten, tree_map, and tree_map_with_path.

This behavior needs extra operations during flattening/unflattening. There would be performance regression for dict and defaultdict. No impact is added for other collection types, such as OrderedDict, list, tuple.

optree v0.8.0: the output dict from tree_unflatten is stored in sorted order.

tree_flatten and tree_unflatten will lose the information above the insertion order of the input dict. Map over a dict with the identity function will change the key order.

>>> optree.tree_map(lambda x: x, {'b': 2, 'a': 1})
{'a': 1, 'b': 2}

optree v0.9.0: the output dict from tree_unflatten will have the consistent key order with the input dict.

>>> optree.tree_map(lambda x: x, {'b': 2, 'a': 1})
{'b': 2, 'a': 1}

As the key order is preserved, it's safe to use tree_map to process "unordered dict" (builtins.dict) (e.g., **kwargs in functions). In this case, users no longer need to convert dict to OrderedDict manually.

def func(*args, **kwargs):
    args, kwargs = optree.tree_map(do_something, (args, kwargs))
    # optree v0.8.0: args, kwargs = optree.tree_map(do_something, (args, OrderedDict(kwargs)))
    ...

Note that tree_map still maps the leaves in sorted key order (the same order as tree_flatten and tree_leaves).

>>> leaves = [] 
...
... def add_leaves(x):
...     leaves.append(x)
...     return x
...
>>> ptree.tree_map(add_leaves, {'b': 2, 'a': 1})
{'b': 2, 'a': 1}
>>> leaves
[1, 2]

Breaking Changes

Revert "Change keyword argument initial to initializer for tree_reduce to align with functools.reduce". The previous change is reverted due to a documentation issue for Python (see also python/cpython#102757 and python/cpython#102759). The argument name is changed back to initial in tree_reduce. Users are recommended to use positional arguments.

Full Changelog [0.9.0] - 2023-03-23

Added

• Preserve dict key order in the output of tree_unflatten, tree_map, and tree_map_with_path by @XuehaiPan in #46.

Changed

• Change keyword argument initializer back to initial for tree_reduce to align with functools.reduce C implementation by @XuehaiPan in #47.

Full Changelog: v0.8.0...v0.9.0

optree v0.8.0

What's New

1. Add new and refactor utility functions for namedtuple and PyStructSequence types.

   • is_namedtuple
   • is_namedtuple_class
   • is_structseq
   • is_structseq_class
   • namedtuple_fields (new)
   • structseq_fields

2. Add more methods and properties for PyTreeSpec class.

   • PyTreeSpec.is_prefix (treespec_is_prefix and __le__ / __lt__)
   • PyTreeSpec.is_suffix (treespec_is_suffix and __ge__ / __gt__)
   • PyTreeSpec.paths (treespec_paths)
   • PyTreeSpec.entries (treespec_entries)

3. Add tree reduce functions.

   • tree_sum
   • tree_max
   • tree_min

4. Miscellaneous changes.

   • Rephrase error messages.

     >>> optree.tree_map(lambda x, y: x + y, {'a': 1, 'b': 2}, {'b': 3, 'c': 4})
     Traceback (most recent call last):
         ...
     ValueError: dictionary key mismatch; expected key(s): ['a', 'b'], got key(s): ['b', 'c'], missing key(s): ['a'], extra key(s): ['c']; dict: {'b': 3, 'c': 4}.

   • Add function tree_broadcast_prefix.

   • Linter integration for flake8 plugins and ruff.

Breaking Changes

1. PyTreeSpec.flatten_up_to now allows ordered (OrderedDict) and unordered (dict and defaultdict) dictionaries mutually exchangeable.

   This behavior will affect all functions based on flatten_up_to, such as tree_map, tree_map_with_path, tree_broadcast_prefix.
   Calling multi-treemap with mixed inputs of ordered and unordered dictionaries will no longer raise ValueError.

   optree v0.7.0: raise ValueError if the dictionary type mismatch

   >>> from collections import *
   >>> import optree
   >>> d = {'a': 1, 'b': 2, 'c': 3}
   >>> od = OrderedDict([('b', 5), ('c', 6), ('a', 4)])
   >>> dd = defaultdict(int, {'c': 9, 'a': 7, 'b': 8})
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, d, od, dd)
   Traceback (most recent call last):
       ...
   ValueError: Expected dict, got OrderedDict([('b', 5), ('c', 6), ('a', 4)]).
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, od, dd, d)
   Traceback (most recent call last):
       ...
   ValueError: Expected collections.OrderedDict, got defaultdict(<class 'int'>, {'c': 9, 'a': 7, 'b': 8}).
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, od, dd, d)
   Traceback (most recent call last):
       ...
   ValueError: Expected collections.defaultdict, got {'a': 1, 'b': 2, 'c': 3}.

   optree v0.8.0: allow mixed input of dict, OrderedDict, and defaultdict.
   The result pytree type and key ordering are determined by the first input pytree.

   >>> from collections import *
   >>> import optree
   >>> d = {'a': 1, 'b': 2, 'c': 3}
   >>> od = OrderedDict([('b', 5), ('c', 6), ('a', 4)])
   >>> dd = defaultdict(int, {'c': 9, 'a': 7, 'b': 8})
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, d, od, dd)
   {'a': 147, 'b': 258, 'c': 369}
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, od, dd, d)
   OrderedDict([('b', 582), ('c', 693), ('a', 471)])
   >>> optree.tree_map(lambda x, y, z: 100 * x + 10 * y + z, dd, d, od)
   defaultdict(<class 'int'>, {'a': 714, 'b': 825, 'c': 936})

2. Use more appropriate exception types.

   • structseq_fields now raises TypeError rather than ValueError when got non-PyStructSequence types

     optree v0.7.0: raise ValueError when got non-PyStructSequence types

     >>> from collections import *
     >>> import optree
     >>> optree.structseq_fields((1, 2))
     Traceback (most recent call last):
         ...
     ValueError: Expected StructSequence, got (1, 2).
     >>> mytuple = namedtuple('mytuple', ['a', 'b'])
     >>> optree.structseq_fields(mytuple(1, 2))
     Traceback (most recent call last):
         ...
     ValueError: Expected StructSequence, got mytuple(a=1, b=2).

     optree v0.8.0: raise TypeError when got non-PyStructSequence types

     >>> from collections import *
     >>> import optree
     >>> optree.structseq_fields((1, 2))
     Traceback (most recent call last):
         ...
     TypeError: Expected an instance of PyStructSequence type, got (1, 2).
     >>> mytuple = namedtuple('mytuple', ['a', 'b'])
     >>> optree.structseq_fields(mytuple(1, 2))
     Traceback (most recent call last):
         ...
     TypeError: Expected an instance of PyStructSequence type, got mytuple(a=1, b=2).

   • optree._C.InternalError now inherits from SystemError rather than RuntimeError.

     optree v0.7.0: need to use RuntimeError to catch InternalError

     >>> import optree
     >>> optree._C.InternalError.mro()
     [<class 'optree._C.InternalError'>, <class 'RuntimeError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>]

     optree v0.8.0: need to use SystemError to catch InternalError

     >>> import optree
     >>> optree._C.InternalError.mro()
     [<class 'optree._C.InternalError'>, <class 'SystemError'>, <class 'Exception'>, <class 'BaseException'>, <class 'object'>]

3. Change keyword argument initial to initializer for tree_reduce to align with functools.reduce.

Full Changelog [0.8.0] - 2023-03-14

Added

• Add methods PyTreeSpec.paths and PyTreeSpec.entries by @XuehaiPan in #43.
• Allow tree-map with mixed inputs of ordered and unordered dictionaries by @XuehaiPan in #42.
• Add more utility functions for namedtuple and PyStructSequence type by @XuehaiPan in #41.
• Add methods PyTreeSpec.is_prefix and PyTreeSpec.is_suffix and function tree_broadcast_prefix by @XuehaiPan in #40.
• Add tree reduce functions tree_sum, tree_max, and tree_min by @XuehaiPan in #39.
• Test dict key equality with PyDict_Contains ($O (n)$) rather than sorting ($O (n \log n)$) by @XuehaiPan in #37.
• Make error message more clear when value mismatch by @XuehaiPan in #36.
• Add ruff and flake8 plugins integration by @XuehaiPan in #33.

Changed

• Allow tree-map with mixed inputs of ordered and unordered dictionaries by @XuehaiPan in #42.
• Use more appropriate exception handling (e.g., change ValueError to TypeError in structseq_fields) by @XuehaiPan in #41.
• Inherit optree._C.InternalError from SystemError rather than RuntimeError by @XuehaiPan in #41.
• Change keyword argument initial to initializer for tree_reduce to align with functools.reduce by @XuehaiPan in #39.

Full Changelog: v0.7.0...v0.8.0