Appropriate way to use buffer protocol objects from rust embedded into python, passing data back to python?

[Cazadorro]

How do we take large arrays of structs, say a structs that represents vec3s or other more complicated but also types made up of mostly primitives, and use them in the python side with out copying? I believe buffer protocol is the answer, and I see there's buffer protocol things in pyo3, but very little explanation of how to actually use them.

• https://pyo3.rs/main/class/protocols
• https://pyo3.rs/main/doc/pyo3/types/struct.pymemoryview#method.from_bound

We have an array of vec3s we'd like to send to a python function callback stored in the rust side of our program. Despite being a pyclass marked struct, and indirectly allocated from the python side, it doesn't appear like we can send it directly to a python function (I guess because nested python classes in a larger class aren't themselves wrapped a py-references?). The cost for re-allocating data and destroying it constantly slows the program down by 100x->10000x, so is not acceptable. We'd like to just be able to give a view of the array of memory to the python functions, but we are struggling with how to square this with using pyo3.

[davidhewitt]

Sorry for the slow reply. I think also relevant here is #3148, which just needs a bit of elbow grease to get across the line. The existing buffer implementation in PyO3 predates PEP 688 and is very low-level.