gh-125420: implement Sequence.__contains__ API on memoryview objects
#125441
Conversation
picnixz
commented
Oct 14, 2024
•
picnixz
commented
- Issue: memoryview is a Sequence but does not implement the full Sequence API #125420
picnixz
changed the title
__contains__ to memoryview objects__contains__ to memoryview objects
picnixz
changed the title
__contains__ to memoryview objectsSequence.__contains__ API on memoryview objects
|
Converting into a draft to decide whether the pure iterator approach is actually efficient enough or if iteration using the underlying multi-dimensional structure would be preferred (without spawning an iterator). |
|
For now, let's just keep this implementation. It's probably faster than the generic implementation since there are less code paths but it could definitely be faster by iterating directly over the buffer (and we don't seem to support iterating over multi-dimensional buffers yet). |
| static int | ||
| memory_contains(PyObject *self, PyObject *value) | ||
| { | ||
| PyObject *iter = PyObject_GetIter(self); |
There was a problem hiding this comment.
I think it might be good to follow the pattern of already implemented methods and have it a bit faster.
Creating iterators when they aren't needed might not be the best option in tensor-like-object.
E.g. I don't think numpy does obj in array.flat for a in array.
I think it is much easier to do it from the beginning compared to all convincing that will be required to change this later.
There was a problem hiding this comment.
I think the best reference for such decisions for memoryview is numpy.array.
Although it is still very primitive in comparison, but it is possible this will change with time.
There was a problem hiding this comment.
As I said on the issue, performances should be addressed in a follow-up PR. Note that this pattern is the pattern used by list objects.
There was a problem hiding this comment.
As I said on the issue, performances should be addressed in a follow-up PR.
I don't think this is only about the performance, but also about the design.
Also, such follow-up PR would pretty much replace the whole implementation of this PR.
Note that this pattern is the pattern used by list objects.
What I am suggesting is that memoryview should be modelled after tensor-like objects and not CPython sequences such as list or tuple. It is slightly different breed, IMO.
There was a problem hiding this comment.
For now, memoryviews are only 1-dimensional =/ we don't have multi-dimensional slicing. And it's been like this for ages. Tensors are generally used for generalizing vectors and matrices, but for now, we do not support them at all.
I can try to use lower-level calls, but this would overcomplicate the implementation itself. My original idea was to inline most of the calls to avoid materializing an iterator and advance item by item manually. But I expected the implementation to be harder to maintain. As Jelle said, in already worked because it delegated to PySequence_Contains. What we needed in this PR was to have an explicit Sequence.__contains__ (same for Sequence.__reversed__ of the other PR).
There was a problem hiding this comment.
No, because the "natural evolution" hasn't changed for the past 10 years...
There was a problem hiding this comment.
And I personally don't mind updating what I wrote later.
There was a problem hiding this comment.
No, because the "natural evolution" hasn't changed for the past 10 years...
It did not evolve much over past 10 years, yes. From my POV, past might not be a very good indicator for the future in this case. I think there are some overlooked opportunities here.
I am personally interested in multi-dimensional slicing of memoryview and there is a reasonable chance that I will make some attempts in reasonably near future. I have been thinking about it approximately since https://discuss.python.org/t/memoryview-multi-dimensional-slicing-support/52776.
And I personally don't mind updating what I wrote later.
This doesn't make much difference, whether it is you or someone else, the biggest cost here is new PR, review process, time delays, etc. Implementation itself would be a minor issue here.
There was a problem hiding this comment.
past might not be a very good indicator for the future in this case.
Unfortunately, it is for most core devs. I will not change my stance on this matter for now, because my aim was to port the implicit dispatch (remember that in already works because it dispatches to Sequence_Contains) to an explicit one. If you want an implementation using the index-based approach, feel free to open an alternate PR.
There was a problem hiding this comment.
It is not as big of a deal as it might seem that I am trying to make. :) Given it happens, this will most likely be changed as part of m-dim-slicing by whoever will be doing that.
But I am still finding trouble to see why not implement it more correctly from the beginning given such low cost. :)
|
Since we discussed about adding this one if needs arise, I will just close them and delete my branch for now (I have too many local branches and I don't want to keep stale PRs). We'll be able to reuse it later if needed. |
