Performance Tip for calling py object's method

[noahkim0723]

Let me share the tips to improve performance for python object's method in particular cases.

I found this while working on performance benchmark between pyo3 and C++ swig extension for python in the firm

Let's say we want to define a caller class that receives multiple py objects and then call their methods with the same signature as follows:

#[pyclass]
struct Caller {
    py_objects: Vec<PyObject>,
}

#[pymethods]
impl Caller {
    #[new]
    fn new() -> Self {
        Self {
            py_objects: Vec::new()
        }
    }

    fn add_py_object(&mut self, py_obj: PyObject) {
        self.py_objects.push(py_obj);
    }

    fn call_methods_unopt(&self, n_iter: i32, method_name: String) {
        Python::with_gil(|py| {
            for _i in 0..n_iter {
                for obj in &self.py_objects {
                    obj.call_method(py, method_name.as_str(), (), None).expect("pyobj call the method");
                }
            }

        });
    }
}

As can be seen from the method ** call_methods_unopt**, the logic is very stratightforward
Given K py objects and N iterations, the time complexity of the method call_methods_unopt is O(K * N)

However, we can further optimize the method call_methods_unopt; actually we can even double the calling throughput for the method call_methods_unopt by simply changing as follows:

    fn call_methods_opt(&self, n_iter: i32, method_name: String) {
        Python::with_gil(|py| {
            let mut callees: Vec<PyObject> = Vec::with_capacity(self.py_objects.len());
            for obj in &self.py_objects {
                callees.push(obj.getattr(py, method_name.as_str()).expect("method should exist"));
            }
            for _i in 0..n_iter {
                for c in &callees {
                    c.call(py, (), None).expect("callable gets called directly");
                }
            }

        });
    }

The difference between the above two methods seems not significant, but the performance gap is huge!! (almost 70% throughput difference without considering the duration for each py object's method)

Why so??

well I noticed this when I look into the actual implementation of the call_method and call in pyo3

   pub fn call(
        &self,
        py: Python,
        args: impl IntoPy<Py<PyTuple>>,
        kwargs: Option<&PyDict>,
    ) -> PyResult<PyObject> {
        let args = args.into_py(py).into_ptr();
        let kwargs = kwargs.into_ptr();
        let result = unsafe {
            PyObject::from_owned_ptr_or_err(py, ffi::PyObject_Call(self.as_ptr(), args, kwargs))
        };
        unsafe {
            ffi::Py_XDECREF(args);
            ffi::Py_XDECREF(kwargs);
        }
        result
    }

    pub fn call_method(
        &self,
        py: Python,
        name: &str,
        args: impl IntoPy<Py<PyTuple>>,
        kwargs: Option<&PyDict>,
    ) -> PyResult<PyObject> {
        name.with_borrowed_ptr(py, |name| unsafe {
            let args = args.into_py(py).into_ptr();
            let kwargs = kwargs.into_ptr();
            let ptr = ffi::PyObject_GetAttr(self.as_ptr(), name);
            if ptr.is_null() {
                return Err(PyErr::fetch(py));
            }
            let result = PyObject::from_owned_ptr_or_err(py, ffi::PyObject_Call(ptr, args, kwargs));
            ffi::Py_DECREF(ptr);
            ffi::Py_XDECREF(args);
            ffi::Py_XDECREF(kwargs);
            result
        })
    }

you may notice as well when you have a look at the above two implementations.

Did you find out??

The answer is due to let ptr = ffi::PyObject_GetAttr(self.as_ptr(), name);

In CPython, everything in python including functions, objects, lists, etc is pointer, when you call a py object's method called "foo" this is what happens:

1. the interpreter checks if the given py object has the method called "foo"
2. since the method itself is pointer, in the above process, there is extra overhead for finding the corresponding method "foo" (this actually corresponds to __getattr__ method in the Python
3. therefore call_method actually involves in calling two methods in sequence: the first one is getattr method (this corresponds to ffi::PyObject_GetAttr(self.as_ptr(), name); in the above) and the second is the original target method.

Therefore, we need to extract the target method pointers before actually executing them to avoid calling getattr method

Let's look at the optimised one again

    fn call_methods_opt(&self, n_iter: i32, method_name: String) {
        Python::with_gil(|py| {
            let mut callees: Vec<PyObject> = Vec::with_capacity(self.py_objects.len());
            for obj in &self.py_objects {
                callees.push(obj.getattr(py, method_name.as_str()).expect("method should exist"));
            }
            for _i in 0..n_iter {
                for c in &callees {
                    c.call(py, (), None).expect("callable gets called directly");
                }
            }

        });
    }

As can be seen, it first of all extracts the target method pointers ahead and then start to execute.

But of course, in real cases, the actual bottleneck exists often in the python side, so this effect may not be significant. Though, this performance gain is huge by changing code a bit

[adamreichold]

I would argue that this (caching attribute look-ups) is a common technique when optimizing pure Python code as well, i.e. not specific to PyO3 or even native extensions. Also note that this changes observable behaviour in a dynamic language like Python, e.g. the objects could theoretically change their method attribute to point to different code with each invocation which would not work with the optimized variant.

[noahkim0723]

well I actually tested in Python 3.8, the performance gap was huge

[adamreichold]

well I actually tested in Python 3.8, the performance gap was huge

Certainly, I am just arguing that the same performance gap exists when pure Python calls methods in a loop, i.e. this is not specific to PyO3. And the proposed optimization has the caveat of changing observable behaviour. (The Python interpreter itself is actually in a good position to make this optimization when there is no such change because it could track if the method attribute was modified since the last access.)