Calling statsmodels methods to use in rust

[dannytoomey]

Hello! I'd like to call methods from the statsmodels python package and use them in rust via PyO3. I'm having trouble with an error I can't quite decipher. Here's a minimal reproducible example:

use polars_core::prelude::*;
use pyo3::prelude::*;
use pyo3_polars::PyDataFrame;

pub fn statsmodels_ols(df: DataFrame) -> PyResult<()> {
    pyo3::prepare_freethreaded_python();
    let smf: PyResult<Py<PyModule>> = Python::with_gil(|py| {
        let smf = py.import("statsmodels.formula.api")?;
        Ok(smf.into())
    });
    let from_ols = Python::with_gil(|py| -> PyResult<Py<PyAny>> {
        smf?.call_method1(
            py,
            "ols",
            (
                "Continuous_Change_A1c~Change_Tchol+Change_BP",
                PyDataFrame(df),
            ),
        )?
        .call_method0(py, "fit")?
        .call_method0(py, "summary")?
        .call_method0(py, "as_text")
    });
    println!("{:?}", from_ols);
    Ok(())
}

Where the polars dataframe passed to the function has the following fields and types:

Continuous_Change_A1c : f64
Change_Tchol : f64
Change_BP : f64

And the error I receive when printing from_ols is:

Err(PyErr { type: <class 'AttributeError'>, value: AttributeError("'NoneType' object has no attribute 'f_locals'"), traceback: Some(<traceback object at 0x1620d1e40>) })

And my Cargo.toml:

[package]
name = "stats_bin"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
pyo3 = "0.20.3"        #{ version = "0.20.3", features = ["extension-module"] }
calamine = "0.24.0"
polars = "0.38.1"
polars-core = "0.38.1"
pyo3-polars = "0.12.0"

My questions are:

1. How would I get the traceback object for future debugging?
2. Why am I receiving this error? I'm not quite sure what I missed here.

Thanks so much for your help!

[davidhewitt]

Sorry for the slow reply 🙈

1. You can use the .traceback() method on the PyErr error type. That has a .format() method on it to turn it into the kind of Python traceback string you're used to seeing.

2. Afraid I don't have a good answer for that either. At a guess, the statsmodels package is expecting to be called from Python rather than Rust, and it's triggered an error. f_locals is referring to Python frame objects. Maybe printing the traceback from (1) can give hints.

[dannytoomey]

Thanks so much! For anyone else who comes across this thread later with a similar question, here's what gave me the expected traceback output:

let _ = match from_ols {
    Ok(_) => {}
    Err(error) => {
        println!(
            "{}",
            Python::with_gil(|py| error.traceback(py).unwrap().format().unwrap())
        );
    }
};

As for my second question, I think your intuition is in the right direction! Now that I have the full traceback, I'm trying to get to the bottom of it, but a proper error stack is exactly what I needed! :)