ImportError('Error importing numpy: you should not try to import numpy from\n its source directory; please exit the numpy source tree, and relaunch\n your python interpreter from there.')

[ZKBL]

my code

use pyo3::types::PyList;
use which::which;
fn main() {
    let path = which("python").unwrap();
    println!("{:?}", path);
    pyo3::Python::with_gil(|py| {
        let sys = py.import("sys").unwrap();
        let path: &PyList = sys.getattr("path").unwrap().extract().unwrap();
        path.append("".to_string()).unwrap();
        for i in path.iter() {
            println!("{:?}", i);
        }
        py.run("import cv2;import numpy as np;", None, None)
            .unwrap();
    })
}

run the command

PYO3_PYTHON=~/ImageRust/.venv/bin/python cargo r --bin test-path
outpu is

warning: file `/home/green/ImageRust/tests/test-path.rs` found to be present in multiple build targets:
  * `bin` target `test-path`
  * `integration-test` target `test-path`
   Compiling pyo3-build-config v0.20.0
   Compiling pyo3-ffi v0.20.0
   Compiling pyo3 v0.20.0
   Compiling image v0.1.0 (/home/green/ImageRust)
   Compiling numpy v0.20.0
   Compiling pyo3-asyncio v0.20.0
    Finished dev [unoptimized + debuginfo] target(s) in 4.90s
     Running `target/debug/test-path`
"/home/green/ImageRust/.venv/bin/python"
'/home/green/.rye/py/cpython@3.11.6/install/lib/python311.zip'
'/home/green/.rye/py/cpython@3.11.6/install/lib/python3.11'
'/home/green/.rye/py/cpython@3.11.6/install/lib/python3.11/lib-dynload'
'/home/green/ImageRust/.venv/lib/python3.11/site-packages'
'/home/green/ImageRust/python'
''
OpenCV bindings requires "numpy" package.
Install it via command:
    pip install numpy
thread 'main' panicked at tests/test-path.rs:14:14:
called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ImportError'>, value: ImportError('Error importing numpy: you should not try to import numpy from\n        its source directory; please exit the numpy source tree, and relaunch\n        your python interpreter from there.'), traceback: Some(<traceback object at 0x7feb79f56c80>) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

The above is the case when the PYTHONHOME and PYTHONPATH environment variables are not set.

---

When I set PYTHHOME and PYTHPATH, the output looks like this

warning: file `/home/green/ImageRust/tests/test-path.rs` found to be present in multiple build targets:
  * `bin` target `test-path`
  * `integration-test` target `test-path`
    Finished dev [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/test-path`
"/home/green/ImageRust/.venv/bin/python"
Python path configuration:
  PYTHONHOME = '/home/green/ImageRust/.venv/bin/python'
  PYTHONPATH = '/home/green/ImageRust/.venv/lib/python3.11/site-packages'
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  safe_path = 0
  import site = 1
  is in build tree = 0
  stdlib dir = '/home/green/ImageRust/.venv/bin/python/lib/python3.11'
  sys._base_executable = '/home/green/ImageRust/.venv/bin/python3'
  sys.base_prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.base_exec_prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.platlibdir = 'lib'
  sys.executable = '/home/green/ImageRust/.venv/bin/python3'
  sys.prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.exec_prefix = '/home/green/ImageRust/.venv/bin/python'
  sys.path = [
    '/home/green/ImageRust/.venv/lib/python3.11/site-packages',
    '/home/green/ImageRust/.venv/bin/python/lib/python311.zip',
    '/home/green/ImageRust/.venv/bin/python/lib/python3.11',
    '/home/green/ImageRust/.venv/bin/python/lib/python3.11/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f6ccb7e3800 (most recent call first):
  <no Python frame>

My Environmental Information

• OS version: Ubuntu 22.04.3
• rust version: 1.74.0
• pyo3 version: 0.20.0
• python version: 3.11.6

[cguentherTUChemnitz]

I run over and over into the same problem as soon as i try to use nix flakes or devenv.nix to manage the development environment:

devenv.nix:

{ pkgs, lib, config, inputs, ... }: let
in {
  languages.python = {
    enable = true;
    venv.enable = true;
    venv.requirements = ''
      numpy
    '';
  };

  languages.rust = {
    enable = true;
    channel = "stable";
  };
}

rust test:

    #[test]
    fn numpy_import() {
        Python::with_gil(|py| {
            py.import_bound("numpy").unwrap();
        })
    }

result

called `Result::unwrap()` on an `Err` value: PyErr { type: <class 'ImportError'>, value: ImportError('Error importing numpy: you should not try to import numpy from\n        its source directory; please exit the numpy source tree, and relaunch\n        your python interpreter from there.'), traceback: Some(<traceback object at 0x7ffff6f1af40>) }

This is totally unclear what happens here. I can confirm that i run in the test command inside the venv. The venv is able to import numpy correctly when i just run it directly from the interactive python session: python -c "import numpy", but an cargo test fails in the same environment.