Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gitpython: Failing on every run #10600

Open
jonathanmetzman opened this issue Jun 26, 2023 · 3 comments
Open

gitpython: Failing on every run #10600

jonathanmetzman opened this issue Jun 26, 2023 · 3 comments
Assignees
@DavidKorczynski

Comments

@jonathanmetzman
Copy link
Contributor

I think the solution is to bundle git. I don't think we have git on our clusterfuzz instances.
It's failing with this error:

Traceback (most recent call last):
  File "git/__init__.py", line 89, in <module>
  File "git/__init__.py", line 76, in refresh
  File "git/cmd.py", line 392, in refresh
ImportError: Bad git executable.
The git executable must be specified in one of the following ways:
    - be included in your $PATH
    - be set via $GIT_PYTHON_GIT_EXECUTABLE
    - explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
    - quiet|q|silence|s|none|n|0: for no warning or exception
    - warn|w|warning|1: for a printed warning
    - error|e|raise|r|2: for a raised exception

Example:
    export GIT_PYTHON_REFRESH=quiet


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "fuzz_config.py", line 20, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 495, in exec_module
  File "git/__init__.py", line 91, in <module>
ImportError: Failed to initialize: Bad git executable.
The git executable must be specified in one of the following ways:
    - be included in your $PATH
    - be set via $GIT_PYTHON_GIT_EXECUTABLE
    - explicitly set via git.refresh()

All git commands will error until this is rectified.

This initial warning can be silenced or aggravated in the future by setting the
$GIT_PYTHON_REFRESH environment variable. Use one of the following values:
    - quiet|q|silence|s|none|n|0: for no warning or exception
    - warn|w|warning|1: for a printed warning
    - error|e|raise|r|2: for a raised exception

Example:
    export GIT_PYTHON_REFRESH=quiet

[1090381] Failed to execute script 'fuzz_config' due to unhandled exception!
"
@jonathanmetzman
Copy link
Contributor Author

@oliverchang another bug caused by oss-fuzz images not being the same as CF images.

@DaveLak DaveLak mentioned this issue Apr 22, 2024
Closed
1 task
@DaveLak
Copy link
Contributor

DaveLak commented Apr 22, 2024

FYI: Since GitPython's fuzz harnesses have been migrated upstream in #11803, I've created gitpython-developers/GitPython#1905 to track this issue in that repository.

DaveLak added a commit to DaveLak/GitPython that referenced this issue Apr 22, 2024
@DaveLak
Fix Fuzzer Crash in ClusterFuzz Due to Missing Git Executable
f4b95cf 
A Git executable is not globally available in the ClusterFuzz container
environment where OSS-Fuzz executes fuzz tests, causing an error in the fuzz
harnesses when GitPython attempts to initialize, crashing the tests before they
can run.

To avoid this issue, we bundle the `git` binary that is available in the OSS-Fuzz
build container with the fuzz harness via Pyinstaller's `--add-binary` flag in
`build.sh` and use GitPython's `git.refresh(<full-path-to-git-executable>)`
method inside a Pyinstaller runtime check to initialize GitPython with the
bundled Git executable when running from the bundled application.

In all other execution environments, we assume a `git` executable is available
globally.

Fixes:
- gitpython-developers#1905
- google/oss-fuzz#10600
@DaveLak DaveLak mentioned this issue Apr 22, 2024
Merged
DaveLak added a commit to DaveLak/GitPython that referenced this issue Apr 26, 2024
@DaveLak
Attempt 2 - Fix Missing Git Executable Causing ClusterFuzz Crash
dac3535 
This is a second attempt at gitpython-developers#1906 and should resolve:
- gitpython-developers#1905
- google/oss-fuzz#10600

PR gitpython-developers#1906 had the right idea but wrong implementation, and the differences between
the ClusterFuzz image that it was supposed to fix and the OSS-Fuzz image where
the fix was tested led to the issue not being fully resolved.

The root cause of the issue is the same: A Git executable is not globally
available in the ClusterFuzz container environment where OSS-Fuzz executes
fuzz tests.

 gitpython-developers#1906 attempted to fix the issue by bundling the Git binary and using
GitPython's `git.refresh(<full-path-to-git-executable>)` method to set it
inside the `TestOneInput` function of the test harness.

However, GitPython attempts to set the binary at import time via its `__init__`
hook, and crashes the test if no executable is found during the import.

This issue is fixed here by setting the environment variable that GitPython
looks in before importing it, so it's available for the import. This was tested
by setting the `$PATH` to an empty string inside the test files, which
reproduced the crash, then adding the changes introduced here with `$PATH` still
empty, which avoided the crash indicating that the bundled Git executable is
working as expected.
@DaveLak DaveLak mentioned this issue Apr 26, 2024
Merged
@DaveLak
Copy link
Contributor

DaveLak commented Apr 28, 2024

This issue was resolved via gitpython-developers/GitPython#1909. Tests appear to be running as expected now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DavidKorczynski DavidKorczynski

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@DavidKorczynski @DaveLak @jonathanmetzman