Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate possibilities to migrate away from wget #60

Open
nkphysics opened this issue Jun 2, 2024 · 0 comments
Open

Investigate possibilities to migrate away from wget #60

nkphysics opened this issue Jun 2, 2024 · 0 comments
Assignees
Labels
dependencies Pull requests that update a dependency file enhancement New feature or request

Comments

@nkphysics
Copy link
Owner

I know and generally like to follow the motto "If it ain't broke don't fix it", but I've been continually reminded of the benefits of minimizing dependencies. The heasoft dependency clearly cannot be dropped for obvious reasons, but autoNICER still relies on wget (not the wget python lib) to retrieve datasets from the heasarc.

See the first part of the pull reduce function:

def pull_reduce(self):
        """
        Downloads the NICER data
        Puts the retrieved data through a standardized data reduction scheme
        """
        downCommand = (
            "wget -q -nH --no-check-certificate --cut-dirs=5 -r -l0 -c -N -np -R "
            + "'index*'"
            + " -erobots=off --retr-symlinks https://heasarc.gsfc.nasa.gov/FTP/nicer/data/obs/"
        )

This potentially could be improved upon with just using a conventional get request with the requests lib, it also could potentially lead to converting all of this to be async, which could certainly improve the code.

@nkphysics nkphysics added enhancement New feature or request dependencies Pull requests that update a dependency file labels Jun 2, 2024
@nkphysics nkphysics self-assigned this Jun 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant