Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle error for gcs list blob #274

Merged
merged 2 commits into from
Aug 16, 2024
Merged

Handle error for gcs list blob #274

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dcd2011
Handle error for gcs list blob
mnazzaro May 17, 2024
fe3222a
raise exception
mnazzaro May 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion arxiv/files/object_store.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,10 @@ def list(self, prefix: str) -> Iterator[FileObj]:
'ps_cache/arxiv/pdf/1212/1212.12345' or
'ftp/cs/papers/0012/0012007'.
"""
return self.bucket.client.list_blobs(self.bucket, prefix=prefix) # type: ignore
try:
return self.bucket.client.list_blobs(self.bucket, prefix=prefix) # type: ignore
except:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will hide too many errors. I'd much rather have the user see a 500 error than a response built with an incorrectly empty list from this.

Replace this except with something like

except Exception as inner_ex:
    raise RuntimeException(f"Could not get obj_store.list() for {bucket} {prefix}") from inner_ex

return iter([])

def status(self) -> Tuple[Literal["GOOD", "BAD"], str]:
"""Gets if bucket can be read."""
Expand Down
Loading