Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stash large response payload to S3 for /descendants/<id> #756

Open
yuanzhou opened this issue Oct 21, 2024 · 0 comments
Open

Stash large response payload to S3 for /descendants/<id> #756

yuanzhou opened this issue Oct 21, 2024 · 0 comments

Comments

@yuanzhou
Copy link
Member

The https://entity.api.hubmapconsortium.org/descendants/HBM925.HSGR.556 request to get back all descendants of Donor HBM925.HSGR.556 retuned 500 with

{
    "message": "Internal server error"
}

Further logging indicates the response payload is 11.7 MB:

Mon Oct 21 20:56:21 2024 - uwsgi_response_write_body_do(): Connection reset by peer [core/writer.c line 341] during GET /descendants/HBM925.HSGR.556 (172.31.23.74)
OSError: write error
[pid: 259|app: 0|req: 2284600/3109683] 172.31.23.74 () {52 vars in 993 bytes} [Mon Oct 21 20:56:16 2024] GET /descendants/HBM925.HSGR.556 => generated 11743268 bytes in 5791 msecs (HTTP/1.1 200) 2 headers in 77 bytes (4 switches on core 10)

Follow the existing implementation, use hubmap_commons.S3_worker to stash the results (when the response payload is >=10MB) in AWS S3 bucket and return a URL and 303 status code.

Note: /descendants/HBM925.HSGR.556?property=uuid returned 200 with 24798 uuids, and the response payload is 848 KB.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Ready
Development

No branches or pull requests

1 participant