Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CORE-207: handle quoted TSV values that look like non-strings #1508

Draft
wants to merge 5 commits into
base: develop
Choose a base branch
from

Conversation

davidangb
Copy link
Contributor

@davidangb davidangb commented Dec 6, 2024

draft, do not review

  • when parsing a TSV upload, if a value is quoted, remove the quotes and always treat that value as a string.
  • when downloading a TSV, if an existing string value looks like a number or a boolean, quote that value. This is necessary so that if the TSV is re-uploaded, Orch will still recognize the value as a string.

This feature adds significant overhead to TSV uploads and downloads.

In rather informal testing on my local machine, I ran this large-TSV download unit test in a tight loop and measured the timing.

This PR: average 44.59 seconds, N=60
develop branch: average 2.51 seconds, N=40


Have you read CONTRIBUTING.md lately? If not, do that first.

I, the developer opening this PR, do solemnly pinky swear that:

  • I've followed the instructions if I've made any changes to the API, especially if they're breaking changes
  • I've updated the FISMA documentation if I've made any security-related changes, including auth, encryption, or auditing

In all cases:

  • Get two thumbsworth of review and PO signoff if necessary
  • Verify all tests go green
  • Squash and merge. Make sure your branch deletes; GitHub should do this for you.
  • Test this change deployed correctly and works on dev environment after deployment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant