Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make columns types an empty list for empty tabular data #13918

Merged
merged 5 commits into from
Mar 1, 2024

Conversation

bernt-matthias
Copy link
Contributor

@bernt-matthias bernt-matthias commented May 13, 2022

  • show number of columns [Feature Request] Tabular dataset improvement: show lines and columns #13258
    • could also be rows x cols (I.e. drop the words lines and columns) for compacter display
    • empty files currently get 0 lines 1 colums?
      • @dannon mentioned that this is because there are tables with enormous numbers of columns (but I don't see this breaking since there is only a max no of lines limit, or?)
        • there is a 1MB limit in the line number estimation function, but the number of columns is taken from the properties filled by set_meta which reads MAX_LINES .. question is if we can assume that set_meta runs before set_peek?
      • guess we should fix this and cover this with a test for Tabular.set_meta .. should this be an inline unit test or another way
      • not entirely sure if fix is 100% correct, since files with a single empty column are now also 0 column files. this is because the tabular datatype assumes empty lines as comments which are not used to determine empty columns
  • shorter display of estimated number of lines line number approximation of tabular data #6506
    • also thought if K, M, G... suffixes might be nicer

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

@github-actions github-actions bot added this to the 22.05 milestone May 13, 2022
@bernt-matthias bernt-matthias force-pushed the tabular-improvement branch 2 times, most recently from 3ad6bef to 943836c Compare May 14, 2022 08:28
@bernt-matthias bernt-matthias force-pushed the tabular-improvement branch 6 times, most recently from 0b937ed to 2604602 Compare May 16, 2022 12:52
@bernt-matthias bernt-matthias changed the title Some improvements to tabular Some improvements to tabular/data May 18, 2022
@mvdbeek mvdbeek requested a review from jdavcs May 25, 2022 14:13
@mvdbeek mvdbeek removed this from the 22.05 milestone May 25, 2022
@jdavcs jdavcs added this to the 22.09 milestone May 26, 2022
@jdavcs
Copy link
Member

jdavcs commented May 26, 2022

Assigning the 22.09 milestone. Please remove if this is premature.

@nsoranzo nsoranzo marked this pull request as draft September 15, 2022 18:13
@dannon dannon modified the milestones: 23.0, 23.1 Jan 10, 2023
@mvdbeek mvdbeek removed this from the 23.1 milestone Jun 21, 2023
to actually check set metadata values
as 0 column files

- refactor type_overrules_type for simplicity
- add tests
@bernt-matthias bernt-matthias changed the title Some improvements to tabular/data Empty tabular data has no column types Feb 19, 2024
@bernt-matthias bernt-matthias changed the title Empty tabular data has no column types Make columns types an empty list for empty tabular data Feb 19, 2024
@bernt-matthias bernt-matthias added this to the 24.0 milestone Feb 20, 2024
@bernt-matthias bernt-matthias marked this pull request as ready for review February 20, 2024 11:08
Copy link
Member

@jdavcs jdavcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @bernt-matthias! Sorry for delaying with this review!

@jdavcs jdavcs merged commit 54294f0 into galaxyproject:dev Mar 1, 2024
43 of 44 checks passed
Copy link

github-actions bot commented Mar 1, 2024

This PR was merged without a "kind/" label, please correct.

@bernt-matthias bernt-matthias deleted the tabular-improvement branch March 1, 2024 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants