Quickly showing a subset of columns

[cool-RR]

I have a need and I wonder whether anyone else has had that need and found the solution.

Here's something that happened today, and happens quite frequently: I opened a sheet with 20+ columns. I needed to look at 3 of its columns side by side so I could compare them. They were not positioned close to each other, so I had to search for them and move them to be closer to each other using Shift-H and Shift-L. This takes about 30 seconds.

I have many CSVs with the same structure that I'm examining as I'm iterating on experiments. I keep having to do this 30 second column shuffle on every open.

I would like to have some automatic way to say "Show only columns foo, bar and baz". Does anyone have a way to do that?

I could write a script for this specific set of columns, but this happens in different kinds of sheets where I want to have different subsets of columns. I considered making a script that lets me define column subsets, and then apply them to the current sheet or edit them. But this would be quite an involved UI for a script. If anyone has a different solution, I'll be happy to hear it.

[frosencrantz]

@cool-RR Did you already figure this one out? If not, here is how I would do it.

Try:

• C go to the column sheet
• gs global select to select all rows, which are columns of the original sheet
• g- to hide all selected rows
• gu to unselect all rows
• | pattern to select the columns of interest. e.g ^(col1name|col8name|col9name|col18name)$
  • If you use the same columns regularly this will be in your history
• space to get to the commands, then resize source-rows-max. I think this is the closest to unhide select rows. Maybe it would be useful if there was a binding for it on the ColumnSheet like g_, but that overwrites the generic sheet one.
  • The other option is to move the cursor to the width column and make a ge global edit or set the width individually.
• then q or ^^ to go back to original sheet.

[cool-RR]

Thanks @frosencrantz . If your instruction list was half the size it currently is, it would still be too long for me.

I think my best solution right now is to use this script of mine below. It basically compresses your seven steps into one. I would still like something faster than that, since typing a regex (or looking through history) is a bit cumbersome, but I've given up on working on that for now.

def _filter_columns_by_regex(sheet: Sheet, regex_string: str, *,
                             always_include_keys: bool = False) -> tuple[list[Column],
                                                                         list[Column]]:
    regex_flags = sum(getattr(re, f.upper()) for f in sheet.options.regex_flags)
    pattern = re.compile(regex_string, regex_flags)
    matching_columns = []
    non_matching_columns = []
    for column in sheet.columns:
        if (always_include_keys and (column in sheet.keyCols)) or pattern.search(column.name):
            matching_columns.append(column)
            for matching_column in matching_columns[:]:
                if column.name.startswith(f'{matching_column.name}.'):
                    matching_columns.remove(matching_column)
                    non_matching_columns.append(matching_column)
        else:
            non_matching_columns.append(column)
    return matching_columns, non_matching_columns


@Sheet.api
def show_only_matching_columns(sheet: Sheet, regex_string: str) -> None:
    matching_columns, non_matching_columns = _filter_columns_by_regex(sheet, regex_string,
                                                                      always_include_keys=True)
    for column_to_hide in non_matching_columns:
        column_to_hide.hide()
    vd.unhide_cols(matching_columns, sheet.visibleRows)


Sheet.addCommand(
    'Ins c',
    'show-only-matching-columns',
    'show_only_matching_columns(input("filter column by regex: ", type="column_regex", '
                                     'defaultLast=True))',
    'Show only the columns matching the given regex, hiding other columns.'
)

[frosencrantz]

Thank you for sharing @cool-RR! I think csvlens has such a command.

I think you could make this a macro except macros do not have a way to provide input. If there was a way for a macro to receive user input (e.g. column regex) when the macro runs it might have been possible to make this a single macro call with input. Would you have used such a macro over writing your own script?

@saulpw would it make sense to have some way to pass input to a macro? Maybe as part of running the macro, it could take input. It looks like it would also be useful to be able to provide a "type" for history purposes, and the ability to specify 'defaultLast=True'.

[cool-RR]

I probably wouldn't use a macro because I never used one. I'm probably not going to think about this issue anytime soon. Thanks for following up anyway.

[gennaro-tedesco]

Not a direct answer to your question, but wouldn't this be better achieved by using awk, cut or similar terminal utilities before loading the data into visidata? Especially if you have a collection of files to do this onto, might as well do it once and for all.

There is the awesome xsv which I often use as companion in crime for visidata. You could do something along the lines of

fd -t f -e csv | xargs -n1 -I {} bash -c 'xsv select 2,1,13,... {} > {}.csv'

(you might need to double check the xargs grammar).

[cool-RR]

I could do that but I do believe that slicing and dicing data is part of VisiData's core competency, and not something I want to outsource to an external tool.