Skip to content

Commit

Permalink
Merge pull request #2238 from saulpw/v3.0.1
Browse files Browse the repository at this point in the history
v3.0.1 Release
  • Loading branch information
anjakefala authored Jan 8, 2024
2 parents 6346387 + 12ee887 commit e492f83
Show file tree
Hide file tree
Showing 38 changed files with 343 additions and 151 deletions.
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
# VisiData version history

# v3.0.1 (2024-01-07)

### Fixes and minor improvements

- [color] add `color_longname` to use instead of `color_keystrokes` #2219
- [columns] add `displayer` attribute to saved column state
- [cmdpalette] Enter executes first row, if partially typed out #2219
- [cmdpalette] add sidebar for longname and aggregator palette #2219
- [guide] add `show-command-info` to display command info for a keystroke #2228
- [keys] add `*BtnUp` pretty keys for `BUTTON#_RELEASED` #2219

- [dup-selected] dup-selected should unselect all rows in copied sheet #2225
- [expr] fix KeyError crash with invalid inputs in `expr` for Python 3.12 #2179
- [help] fix columns sheet sidebar
- [loaders html] ignore parsing exceptions on invalid urls #2227
- [sheets] limit end separators to rightmost visible column of sheet (PR by @midichef #2237)
- [sidebar] fix fmtstr on DirSheet sidebar for case where source is BaseSheet #2239
- [sort] maintain ordering on sheet copies #2190
- [test] update unit tests to use packaged sample.tsv and benchmark.csv #2218
- [threads] do not try to cancel already finished thread #2235

- [tests] add `assert-expr` and `assert-expr-row` to evaluate Python expressions, and assert result is truthy
- [tests] parametrize feature tests (PR by @ajkerrigan #2230)

# v3.0 (2023-12-30)

- [reorg] move independent modules into visidata/{features|experimental|themes}
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ include visidata/man/visidata.1
include visidata/ddw/input.ddw
include visidata/ddw/regex.ddw
include visidata/tests/sample.tsv
include visidata/tests/benchmark.csv
include visidata/desktop/visidata.desktop
include visidata/experimental/noahs_tapestry/*.json
include visidata/experimental/noahs_tapestry/*.md
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# VisiData v3.0
# VisiData v3.0.1

[![Tests](https://github.com/saulpw/visidata/workflows/visidata-ci-build/badge.svg)](https://github.com/saulpw/visidata/actions/workflows/main.yml)
[![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
Expand Down
96 changes: 43 additions & 53 deletions dev/ROADMAP
Original file line number Diff line number Diff line change
@@ -1,60 +1,50 @@
# Roadmap for 2.x

1. holdovers from 2.0
- [canvas] API docs
# Roadmap for v3.x

1. help: guides and internal docs
- write docs/monorepo (manifesto/explanation about monorepo strategy)
- cleanup files in dev
- review files in docs
- review and commit dev/checklist/feature.md
- link to guidelines in CONTRIBUTING.md
- review/edit docs/api/module.md
- review/edit/commit dev/philosophy.md
- review/edit /docs/customize
- manpage overhaul
- from md instead of troff
- mention "histogram" near Shift+F
- vd.org/formats
- rename 'VisiData loader' column
- from 'aliases' column

2. hierarchical display
- hierarchical column #692
- collapsible rows
- org-mode
- json arrays

3. options
- [options] option enums
- [options] user-defined option aliases
- [splitpane] File preview in directory view
+ [defermods] 'modified since last save' indicator on status bar
- [scroll cell] zh/zl Left hand side of a cell with content hidden doesn't show ellipsis #751

2. Persistence
+ input history #736 #468
- key indexes for better joining and lookups

3. Interface discoverability for commands #247 #742
- clickable menu canvas
- more clickable affordances all around
- clickable motd
- possibly add popup modals

4. more expressive expressions
+ Memory sheet; can give things names and use in expressions. #392
- access column values from execstr #655
+ shortcut name for current column #659

5. better asynchronicity
- async thread pool
- streaming architecture #366 #656

6. Loaders/Savers
- frictionless saver #237
- RSS reader #157
- toml loader #735
- HexSheet for unknown/binary files #548
- .ods loader (LibreOffice/OpenOffice spreadsheet) #473

- jsonl load and save round-trip (minimal diff) #429
- [options] organized options sheet

## other features
4. tests
- rework sample_data
- basic vdsql tests in CI

a) generate non-terminal graphs (ggplot)
b) automatic reload into time series
c) intra-cell coloring (for search results, markup)

# major plugin projects

1. fully operational SQL viewer/editor
5. internals
- async thread pool
- Path.options
- invert input/inputMultiple
- handle all single-line inputs, and all get/setLastArgs

- #282: Select starting table in postgres from command-line
- #579: [Postgres] Allow inserting / deleting rows
- #522: [postgres] parms in options
- #586: SQL query data
- #727: [postgres] Transaction error when viewing table
- #729: Integrate generic SQL loader
6. default theme
- tooling to make it easier to browse color combos

2. web scraper
7. website
- add vd.org/contributors
- add vd.org/pricing

- #480: HTTP API pagination loader
- #465: Ability to load from the contents of a cell.
+ #505: [html] Provide way to access non-table elements
8. new clients
- vd tui client: connect to vd server as separate terminal process
- vd web client
- lightweight JS widget to display/interact with command server
11 changes: 11 additions & 0 deletions dev/checklists/manual-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,14 @@
22. Test `open-row` on an html link: https://hls.gsfc.nasa.gov/data/
23. That DirSheet requires a commit-sheet before changes on filesystem
24. Test adding multiple aggregators via palette (+)
25. cmdpalette
- longname
- Enter executes top recommended
- Tab + Enter cycles through + executes
- 0-9 executes numbered
- aggregators
- Enter executes top
- Tab cycles through
- Space adds aggreqators
- Enter submits aggregators. without adding (except if no aggregators)
- 0-9 adds a number
7 changes: 4 additions & 3 deletions dev/checklists/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@

3. Verify that VisiData is up-to-date:
* help menu
* plugins.jsonl

3. Ensure `develop` automated tests run correctly with dev/test.sh

Expand Down Expand Up @@ -54,7 +53,7 @@

9. Push code to stable

10. Push `develop` to pypi
10. Push `stable` to pypi

a. set up a ~/.pypirc

Expand All @@ -77,8 +76,10 @@
Push to pypi
```
rm -rf dist/
python3 setup.py sdist bdist_wheel
chmod -R a+rX .
chmod -R a+rX dist
ls dist/
twine upload dist/*
```
Expand Down
4 changes: 4 additions & 0 deletions dev/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,10 @@ for i in $TESTS ; do
PYTHONPATH=. bin/vd --overwrite=False --play "$i" --batch --output "$goldfn" --config tests/.visidatarc --visidata-dir tests/.visidata
echo "save: $goldfn"
done
else
echo $TEST
PYTHONPATH=. bin/vd --play "$i" --batch --config tests/.visidatarc --visidata-dir tests/.visidata
echo "done"
fi
done

Expand Down
14 changes: 7 additions & 7 deletions docs/man.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ vd(1)
<span style="font-weight:bold;">U</span> undo the most recent modification (requires enabled <span style="font-weight:bold;">options.undo</span>)
<span style="font-weight:bold;">R</span> redo the most recent undo (requires enabled <span style="font-weight:bold;">options.undo</span>)

<span style="font-weight:bold;">Space</span> <span style="text-decoration:underline;">longname</span> open command palette; execute command by its <span style="text-decoration:underline;">longname</span>
<span style="font-weight:bold;">Space</span> <span style="text-decoration:underline;">longname</span> open command palette; <span style="font-weight:bold;">Enter</span> to execute top command by its <span style="text-decoration:underline;">longname</span>

<span style="font-weight:bold;">Command</span> <span style="font-weight:bold;">Palette</span>
<span style="font-weight:bold;">Tab</span> Move to command palette, and cycle through commands
Expand Down Expand Up @@ -516,7 +516,7 @@ vd(1)
<span style="font-weight:bold;">--matrix-device-id</span>=<span style="text-decoration:underline;">str</span> VisiData device ID associated with matrix login
<span style="font-weight:bold;">--reddit-client-id</span>=<span style="text-decoration:underline;">str</span> client_id for reddit api
<span style="font-weight:bold;">--reddit-client-secret</span>=<span style="text-decoration:underline;">str</span> client_secret for reddit api
<span style="font-weight:bold;">--reddit-user-agent</span>=<span style="text-decoration:underline;">str</span> 3.0 user_agent for reddit api
<span style="font-weight:bold;">--reddit-user-agent</span>=<span style="text-decoration:underline;">str</span> 3.0.1 user_agent for reddit api
<span style="font-weight:bold;">--zulip-batch-size</span>=<span style="text-decoration:underline;">int</span> -100 number of messages to fetch per call (&lt;0 to fetch before anchor)
<span style="font-weight:bold;">--zulip-anchor</span>=<span style="text-decoration:underline;">int</span> 1000000000 message id to start fetching from
<span style="font-weight:bold;">--zulip-delay-s</span>=<span style="text-decoration:underline;">float</span> 1e-05 seconds to wait between calls (0 to stop after first)
Expand Down Expand Up @@ -632,13 +632,15 @@ vd(1)
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">code</span> bold white on 237 color of code sample
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">heading</span> bold 200 color of header
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">guide</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">unwritten</span> 243 on black color of unwritten guides in GuideGuide
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">rstatus</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">fmt</span> {sheet.threadStatus} {sheet.keystrokeStatus} {sheet.longname} {sheet.nRows:9d} {sheet.rowtype} {sheet.modifiedStatus}{sheet.selectedStatus}{vd.replayStatus}
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">rstatus</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">fmt</span> {sheet.threadStatus} {sheet.keystrokeStatus} [:longname]{sheet.longname}[/] {sheet.nRows:9d} {sheet.rowtype} {sheet.modifiedStatus}{sheet.selectedStatus}{vd.replayStatus}
right-side status format string
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">status</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">fmt</span> [:onclick sheets-stack]{sheet.shortcut}› {sheet.name}[/]|
status line prefix
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">lstatus</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">max</span> 0 maximum length of left status line
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">status</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">sep</span> │ separator between statuses
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">keystrokes</span> bold white on 237 color of input keystrokes on status line
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">keystrokes</span> bold white on 237 color of input keystrokes
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">longname</span> bold 52 on 114 green
color of command longnames
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">keys</span> bold color of keystrokes in help
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">status</span> bold on 238 status line color
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">error</span> 202 1 error message color
Expand Down Expand Up @@ -692,8 +694,6 @@ vd(1)
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">cmdpal</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">max</span> 10 max number of suggestions for command palette
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">shellcmd</span> 21 on 114 green
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">colname</span> underline
<span style="font-weight:bold;">color</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">longname</span> bold 52 on 114 green

<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">scroll</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">context</span> 0 minimum number of lines to keep visible above/below cursor when scrolling
<span style="font-weight:bold;">disp</span><span style="text-decoration:underline;">_</span><span style="font-weight:bold;">sparkline</span> ▁▂▃▄▅▆▇ characters to display sparkline

Expand Down Expand Up @@ -788,5 +788,5 @@ vd(1)
<span style="font-weight:bold;">AUTHOR</span>
<span style="font-weight:bold;">VisiData</span> was made by Saul Pwanson &lt;<span style="text-decoration:underline;">vd@saul.pw</span>&gt;.

Linux/MacOS December 29, 2023 Linux/MacOS
Linux/MacOS January 7, 2024 Linux/MacOS
</pre></section>
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

# tox can't actually run python3 setup.py: https://github.com/tox-dev/tox/issues/96
# from visidata import __version__
__version__ = "3.0"
__version__ = "3.0.1"

setup(
name="visidata",
Expand Down Expand Up @@ -51,7 +51,7 @@
package_data={
"visidata.man": ["vd.1", "vd.txt"],
"visidata.ddw": ["input.ddw", "regex.ddw"],
"visidata.tests": ["sample.tsv"],
"visidata.tests": ["sample.tsv", "benchmark.csv"],
"visidata.desktop": ["visidata.desktop"],
"visidata.experimenta.noahs_tapestry": [
"clues.json",
Expand Down
3 changes: 3 additions & 0 deletions tests/golden/issue2227.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
url url_link0
1 https://google.com]
2 https://example.com
5 changes: 2 additions & 3 deletions tests/issue2190.vdj
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
#!vd -p
{"sheet": null, "col": null, "row": null, "longname": "open-file", "input": "sample_data/sample.tsv", "keystrokes": "o", "comment": null}
{"sheet": "sample", "col": "Region", "row": "", "longname": "sort-asc", "input": "", "keystrokes": "[", "comment": "sort ascending by current column; replace any existing sort criteria"}
{"sheet": "sample", "col": "", "row": "", "longname": "dup-rows", "input": "", "keystrokes": "g\"", "comment": "open a duplicate sheet with only the selected rows"}
{"sheet": "sample", "col": "Rep", "row": "", "longname": "sort-asc-add", "input": "", "keystrokes": "z[", "comment": "sort ascending by current column; add to existing sort criteria"}
{"sheet": "sample", "col": "Item", "row": "", "longname": "sort-desc-add", "input": "", "keystrokes": "z]", "comment": "sort descending by current column; add to existing sort criteria"}
{"sheet": "sample", "col": "", "row": "", "longname": "select-rows", "input": "", "keystrokes": "gs", "comment": "select all rows"}
{"sheet": "sample", "col": "", "row": "", "longname": "dup-selected", "input": "", "keystrokes": "\"", "comment": "open a duplicate sheet with only the selected rows"}
{"sheet": "sample_selectedref", "col": "", "row": "", "longname": "unselect-rows", "input": "", "keystrokes": "gu", "comment": "unselect all rows"}
{"longname": "assert-expr", "input": "sheet._ordering[0][0] is sheet.column(\"Region\")"}
3 changes: 3 additions & 0 deletions tests/issue2225-nosave.vdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
select-row
dup-selected
assert-expr sheet.nSelectedRows == 0
5 changes: 5 additions & 0 deletions tests/issue2227.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<table>
<tr><th>url</th></tr>
<tr><td><a href="https://google.com]">1</a></td></tr>
<tr><td><a href="https://example.com">2</a></td></tr>
</table>
3 changes: 3 additions & 0 deletions tests/issue2227.vdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# VisiData v3.0
open-file tests/issue2227.html
open-row
2 changes: 1 addition & 1 deletion visidata/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
'VisiData: a curses interface for exploring and arranging tabular data'

__version__ = '3.0'
__version__ = '3.0.1'
__version_info__ = 'VisiData v' + __version__
__author__ = 'Saul Pwanson <vd@saul.pw>'
__status__ = 'Production/Stable'
Expand Down
15 changes: 14 additions & 1 deletion visidata/aggregators.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,20 @@
from visidata import Progress, Sheet, Column, ColumnsSheet, VisiData
from visidata import vd, anytype, vlen, asyncthread, wrapply, AttrDict

vd.help_aggregators = '# Aggregators Help\nHELPTODO'
vd.help_aggregators = '''# Choose Aggregators
Start typing an aggregator name or description.
Multiple aggregators can be added by separating spaces.
- `Enter` to select top aggregator.
- `Tab` to highlight top aggregator.
## When Aggregator Highlighted
- `Tab`/`Shift+Tab` to cycle highlighted aggregator.
- `Enter` to select aggregators.
- `Space` to add highlighted aggregator.
- `0-9` to add numbered aggregator.
'''

vd.option('null_value', None, 'a value to be counted as null', replay=True)

Expand Down
Loading

0 comments on commit e492f83

Please sign in to comment.