Teach --list-archives about --null

[cperciva]

We should make --list-archives respect --null, to avoid problems if people do silly things like putting newline characters into their archive names.

This is a backwards-compatible change in behaviour, because currently --null errors for --list-archives.

It's not clear how --null should interact with -v -v, so leave that as an error for now. But --list-archives -v -null should print \0\0\0\0...

Once this is done, tarsnap-gui should make use of it. Right now it's just splitting on newlines, which could cause problems if processing a list of archives created via the command line.

[gperciva]

What do you mean by

It's not clear how --null should interact with -v -v, so leave that as an error for now. But --list-archives -v -null should print \0\0\0\0...

tarsnap-gui currently does --list-archives -vv, so leaving that as an error won't help.

My first thought that -v -v --null should simply print a \0 instead of each \t or \n that we're currently printing. Admittedly this means parsing the output would require knowing whether you called --list-archives with 0, 1, or 2 vs, but I think that's an acceptable requirement.

So that we have a convenient reference, here's the current output:

td@mac: ~/src/tarsnap/build (master)
$ ./tarsnap --list-archives -v
a3	2017-11-18 20:45:05
tiny	2018-02-21 21:48:38
b	2018-05-20 22:14:37
td@mac: ~/src/tarsnap/build (master)
$ ./tarsnap --list-archives -v -v
a3	2017-11-18 20:45:05	./tarsnap -c -f a3 --upload-progress-bytes 100000 tar/
tiny	2018-02-21 21:48:38	./tarsnap -c -f tiny caps/
b	2018-05-20 22:14:37	./tarsnap -c -f b backup.txt
td@mac: ~/src/tarsnap/build (master)
$ ./tarsnap --list-archives -vv
a3	2017-11-18 20:45:05	./tarsnap -c -f a3 --upload-progress-bytes 100000 tar/
tiny	2018-02-21 21:48:38	./tarsnap -c -f tiny caps/
b	2018-05-20 22:14:37	./tarsnap -c -f b backup.txt

I don't see how -vv is a bigger problem than -v.

[cperciva]

The problem with -vv is that it's printing a command line. The whole point of --null is that it makes things unambiguous; but tarsnap --list-archives -vv will print the same output for an archive created by tarsnap -c -f foo a b c as it does for an archive created by tarsnap -c -f foo "a b c".

[gperciva]

but tarsnap --list-archives -vv will print the same output for an archive created by tarsnap -c -f foo a b c as it does for an archive created by tarsnap -c -f foo "a b c".

but as far as the tarsnap binary is concerned, there's no such thing as "a b c", because that's interpreted by the shell:

$ ../tarsnap -c --dry-run --dump-config "a b"
Command-line:
  ../tarsnap -c --dry-run --dump-config a b
Reading from config file: /home/td/.tarsnaprc
  no-print-stats
  cachedir ~/.test-tarsnap/backup.cache
  keyfile ~/.test-tarsnap/backup.keys
Reading from config file: /usr/local/etc/tarsnap.conf
  nodump

The "Command-line" output is printing the raw contents of argv[]. We can force double quotes into the command-line by doing:

$ ../tarsnap -c --dry-run "\"a b\""

but that will be looking for a filename which contains double quotes.

EDIT: deleted some complete garbage that I wrote.

[gperciva]

Nice discussion about special characters in filenames by David A. Wheeler: https://dwheeler.com/essays/fixing-unix-linux-filenames.html

(probably nothing new there for @cperciva, but I'm adding here as a reminder for myself)

[gperciva]

I think there's a few different issues mingling here, and it's not clear to me which one(s) we want to solve:

• can we get the GUI to support newlines in archive names?
• are there any limitations of Tarsnap archive names, other than "not NUL" ? like, is it vitally important that we support 0x07 BEL and 0x08 BS inside archive names?
• can we print the archive's argv[] to the command-line in an unambiguous manner?
• for that matter, what's the intended use case of saving the command-line? I'm also wondering why we might want to display it as --list-archives -vv instead of --list-archives -f MYARCHIVE -vv. (and yes, I know that we don't allow -f in --list-archives right now)

[gperciva]

BTW, if anybody relies on BEL in Tarsnap archive names to keep children warm in winter, they win 100 Internet points.

[cperciva]

as far as the tarsnap binary is concerned, there's no such thing as "a b c"

If you run

tarsnap -c -f foo a b c

then tarsnap will create an archive containing the files a, b, and c. If you run

tarsnap -c -f foo "a b c"

then tarsnap will create an archive containing the single file a b c.

The point of --null is to remove ambiguity, so there's no point applying it to --list-archives -vv unless we also remove this ambiguity.

[cperciva]

• what's the intended use case of saving the command-line

It's useful for users -- at least if they're creating archives by hand. I've personally used it when I couldn't remember which directories I had archived.

[gperciva]

Yes, I get that. But there's no way for the binary to distinguish between

$ tarsnap -c -f foo "a b c"

and

$ tarsnap -c -f foo a\ b\ c

so AFAIK the recorded command-line cannot be guaranteed to match the given command-line.

[cperciva]

Right. I'm just saying that there's no point making --null apply to the command lines unless we distinguish those from the "three separate files" case.

[gperciva]

Right. I'm just saying that there's no point making --null do anything if the GUI uses --list-archives -vv and it doesn't apply to that. :)

(I'm now wondering if the GUI should do -v instead)

[cperciva]

Hmm, I didn't realize the GUI was using -vv.

[gperciva]

Yeah, and I'm not really certain why. I mean, sure, that gives it the command-lines... but it has a ton of options. I mean, look at this:

$ ./tarsnap --list-archives -vv
brackets	2018-09-28 09:32:29	./tarsnap -c -f brackets aa[bb]cc
filesize-test	2019-01-10 11:57:00	tarsnap -cf filesize-test /home/td/backup-tests
zeros-1000	2018-08-18 09:37:31	./tarsnap -c -f zeros-1000 zeros-1000
Job_h_2019-01-08_19-40-59	2019-01-08 19:40:59	/usr/bin/tarsnap --no-default-config --keyfile /home/td/.local/share/Tarsnap Backup Inc./Tarsnap/tarsnap.key --cachedir /home/td/.cache/Tarsnap Backup Inc./Tarsnap -P --creationtime 1547005259 --quiet --print-stats --no-humanize-numbers -c -f Job_h_2019-01-08_19-40-59 /home/td/backup-tests/backup
links	2019-01-09 19:52:51	tarsnap -cf links links/

One of those archives is much less informative than the others!

[gperciva]

(err, I'm arguing that by including so many details, it's harder for humans to see the important bits. So it's less informative because there's too much information.)