forked from solsticedhiver/atool2
-
Notifications
You must be signed in to change notification settings - Fork 0
/
TODO
421 lines (318 loc) · 13.8 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
FIXME:
update web page and links etc
---
when repacking archives, don't specify '.'. Instead specify each extracted file or
directory. (Jens Schleusener)
---
FIXME: implement -O fully! also, use if for compression only! or?
---
$ arepack today.log.bz2 today.log.sz [today.log.bz2 exists, today.log.sz doesn't]
arepack: today.log.sz: no such file and cannot identify format from extension
Use of uninitialized value $fmt2 in string eq at /home/usel/sys/bin/arepack line 354.
today.log.bz2: extracted to `Unpack-4162/today.log'
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1348.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1349.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1350.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1351.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1352.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1353.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1354.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1355.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1356.
Use of uninitialized value $fmt in string eq at /home/usel/sys/bin/arepack line 1357.
arepack: ../today.log.sz: no such file and cannot identify format from extension
---
From Francois Marier: omit_garbage option, do not unpack
Thumbs.db
__MACOSX/
._filename
.Trashes
.DS_Store
See also: http://unmac.rubyforge.org/
http://github.com/fxn/unmac/blob/HEAD/lib/unmacer.rb
add compression level control arguments: store (0) or 1-9
allow user to add customer parameters to each archiver??
make use of modern tar options, like '--use-compress-program=prog'
cfg_use_tar_compress_option?
if making RPM, then upload SRPM as well!
improve messages printed from arepack. sometimes prints unnecessary messages like "data.tar.gz: extracted to `Unpack-2962/data'"
cat mode is not supported for lzop, because it won't write to standard out
unless we give it the -f option. Because of this we use lzop as if -c was
not supported. This should be fixed in the future.
--no-umask option from Reuben to fix permissions of extracted files. perhaps change owner etc.
aunpack -e is default - or perhaps with option to make it default?
aremove tool
how it should work:
unpack archive to a separate directory
for each unpacked file, check if it exists and is the same in the current directory.
if yes, remove that file in the current directory, and (optionally) from unpacked directory as well.
if no, remove that file from the unpacked directory
From Francois Marier <francois@debian.org>: check if pbzip2 is in the path and use it
rewrite atool implement extract_archive that is sensitive or not
add option to assume format with -F in some situations (like tar.gz)?
add option to always assume -e? and add some option for the reverse (-s, --single)
assume -e when passing a single directory as argument?
change back error messages to:
cannot foo `bar': No such file or directory
make function to format error message?
have autoconf generate files in debian subdir as well?
in readconfig, add datatype checking to options (bool/str/int/..)
add 'atest': test archive integrity
make sure atool has non-zero rc when acat can't find a file (.arc files and possibly other)
improve error message when some program doesn't exist (now gets unnecessary lines)
arepack -F.tar.bz2 foo.tar.gz would only gunzip then bzip2. tar not involved!
tests:
does gzip support --?
does tar support -z?
does tar support -I?
does tar support --?
does zip treat *?[]\ chars in filenames specially?
use "--" with gzip and others?
Reasons to rewrite in C:
- use manual ZIP code so that we can extract archive with '../foo' files too!
tar fails entirely when the archive contains '../xxx' members.
- the option --save-outdir in extract-to mode could be implemented
easier.
- multiple commands per action more easily (same with perl tho)
- always identify format using magic values, ignoring extension (fast!)
- i18n much easier (and faster)
Bugs
implement list command for single compressed archives? only gzip supports -l
consider:
apack dummy.tar.gz /etc/hostname
mkdir etc
aunpack dummy.tar.gz
Archive is now extracted with file as dummy/etc/hostname
Should it be dummy/hostname or etc-1/hostname instead?
Other formats
add support for SEA arc.
add support for szip (hard because its interface is different from gzip)
make an ACE archive (using winace stuff), test ACE extraction!
add support for DEB files
list: dpkg-deb -c FILE.deb
extract: dpkg-deb -x FILE.deb $outdir/CONTENTS
dpkg-deb -e FILE.deb $outdir/DEBIAN
JAR/ZIP:
Implement 'use_jar_for_zip' - use jar for zip archives (fastjar) - test
Rename 'use_zip_for_jar' - use zip for jar archives - test
Enable add command for jar archives!
Other features
make a script to detect system settings and configure atool appropriately
Make it possible to send additional arguments to RAR etc. engine configurable via command line?
especially adiff diff -ruN (note the N) option
--------------------------------------------------------------------------------
libarchive:
OpenArchive(fd)->archive
CloseArchive(archive)
GetType(archive)
BeginJob(archive)->job
PerformJob(job)
EndJob(job)
GetError(job)
ErrorManger(job,callback)
ProgressManager(job,callback)
CreateArchive(file)->archive
FileDescriptor
close()
read(data,amount)
write(data,amount)
seek(pos,whence)
FileIterator(job,fileselector,filecallback)
DeleteFile(job,fh)
ExtractFile(job,fh,outfd)
ReplaceFile(job,fh,infile)
AddFile(job,filename,extfile)
---
verbosity levels:
0 default, quiet
1 verbose, print filenames
2 very verbose, print long list format
extract: what to when local file exists, only when archive contains one file
(and not in interactive mode)
* (m)ove to subdirectory
* (r)ename
* move files to subdir, rename directory
* for interactive mode only: (o)verwrite
how to extract single compressed file
* delete original file afterwards
* keep original file
* don't extract at all (refuse)
add:
* apack: create new archive (refuse if existing exists)
* apack -a: add to existing archives (create if necessary)
---
MAJOR options:
aunpack -X -x [DIR] extract to dir
apack -A -a add files to archive (or create)
apack -C -c create only
als -L -l list files in archive
acat -P -c/-p print file (to standard out) in archive
adiff -D -d print difference between two archives
afind -F -f find files in archive (emulate find) [?]
arepack -R -r extract and pack archives
als ARCHIVE1..
-l -l (long) like ls
-f -f (classify) like ls
-m[TYPE] from all archives, list only files matching (remaining args are matches)
prev: als -e *.tar now: als *.tar
prev: als foo.tar f* now: als foo.tar -m f*
aunpack ARCHIVE1..
-X DIR where to extract to (otherwise be smart)
-m[TYPE] from all archives, extract only files matching (remaining args are matches)
apack ARCHIVE FILE1..
-a add files if archive already exists (without -a, refuse to overwrite existing)
-e all arguments are files (or dirs) to compress individually
-F FORMAT force file format
-f allow overwriting of archives (clobber). NOTE: for safety, this implies
--------------------------------------------------------------------------------
New rules format:
# This document shows how a future rules file format for atool could look. It
# was written to outline what was necessary for such a format. It is quite
# possible this format will never become a reality due to its complexity.
# use ; to separate commands!
# rules and configuration files are the same
noinclude # don't include default rules/configuration file!
# (otherwise /etc/atool.rules will be included first)
# useful if you want to make your own rules completely
set path_zip "foobar" # example of variable set in a configuration file
# (can only be done after option declared in format)
# the extract command defined here also handles extract-in (only outdir changes)
# the create command refuses to create an existing archive
# the add command will turn into a create command if the archive doesn't exist
# the print command outputs a file in the archive to standard out
# variable namespace:
# read-only variables - $outdir, $archive, $args
# format-local variables - declared with 'variable'
# options - declared with option
# FIXME: what about about special exit codes (zip)? CHECK atool
format {
name { zip jar } # a name for users to use in -F option (force format)
extension { .zip .jar } # possible file extensions (ignored?)
magic { PK\003\004 } # magic bytes (kinda regex?)
option string path_zip "zip"
option string path_unzip "unzip"
command extract { $path_unzip "-d" $outdir $archive $args }
command add,create { $path_zip "-r" $archive $args }
command print { $path_unzip "-p" $archive $args }
command list { $path_unzip "-l" $archive $args }
command common {
[ $verbosity > 1 ] "-v" # this format means: add "-v" only if $verbosity > 1
[ $verbosity < 0 ] "-qq" # (the first string after [] applies, or block if { } )
[ $verbosity == 0 ] "-q"
}
multiarchive true # specifies that this archive may contain multiple files
needs_cwd false # specifies that this archive doesn't need to cwd to outdir on extract
# (technically we could autodetect this by seeing of $outdir is referenced
# in the extract command)
}
format {
name { lha lzh }
extension { .lha .lzh }
magic { "..-lh[01234567 d]-" "..-lz[45s]-" } # what about >20 byte x ; 2 string -lh6-
option string path_lha "lha"
command common { $path_lha }
command extract { "x" $archive $args $outdirslash } # outdirslash = outdir that ends in slash
command add,create { "a" $archive $args }
command print { "p" $archive $args }
command list {
"a"
[ $verbosity >= 3 ] "v"
[ $verbosity = 2 ] "l"
[ $verbosity <= 1 ] "lq"
}
multiarchive true
needs_cwd false
}
format {
name { tar[+.]gz(ip)? }
extension { .tgz .tar.gz }
option string path_tar "tar" # this one assume tar supports -z
variable string opts
command common { $path_tar [ $verbosity >= 1 ] [ $opts = "v" ] } # a conditional
command print { xO ++ $opts ++ zf } # ++ means concatenate
command extract { x ++ $opts ++ zf -C $outdir } # -C works on Sun too
command list { t ++ $opts ++ zf }
command create { c ++ $opts ++ zf } # tar can append (-r), but not tar+gz
command common { $archive $args }
multiarchive true
needs_cwd false
}
format {
disabled # do not use this format at all
name { tar[+.]gz(ip)? }
extension { .tgz .tar.gz }
option string path_tar "tar" # this one does NOT assume tar supports -z
option string path_gzip "gzip"
variable string opts
command common { [ $verbosity >= 1 ] [ $opts = "v" ] }
command print { $path_gzip -cd $archive | $path_tar xO ++ $opts $args }
command extract { $path_gzip -cd $archive | $path_tar x ++ $opts $args }
command list { $path_gzip -cd $archive | $path_tar t ++ $opts $args }
command create { $path_tar c ++ $opts $args | $path_gzip > $archive }
multiarchive true
needs_cwd false
create_only true # FIXME: maybe not necessary? (see create/add above)
}
--------
TODO.v2:
better extraction semantics, cleaner code
* always use change to directory for extraction, never specify dir on cmd command-line?
use ../for instead of absolute dir
merges extract-to and extract
* core support for multiple commands per mode (think debian)
* better control of user arguments to external command (@args)
* better solution than multiarchivecmd - cleaned up
* more general commands:
test
* more general options
compression level
* stripext only based on format! (e.g. unpack gz then strip .gz, not .tar.gz)
want aunpack -e to be default!
remove dir: failure to remove a single file is fatal. chmod error isn't.
chmod errors should be ignored altogheter?
always chdir on extract?
unified handling of single & non-single (wrt extract etc)
acat displays *all* files or just first?
apack creating a single archive with many files = ERROR
extract-logic with -S:
tar xvfz ...
content-move DIR or atool --handle-extracted DIR
does unlink_dir handle . and .. (i.e. cur and up)?
write safety check for those
use File::Spec abs2rel
---
COMMANDS
apack - create a new archive
aunpack - extract files from an archive
als - list archive like ls
afind - list archive like find
acat - extract files from an archive to standard out
aless - extract files from an archive and pipe into a pager
amore - extract files from an archive and pipe into a pager
adiff - compare archives using diff
agrep - search files for a regular expression using grep
aegrep - search files for a regular expression using egrep
afgrep - search files for a regular expression using fgrep
arepack - extract archives of one type and create archives of another
acmp - compare archives using cmp
aadd - add files to an existing archive (may require unpacking)
atest - test archive integrity (may unpack)
amcfs - midnight commander virtual file system (extfs)
OPTIONS
-F,--format major format
--format-in format of files being read
--format-out format of files being created
-f,--files hereafter comes files (not archives)
-o,--options hereafter comes options for archiver programs
-X,--extract-to
-E,--explain
-S,--simulate
-q,--quiet display no regular output
-v,--verbose display a little more output
-z,--compression-level compression level for gzip, bzip2, "1" to "9"
ERRORS
display at max two errors: one detailed ($!), one general ("command failed")
LOCALIZATION
FORMATS
add SEA arc
add szip