diff --git a/README.md b/README.md index c32f538..4efa9ba 100644 --- a/README.md +++ b/README.md @@ -1,891 +1 @@ -![Linux build test](https://github.com/bepaald/signalbackup-tools/actions/workflows/test-linux-build.yml/badge.svg) ![macOs build test](https://github.com/bepaald/signalbackup-tools/actions/workflows/test-macos-build.yml/badge.svg) - -# signalbackup-tools -Tool to work with backup files generated by the Signal Android application (https://signal.org/). The tool is provided as-is, there may be bugs. The tool and I are not affiliated with or endorsed by the Signal Foundation. - -- [Important Note](#important-note) -- [Requirements](#requirements) -- [Obtaining](#obtaining) - - [Windows binary](#windows) - - [macOS](#macos) - - [Linux packages](#linux_packages) - - [Compiling](#compiling) -- [Running](#running) - - [Dump decrypted database to disk](#dump) - - [Dump media to disk](#dumpmedia) - - [Fixing broken backups](#fix) - - [Export HTML, TXT, CSV & XML](#export) - - [Cropping to certain conversations or dates](#crop) - - [Merging backups](#merge) - - [Importing conversations from Signal-Desktop](#desktop) - - [Importing conversations from Telegram / JSON file](#json) - - [Deleting/Replacing attachments](#deleting_attachments) - - [Operations for Signal Desktop](#desktop_functions) - - [Various](#various) - - [Advanced options](#advanced) -- [Future plans](#future-plans) -- [Donate](#donate) - -### Important Note - -Signal is an actively developed application and consequently, the database format changes regularly. Often the changes do not affect the backup file format or the working of this program, but every once in a while a change does break (some of) the functionality of this program. It has happened before and it will happen again. Sometimes I fix it within hours, but when I am short on time, it may take a little longer. Any breakage will be dealt with as soon as I have some spare time. - -### Requirements - -To compile this project, current stable released versions of the following are needed: -- A C++ compiler supporting at least the C++17 standard (tested with [GCC](https://gcc.gnu.org) 14.1.1 and [Clang](https://clang.llvm.org) 17.0.6, also tested and working with a few older compiler versions) -- [OpenSSL](https://www.openssl.org/) (any reasonably recent version from either the 3.X or 1.1x series) -- [SQLite3](https://www.sqlite.org/) (any reasonably recent version) -- Only on Linux: dbus. Optional, but required by default. See the [compiling](#compiling) section to build without `dbus`. If the program is compiled without `dbus`, operations that need to open the Signal Desktop client database will not work unless manually provided with the decrypted encryption key. - -### Obtaining - -**Windows binary** - -For the most recent Windows executable, check [the releases page](https://github.com/bepaald/signalbackup-tools/releases). This executable is a static build, cross compiled from my Arch Linux system. It is only minimally tested, but generally appears to work just fine. - -Note for Windows users: this is a command line application. This means you can not just double-click the executable to run it, you need to run it from a terminal. Common terminals for Windows are `cmd` (Command Prompt) and `PowerShell`. An example of running the program on Windows 10 can be seen [here](https://github.com/bepaald/signalbackup-tools/issues/148#issuecomment-1732375861). - -**Linux packages** -- For **Arch** users, an AUR package [is available](https://aur.archlinux.org/packages/signalbackup-tools-git). -- A pre-built rpm for **openSUSE** is available [here](https://software.opensuse.org/package/signalbackup-tools?search_term=signalbackup-tools), thanks to @marfrh (https://github.com/bepaald/signalbackup-tools/issues/205). -- The program is also available in `nixpkgs` as [`signalbackup-tools`](https://search.nixos.org/packages?channel=unstable&type=packages&query=signalbackup-tools) and can be installed on **NixOS** or any system that supports the [Nix package manager](https://nixos.org/manual/nix/stable/). For those looking for more information on installing and running the Nix package, or those wanting to help others, there is an issue where information can be found and posted [here](https://github.com/bepaald/signalbackup-tools/issues/149). -- Alternatively, a Dockerfile has been kindly provided by David J. Meier, and is available at his gitlab page: . - -**macOS** - -A homebrew formula is provided in [homebrew/signalbackup-tools.rb](https://raw.githubusercontent.com/bepaald/signalbackup-tools/master/homebrew/signalbackup-tools.rb). On modern macOS versions, with [homebrew set up](https://brew.sh/), compiling should be as simple as running `brew install --HEAD --formula [path/to/signalbackup-tools.rb]`. Once installed, the program can be upgraded by running `brew upgrade --fetch-HEAD`. - -Manually compiling should also be possible assuming the dependencies are installed, for more info see [here](https://github.com/bepaald/signalbackup-tools/issues/9), or more recently [here](https://github.com/bepaald/signalbackup-tools/issues/85). macOS users might also consider the aforementioned [Nix package](https://search.nixos.org/packages?channel=unstable&type=packages&query=signalbackup-tools). - -**Compiling** - -To compile the program, three main options are available: - -- CMake. Make sure to have `cmake` installed. On Linux this method also requires `pkg-config` (unless building without `dbus`). From the project directory, run: - ```Shell - $ cmake -B build -DCMAKE_BUILD_TYPE=Release - $ cmake --build build -j $(nproc) - ``` - To build without `dbus` (and `pkg-config`), add `-DWITHOUT_DBUS=1` to the first command. - -- The bash script. In the project directory is a bash script `BUILDSCRIPT.bash`. Simply run it: - ```Shell - $ ./BUILDSCRIPT.bash - ``` - To build without `dbus`, add `--config without_dbus` to the above command. the script can of course be edited at will to change compilation behavior. The flags can also be changed on the command line when running, for example to build with `clang++` instead of `g++`, simply run `$ CXX=clang++ ./BUILDSCRIPT.bash`. - -- Manually. The program can be manually compiled simply by running `g++ -std=c++20 */*.cc *.cc -lcrypto -lsqlite3`. On linux, by default one needs to add the location of the dbus headers and libraries (simplest way, add: `$(pkg-config --cflags --libs dbus-1)`). Alternatively, to build on Linux without `dbus`, add `-DWITHOUT_DBUS=1`. On macOS, the program must be linked to the Security and CoreFoundation frameworks by adding `-framework Security -framework CoreFoundation` to the build command. Any compiler flags you feel useful can be added, personally I use at least `-O3 -Wall -Wextra`. When compiling with an old compiler version (gcc 8.x or clang <= 7), also add the -lstdc++fs flag and replace -std=c++20 with -std=c++17. - -### Running - -> [!TIP] -> In all examples below, one or more passphrases are provided on the command line. If so desired, these can be omitted in which case you are prompted for the passphrase at runtime. - -In its simplest form, this tool is run as such: - -``` -signalbackup-tools [input] [passphrase] -``` - -This will open the file `input` using the provided `passphrase`, and do nothing with it. If an output file is supplied the backup is written to that file: - -``` -signalbackup-tools [input] [passphrase] --output [output] -``` - -Optionally, a new passphrase can be provided for the output file with the option `--opassphrase`, or `-op` for short. If not provided, as in the above example, the input passphrase is used again. - - -**Dump decrypted database to disk** - -The program can dump the decrypted backup components to a directory, or read the contents of a directory and pack and encrypt it back into a valid backup file. When dumping, make sure the directory to dump to is empty to start with. In theory, the decrypted files could be edited before re-encrypting. The tool can be called the same as above, except the output should be a directory: - - -``` -signalbackup-tools [input] [passphrase] --output [outputdirectory] -``` - -To skip exporting media (like message attachments, avatars and stickers), add the option `--onlydb`. To re-encrypt the contents of a directory into a valid backup file, use the directory as `input` and provide the `--output` and `--opassphrase` options. - -
- Example (click to show) -

- -```Shell -[~/programming/signalbackup-tools] $ mkdir RAWBACKUP -[~/programming/signalbackup-tools] $ ll RAWBACKUP/ -total 0 -[~/programming/signalbackup-tools] $ ./signalbackup-tools ~/PHONE/signal-2019-07-14-06-59-26.backup 949543591444534240555456749437 --output RAWBACKUP/ -IV: (hex:) 13 3f 94 13 be 5a 6d 1c 97 d0 20 88 4e f8 64 46 (size: 16) -SALT: (hex:) 5e 89 ec d8 f3 99 68 5b 9b a6 8b d8 3b b7 7d 8f e5 6a 2a 03 bb 2c c0 b9 f6 a1 0e bc bf ba 1a 25 (size: 32) -BACKUPKEY: (hex:) 38 4c a3 1c 17 9c f7 9b 27 30 98 bc 13 bf b6 5d 1d 90 df 13 c1 11 79 a4 ef d0 65 75 b9 55 cc 61 (size: 32) -CIPHERKEY: (hex:) 25 15 18 5f ac 06 3f 13 b5 0d c6 eb 8b e0 84 34 13 3f 84 f7 77 9b f6 ec 44 00 cb c0 77 2d 70 1f (size: 32) -MACKEY: (hex:) f3 00 34 77 1f a3 74 74 56 42 5e ad 6b d7 71 bf 40 7f e0 4f df 3a d1 1a 22 79 91 3a 97 73 88 28 (size: 32) -COUNTER: 322933779 -Reading backup file... -FRAME 42337 (100.0%)... Read entire backup file... -done! -Writing HeaderFrame... -Writing DatabaseVersionFrame... -Writing Attachments... -Writing Avatars... -Writing SharedPrefFrame(s)... -Writing StickerFrames... -Writing EndFrame... -[~/programming/signalbackup-tools] $ ll RAWBACKUP/ -total 2204384 --rw-r--r-- 1 bepaald bepaald 118871 jul 19 15:40 Attachment_1000_1518474349909.bin --rw-r--r-- 1 bepaald bepaald 16 jul 19 15:40 Attachment_1000_1518474349909.sbf --rw-r--r-- 1 bepaald bepaald 30017 jul 19 15:40 Attachment_1001_1518475497752.bin - [...] --rw-r--r-- 1 bepaald bepaald 9363456 jul 19 15:40 database.sqlite --rw-r--r-- 1 bepaald bepaald 4 jul 19 15:40 DatabaseVersion.sbf --rw-r--r-- 1 bepaald bepaald 2 jul 19 15:40 End.sbf --rw-r--r-- 1 bepaald bepaald 54 jul 19 15:40 Header.sbf --rw-r--r-- 1 bepaald bepaald 96 jul 19 15:40 SharedPreference_0.sbf --rw-r--r-- 1 bepaald bepaald 97 jul 19 15:40 SharedPreference_1.sbf -[~/programming/signalbackup-tools] $ ./signalbackup-tools RAWBACKUP/ --output NEWBACKUPFILE --opassphrase 949023591444534240555368549425 -Exporting backup to 'NEWBACKUPFILE' -Writing HeaderFrame... -Writing DatabaseVersionFrame... -Writing SqlStatementFrame(s)... - Dealing with table 'sms'... 34595/34595 entries...done - Dealing with table 'mms'... 2370/2370 entries...done - Dealing with table 'part'... 1934/1934 entries...done - Dealing with table 'thread'... 29/29 entries...done - Dealing with table 'identities'... 21/21 entries...done - Dealing with table 'drafts'... 0/0 entries... - Dealing with table 'push'... 0/0 entries... - Dealing with table 'groups'... 10/10 entries...done - Dealing with table 'recipient_preferences'... 67/67 entries...done - Dealing with table 'group_receipts'... 1320/1320 entries...done - Dealing with table 'job_spec'... 1/1 entries...done - Dealing with table 'constraint_spec'... 0/0 entries... - Dealing with table 'dependency_spec'... 0/0 entries... - Dealing with table 'sticker'... 0/0 entries... -Writing SharedPrefFrame(s)... -Writing EndFrame... -Done! -[~/programming/signalbackup-tools] $ cmp ~/PHONE/signal-2019-07-14-06-59-26.backup NEWBACKUPFILE && echo "Files are identical" -Files are identical -[~/programming/signalbackup-tools] $ -``` - -_NOTE The original and new files are not actually guaranteed to be identical, it just so happens that in this case the AvatarFrames are read from the filesystem in the order they appeared in the original._ - -

-
- -**Dump media to disk** - -##### Dumping message attachments - -To only export media attachments from one or all of the threads in a backup, run with `--dumpmedia` as follows: - -``` -signalbackup-tools [input] [passphrase] --dumpmedia [outputdirectory] -``` - -Where `outputdirectory` is an empty directory, or does not exist (in which case it will be created). - -To limit the export to certain threads, the option `--limittothreads [LIST_OF_THREADS]` can be added. The list of threads can contain both ranges and comma separated values, e.g. `--limittothreads 1,2,3,8-16,20`. The thread numbers can be obtained from `--listthreads`. Additionally, threads can be identified by a string representing the display name, phone number or username of the recipient: `--limittothreadsbyname "Alice","Family Group","+14255550123"`. Similarly, the option `--limittodates [LIST_OF_DATES]` will limit the output to media from the time periods listed. For the format of the date list, see the [crop to dates](#crop) option. - -Normally, stickers are included in the media export, as they are normal attachments in the database. To prevent this, add the option `--excludestickers`. - -##### Dumping avatars - -To only export avatars from one or all contacts in a backup, run with `--dumpavatars` as follows: - -``` -signalbackup-tools [input] [passphrase] --dumpavatars [outputdirectory] -``` - -Where `outputdirectory` is an empty directory, or does not exist (in which case it will be created). - -To limit the export to certain contacts, add the option `--limitcontacts [LIST_OF_CONTACTS]`. The list should look like this: `"Alice,Bob,John Doe(,...)"`, where each name is exactly as it appears in Signal's conversation overview or from this program's `--listrecipients` output. - -**Fixing broken backups** - -> [!IMPORTANT] -> Around version 6.26 of Signal Android (circa July 2023), the backup format was changed in a way that makes it impossible to recover from data corruption that happens across fame boundaries. This functionality is disabled for newer backups. In other cases (corruption within a single frame, the occasional bug in Signal), part of the data could possibly still be recovered, though it might require a custom function. You could always open an issue if you need help. Note that this type of corruption, where only a single frame is affected, is rare and recent versions of Signal Android usually deal with this case quite well. - -At the moment it has been used successfully to fix backups that were corrupted for some reason (see https://github.com/signalapp/Signal-Android/issues/8355, and https://community.signalusers.org/t/tool-to-re-encrypt-signal-backup-optionally-changing-password-or-dropping-bad-frames/6497). If you want to fix a broken backup, run the tool as follows: - -``` -signalbackup-tools [input] [passphrase] --output [outputfile] (--opassphrase [newpassphrase]) -``` - -_NOTE: if the corruption happens outside of attachment data, which is usually unlikely, chances of recovery are much lower._ - -If the output passphrase is omitted, the input passphrase is used to encrypt the new backup file. If the 'input' is a directory, it is assumed to contain a decrypted dump of the backup (as made by this tool) and the input passphrase can be omitted. In this case the output passphrase is required, unless 'output' is also a directory. - -If the 'output' is omitted only the scan is done, and the broken message is identified, giving you the option to delete it from the phone. The corrupted attachment data is dumped to file. - -
- Example (click to show) -

- -```Shell -[~/programming/signalbackup-tools] $ ./signalbackup-tools CORRUPTEDSIGNALBACKUPS/signal-2019-05-20-05-29-06.backup3 949543593573534240555368549437 --output NEWBACKUPFILE --opassphrase 949543593573534240555368549437 -signalbackup-tools source version 20190926.164320 -IV: (hex:) 12 16 72 95 7a 00 68 44 7e cf 7d 20 26 f9 d3 7d (size: 16) -SALT: (hex:) cc 03 85 02 61 97 eb 5b ed 3e 05 00 c4 a8 77 40 28 08 aa 9f e5 a8 00 74 b4 f8 56 aa 24 57 a9 5d (size: 32) -BACKUPKEY: (hex:) 8f ff df 2b 9f 96 73 9a 63 95 0f ea 3f b1 e5 a4 87 12 19 ca 93 31 86 2a 60 3f 41 ef 6d a4 08 44 (size: 32) -CIPHERKEY: (hex:) ce 53 c1 f2 92 4b e3 b8 e1 56 85 61 14 96 82 8b 83 7f 07 21 83 52 1a c2 3f 6b 16 83 3e 33 94 a3 (size: 32) -MACKEY: (hex:) c2 77 af 1e 4b 05 db 62 52 57 af 8a d6 a4 d4 e9 6c 93 53 81 9a e7 6f 12 2c ce 13 8f b3 5e 8d 3a (size: 32) -COUNTER: 303461013 -Reading backup file... -FRAME 37636 (071.5%)... -WARNING: Bad MAC in attachmentdata: theirMac: (hex:) 30 75 bb b3 fb 65 a5 2a 5f b5 - ourMac: (hex:) ff f2 37 c1 f0 d4 2c 67 a3 cf 6c 41 55 bd 9c 1d 85 84 0e 66 96 ae 52 4e 90 b5 a3 37 33 3c b4 fc - -WARNING: Bad MAC in frame, trying to print frame info: -Frame number: 37637 - Type: ATTACHMENT - - row id : 1317 (8 bytes) - - attachment id : 1536842122829 (8 bytes) - - length : 1516761 (8 bytes) - - attachment : (hex:) 47 49 46 38 39 61 e0 01 09 01 f7 00 30 00 ff 00 01 00 02 01 00 05 01 00 05 ... (1516761 bytes total) -Frame is attachment, it belongs to entry in the 'part' table of the database: - - _id : 1317 - - mid : 1552 - - seq : 0 - - ct : image/gif - - name : (NULL) - - chset : (NULL) - - cd : (NULL) - - fn : (NULL) - - cid : (NULL) - - cl : (NULL) - - ctt_s : (NULL) - - ctt_t : (NULL) - - encrypted : (NULL) - - pending_push : 0 - - _data : /data/user/0/org.thoughtcrime.securesms/app_parts/part2625620938717109701.mms - - data_size : 1516761 - - file_name : (NULL) - - thumbnail : (NULL) - - aspect_ratio : 2 - - unique_id : 1536842122829 - - digest : (NULL) - - fast_preflight_id : 5897879359555196456 - - voice_note : 0 - - data_random : (hex:) f7 1e 34 f3 ba 07 34 44 56 04 15 dc 80 88 b7 10 9e c1 18 80 65 c7 7f 60 d9 cc 0f c9 d4 95 ce b4 - - thumbnail_random : (hex:) 14 f7 79 84 e5 a5 68 fe 98 a4 cb db 36 1f 6f c8 ca 3c 57 45 60 e2 d2 f2 f6 ee 42 71 42 7b 8e d7 - - width : 480 - - height : 265 - - quote : 0 - - caption : (NULL) - -Which belongs to entry in 'mms' table: - - _id : 1552 - - thread_id : 1 - - date : 2018-09-13 14:35:22 +0200 (1536842122790) - - date_received : 2018-09-13 14:35:22 +0200 (1536842122809) - - msg_box : 10485783 - - read : 1 - - m_id : (NULL) - - sub : (NULL) - - sub_cs : (NULL) - - body : - - part_count : 1 - - ct_t : (NULL) - - ct_l : (NULL) - - address : +316XXXXXXXX - - address_device_id : (NULL) - - exp : (NULL) - - m_cls : (NULL) - - m_type : 128 - - v : (NULL) - - m_size : (NULL) - - pri : (NULL) - - rr : (NULL) - - rpt_a : (NULL) - - resp_st : (NULL) - - st : (NULL) - - tr_id : (NULL) - - retr_st : (NULL) - - retr_txt : (NULL) - - retr_txt_cs : (NULL) - - read_status : (NULL) - - ct_cls : (NULL) - - resp_txt : (NULL) - - d_tm : (NULL) - - delivery_receipt_count : 1 - - mismatched_identities : (NULL) - - network_failures : (NULL) - - d_rpt : (NULL) - - subscription_id : -1 - - expires_in : 0 - - expire_started : 0 - - notified : 0 - - read_receipt_count : 0 - - quote_id : 0 - - quote_author : (NULL) - - quote_body : (NULL) - - quote_attachment : -1 - - shared_contacts : (NULL) - - quote_missing : 0 - - unidentified : 0 - - previews : (NULL) -Trying to dump decoded attachment to file 'attachment_1552.bin' -FRAME 37637 (071.6%)... Failed to read next frame (4294967295 bytes at filepos 1611402482) -Starting bruteforcing offset to next valid frame... -Checking offset 802590 bytes -GOT GOOD MAC AT OFFSET 802591 BYTES! -Now let's try and find out how many frames we skipped to get here.... -Checking if we skipped 0 frames... nope! :( -Checking if we skipped 1 frames... nope! :( -Checking if we skipped 2 frames... nope! :( -Checking if we skipped 3 frames... YEAH! -Frame number: 37641 - Type: SQLSTATEMENT - - (statement: "INSERT INTO part VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)" (83 bytes) - - (uint64 parameter): "1319" - - (uint64 parameter): "1554" - - (uint64 parameter): "0" - - (string parameter): "image/jpeg" - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (uint64 parameter): "0" - - (string parameter): "/data/user/0/org.thoughtcrime.securesms/app_parts/part7691613523019485618.mms" - - (uint64 parameter): "133247" - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (uint64 parameter): "1537091993419" - - (bool parameter) : "true" (value: "1") - - (bool parameter) : "true" (value: "1") - - (uint64 parameter): "0" - - (binary parameter): "(hex:) d3 a6 ea 3c 27 90 0f 12 74 71 54 ac 94 92 0f 08 30 04 e0 e1 b3 41 36 37 6d 8a 5d 44 fb 23 6e b5" - - (bool parameter) : "true" (value: "1") - - (uint64 parameter): "720" - - (uint64 parameter): "1280" - - (uint64 parameter): "0" - - (bool parameter) : "true" (value: "1") - -Got frame, breaking -FRAME 39960 (100.0%)... Read entire backup file... -done! -Removing 1 bad frames from database... -Exporting backup to 'NEWBACKUPFILE' -Writing HeaderFrame... -Writing DatabaseVersionFrame... -Writing SqlStatementFrame(s)... - Dealing with table 'sms'... 32752/32752 entries...done - Dealing with table 'mms'... 2212/2212 entries...done - Dealing with table 'part'... 1814/1814 entries...done - Dealing with table 'thread'... 27/27 entries...done - Dealing with table 'identities'... 19/19 entries...done - Dealing with table 'drafts'... 0/0 entries... - Dealing with table 'push'... 0/0 entries... - Dealing with table 'groups'... 10/10 entries...done - Dealing with table 'recipient_preferences'... 63/63 entries...done - Dealing with table 'group_receipts'... 1195/1195 entries...done - Dealing with table 'job_spec'... 1/1 entries...done - Dealing with table 'constraint_spec'... 0/0 entries... - Dealing with table 'dependency_spec'... 0/0 entries... -Writing SharedPrefFrame(s)... -Writing EndFrame... -Done! -[~/programming/signalbackup-tools] $ -``` - -

-
- -**Export HTML, TXT, CSV & XML** - -##### Export to HTML - -_NOTE: Note that while the the generated HTML is heavily inspired by Signal's look it does not aim to be a perfect reproduction of it. The generated HTML and CSS are only tested on Firefox (but both pass W3C validation). It is possible that some (very) old backups are not supported. If there is demand, support for older databases may be added in the future._ - -To export your messages to HTML, use `--exporthtml [DIRECTORY]`. To limit the output to certain threads the option `--limittothreads [LIST_OF_THREADS]` can be added. The list of threads can contain both ranges and comma separated values, e.g. `--limittothreads 1,2,3,8-16,20`. The thread numbers can be obtained from `--listthreads`. Additionally, threads can be identified by a string representing the display name, phone number or username of the recipient: `--limittothreadsbyname "Alice","Family Group","+14255550123"`. Similarly, the option `--limittodates [LIST_OF_DATES]` will limit the output to messages within the time periods listed. For the format of the date list, see the [crop to dates](#crop) option. Because writing out all media files can be a long process, the option `--append` can be added to reuse any existing media files, only new media and the HTML-files will be rewritten. Example: - -``` -signalbackup-tools [input] [passphrase] --exporthtml [directory] -``` - -Because browsers may have difficulty loading an entire conversation if it consists of a large number of messages, the option `--split [N]` can be added to split the output HTML in multiple pages. The optional number `N` is the maximum number of messages on each generated page (default: 1000). Alternatively the option `--split-by [period]` will generate separate pages for each calender `[period]`. Currently supported periods are 'year', 'month', 'week', and 'day'. Note the `-split` and `--split-by` options are mutually exclusive. - -By default, the function will create a HTML page resembling Signal's dark mode. If you prefer a light theme, add the `--light` option. If you want to be able to switch between the two modes without generating a new HTML page, you could add the `--themeswitching` option to the command. This will add a button to switch themes. Be aware this causes the page to use JavaScript and cookies. - -Other options that can be used together with `--exporthtml`: - -- `--searchpage` Generates a page from which conversations can be searched. This page requires JavaScript and generates an extra file named `searchidx.js` in the directory to facilitate searching. -- `--includecalllog` Generates a page showing the call-log. -- `--stickerpacks` Generates an overview of installed and known stickerpacks. -- `--includeblockedlist` Generates an overview of blocked contacts in the backup. -- `--addexportdetails` Adds some meta information about the backup (like size, filename, and database version) and this tool to the generated pages when printing. -- `--includesettings` Generates a page showing settings found in the backup file. -- `--includefullcontactlist` Generates a page showing _all_ contacts present in the database, including contacts with whom no thread exists, who are blocked or hidden, or who appear in your system contact list and may not have Signal installed. -- `--allhtmlpages` Enables all of the above options, plus `--themeswitching`. Any specific option can be excluded by adding `--no-(option)` after this option on the command line. -- `--includereceipts` Adds available information from read/delivery receipts to outgoing messages as a popup when hovering the checkmarks. Be aware this has the potential to significantly slow down page loading for larger conversations. In this case it is recommended to also use the `--split [N]` option to limit the page size. - -> [!NOTE] -> A big thanks to [Gertjan van der Burg](https://github.com/GjjvdBurg)! While HTML export was always a planned feature of this program, it would not have happened this quickly without his project [signal2html](https://github.com/GjjvdBurg/signal2html). The HTML this function generates is modified from the template from his original project. - -> [!NOTE] -> An experimental feature to export Signal Desktop data to HTML exists. See [Operations for Signal Desktop](#desktop_functions). - -##### Export to TXT - -To export to plain text use `--exporttxt [DIRECTORY]`. Some data is omitted from this export, such as attachment data and quotes. To limit the output to certain threads the option `--limittothreads [LIST_OF_THREADS]` can be added. The list of threads can contain both ranges and comma separated values, e.g. `--limittothreads 1,2,3,8-16,20`. The thread numbers can be obtained from `--listthreads`. Additionally, threads can be specified by display name, phone number or username: `--limittothreadsbyname "Alice","Group Name","+14255550123"`. Similarly, the option `--limittodates [LIST_OF_DATES]` will limit the output to messages within the time periods listed. For the format of the date list, see the [crop to dates](#crop) option. Example: - -``` -signalbackup-tools [input] [passphrase] --exporttxt [directory] -``` - -The output will look something like this: - -``` -[2023-07-10 01:23] Where are you? -[2023-07-10 01:25] I'm at the beach. -[2023-07-10 01:26] *** sent "Signal-1.jpeg" -[2023-07-10 01:27] Come home. You haven't washed the dishes. (Bob: 😮) -``` - -##### Export to CSV - -To export the tables to a file of comma separated values (CSV), use `--exportcsv [table1]=[filename1],[table2]=[filename2],...`. To get all messages from the database, only the 'message' table needs to be exported. To get all messages out of older databases, the 'sms' and 'mms' tables need to be exported. - -##### Export to XML - -To export to XML file, use `--exportxml [filename]`. The exported XML file is intended to be compatible with [SMS Backup & Restore](https://www.synctech.com.au/sms-backup-restore/)'s format (see the [schema](https://synctech.com.au/wp-content/uploads/2018/01/sms.xsd_.txt) and [description](https://www.synctech.com.au/sms-backup-restore/fields-in-xml-backup-files/)). It has been successfully used to import Signal messages into messaging apps on phones, and — when Signal still supported this — importing these SMS into Signal. This way some messages could be moved from Signal Android to Signal iOS (which does not currently support backups). The XML format (and SMS in general) does not support many features found in Signal (quotes, for example), so the exported file will not be a full representation of the backup's contents. The resulting XML file will likely be quite large, around 30% larger than the input backup file, due to the base64 encoding of attachments. - -_NOTE: Over time changes in Signal's database format have broken specifically this feature multiple times. It is not very well tested and its current working status is not very well known._ - -**Cropping to certain conversations or dates** - -_NOTE: This feature is experimental (even more so than the others). I test it fairly well myself, but I have no knowledge of it being used by other people. If you use it, please let me know if it works for you._ - -##### Crop to threads - -To crop a backup file to certain threads, run: - -``` -signalbackup-tools [input] [passphrase] --croptothreads [list-of-threads] --output [output] (--passphrase [newpassphrase]) -``` - -Where the list of threads are the ids as reported by `signalbackup-tools [input] [passphrase] --listthreads`. The list supports commas for single ids and hyphens for ranges, for example: `--croptothreads 1,2,5-8,10`. Additionally, threads can be specified by display name, phone number or username: `--croptothreadsbyname "Alice","Some Group","+14255550123"`. - -##### Crop to dates - -To crop a backup file to certain dates, run: - -``` -signalbackup-tools [input] [passphrase] --croptodates begindate1,enddate2(,begindate2,enddate2(,...)) --output [output] (--opassphrase [newpassphrase]) -``` - -The 'begindate' and 'enddate' must always appear in pairs and can be either in "YYYY-MM-DD hh:mm:ss" format or as a single number of [milliseconds since epoch](https://en.wikipedia.org/wiki/Unix_time). For example, the following commands are equivalent (in my time zone) and both crop the database to the messages between Sept. 18 2019 and Sept 18 2020: `--croptodates "2019-09-18 00:00:00","2020-09-18 00:00:00"` or `--croptodates 1568761200000,1600383600000`. - -**Merging backups** - -_NOTE: Although this feature generally seems to work quite well, it requires constant maintenance to keep up with changes in Signal's internal database. You may encounter problems if this program happens to be slightly out of date when you run it. As always, feel free to open an issue to notify me of problems._ - -To merge two backups, the backups must be at compatible database versions. The database version can be found by running `signalbackup-tools [input] [passphrase] --listthreads`. Though many database versions work perfectly fine together, sometimes breaking changes are made. For example two databases at versions before and after 168 can not be merged successfully. Before opening an issue, if needed, import the backups into Signal and export them again to get them updated and at equal versions. To import all threads from one database into another, run: - -``` -signalbackup-tools [first_database] [passphrase] --importthreads ALL --source [second_database] --sourcepassphrase [passphrase] --output [output_file] (--opassphrase [output passphrase]) -``` - -It is recommended to use the larger (containing the most data (contacts, threads,...)) as the 'first_database' and the smaller one source. If not all threads should be imported from the source, a list of thread ids can be supplied (e.g. `--importthreads 1,2,3,8-16,20`). The thread ids can be determined from the output of `--listthreads`. Threads can additionally be specified by display name, phone number or username by using `--importthreadsbyname "Bob","Family Group","+14255550123"`. - -Note this function does not automatically discard duplicate messages. If the backups you are merging contain (partly) the same messages — for example if they originate from some common backup/installation — you will probably want to [crop the source backup by date](#crop) before merging so it only contains messages not in the target. For newer databases, omitting this step will cause errors, as Signal does not allow duplicate messages in its database anymore. - -If you use this option and read this line, I would really appreciate it if you let me know the results. Either send me a mail (basjetimmer at yahoo-dot-com) or feel free to just open an issue on the tracker for feedback. - -**Importing conversations from Signal-Desktop** - -_NOTE: This feature is highly experimental, problems may occur. Make sure to always keep a copy of your original backup file. Feedback is appreciated_ - -_NOTE 2: While this program will compile and work with almost any version of SQLite3, this specific feature requires that the SQLite3 version used is at least as new as the one used by the Signal Desktop client. Older versions will likely not be able to read Signal Desktop's database. For example, as of writing, the version available in Ubuntu is older than the one used by Signal Desktop. For Ubuntu(-like) distributions a PPA exists with a more up-to-date version [here](https://launchpad.net/~linuxgndu/+archive/ubuntu/sqlitebrowser) (disclaimer: I am not affiliated with this PPA, and never used it)._ - -To import conversations from a Signal-Desktop installation, run: -``` -signalbackup-tools [input] [passphrase] --importfromdesktop --output [output] (--opassphrase [newpassphrase]) -``` - -As with all commands this program supports, `[input]` is an existing Signal Android backup file. The messages from the desktop are imported into this backup file. - -Make sure your Signal-Desktop instance is cleanly shut down before running, if this fails for some reason the option `--ignorewal` can be added (the program will warn about this and exit if necessary), but this may cause the database to appear in an out-of-date state. This function requires some files belonging to your Signal Desktop installation: `config.json` and `sql/db.sqlite`. It tries to locate them at their default location (Linux: `~/.config/Signal/`, macOS: `~/Library/Application Support/Signal/`, Windows: `C:/Users//AppData/Roaming/Signal/`). If this fails, the default location for Signal Beta is attempted. In some cases one may want to specify the location this tool should look for the files. For example if wanting to work with the Signal Desktop Beta data, while the non-Beta is also present (it would be found first), or the files are in some non standard location (a backup for example). In such a case, the directory containing the files (_not_ the files themselves) can be passed by using `--desktopdir `. - -To limit the message import to a certain time frame, the option `--limittodates ` can be added. The format of the list of dates is identical to that of the [croptodates function](#crop-to-dates). In most cases, the option `--autolimitdates` can be used to automatically only import messages from the Desktop database before the first, or after the last message in the input backup. - -This function has some limitations, most notably the contacts referenced in the data that is to be imported must be present in the Android backup. If a message is found that is sent by/to an unknown contact, it is skipped. For other limitations see [here](https://github.com/bepaald/signalbackup-tools/issues/57#issuecomment-1329475240). - -**Importing conversations from Telegram / JSON file** - -The program has successfully been used to import messages from a Telegram export (in JSON format). Telegram's JSON format is [publically documented](https://core.telegram.org/import-export), so any data that can be converted to this format can be imported. - -This feature will be better documented in the future. For now, more details are available [here](https://github.com/bepaald/signalbackup-tools/issues/153), and any questions and remarks can be added there. General usage: - -``` -signalbackup-tools [INPUT] [PASSPHRASE] --importtelegram [JSONFILE] -o [OUTPUT] -``` - -The program will attempt to map the contacts present in the JSON file to those present in the Android backup. It is important all contacts exist in the Android backup, new contacts can not be created. For any JSON contact that the program can not automatically map, this mapping must be done manually using `--mapjsoncontacts`. - -Other related options: -- `--importjson [JSONFILE]` Simply an alias for `--importtelegram`. -- `--listjsonchats [JSONFILE]` Lists the chats found in the JSON file. This option does not require an Android backup to be passed as `[INPUT]`. -- `--selectjsonchats [list-of-indices]` Only import chat in the list. The indices are obtained from `--listjsonchats`. -- `--jsonprependforward` Forwarded messages are marked as such in Telegram, but not in Signal. This option prepends forwarded messages with the string "_Forwarded from NAME:_". -- `--preventjsonmapping` If the auto mapping makes a mistake for any reason (for example, multiple contacts with the same name), `--preventjsonmapping "Bob Smith"` will prevent the auto mapping of that specific name. It will then need to be mapped manually (using a unique identifier such as the id) with `--mapjsoncontacts`. -- `--jsonmarkdelivered` The Telegram export does not contain message delivery information. This option marks all messages imported from the JSON file as 'delivered'. Defaults to true. -- `--jsonmarkread` This option marks all messages imported from the JSON file as 'read'. Defaults to false. - -**Deleting/Replacing attachments** - -_NOTE: This feature is highly experimental, problems may occur. Make sure to always keep a copy of your original backup file. Feedback is appreciated_ - -##### Deleting attachments - -To remove attachments from the database, while keeping the message bodies (for example to shrink the size of the backup) the option `--deleteattachments` can be used: - -``` -signalbackup-tools [input] [passphrase] --deleteattachments --output [output] (--opassphrase [newpassphrase]) -``` - -To further specify precisely which attachments are to be deleted, the following options can be added: -- `--onlyinthreads [list-of-threads]`. The list supports commas for single ids and hyphens for ranges, for example: `--onlyinthreads 1,2,5-8,10`. To obtain the number-id of threads use `--listthreads`. -- `--onlyolderthan [date]`/`--onlynewerthan [date]`. Where 'date' supports the same format as the `--croptodates` option ([here](#crop)). -- `--onlylargerthan [size]`. The size is specified in bytes. -- `--onlytype [mime type]`. This argument can be repeated. Only selects attachments which match 'mime type*' (note the asterisk). For example `--onlytype image/j` will match both 'image/jpg' and 'image/jpeg'. To delete all image type attachments, simply use `--onlytype image`. -- `--prependbody [string]`/`--appendbody [string]`. Prepend or append the message body with the supplied string. If the message was otherwise empty, the body will equal the supplied string. Otherwise, it will be appended or prepended and a blank line will be inserted automatically. Suggested use: `--prependbody "(One or more media attachments for this message were deleted)"`. - -When adding this specifying options, only attachments which match _all_ given options are deleted. - -##### Replacing attachments - -There are two ways to replace attachments in a database. Currently attachments can only be replaced with image files of type jpeg, png or gif (non-animated). - -###### Option 1 - -To replace attachments in a backup file one can use the option `--replaceattachments [type=image,type2=image2,...]`. Where '_type_' is a mime type and image is the new attachment. To narrow the selection of attachments being replaced, all the same options mentioned above can be used (`--onlyinthreads`, `--onlyolderthan`, `--onlylargerthan`, `--onlytype`). - -
- Example and screenshots (click to show) -

- -``` -$ ls -lh -total 3,0G --rw-r--r-- 1 bepaald bepaald 148 feb 5 21:23 GIF.png --rw-r--r-- 1 bepaald bepaald 195 feb 5 21:23 IMAGE.png --rw-r--r-- 1 bepaald bepaald 3,0G feb 13 15:46 signal-2022-02-14-00-00-00.backup --rw-r--r-- 1 bepaald bepaald 189 feb 5 21:23 VIDEO.png -$ ../signalbackup-tools signal-2022-02-14-00-00-00.backup 111112222233333444445555566666 --replaceattachments "image=IMAGE.png,image/gif=GIF.png,video=VIDEO.png" -o signal-2022-02-14-00-00-01.backup -signalbackup-tools (../signalbackup-tools) source version 20220111.170852 (OpenSSL) -IV: (hex:) c3 05 25 [...] -SALT: (hex:) 90 38 9e [...] -BACKUPKEY: (hex:) 33 78 2f [...] -CIPHERKEY: (hex:) bb dc b0 [...] -MACKEY: (hex:) a3 92 76 [...] -COUNTER: 3271894439 -Reading backup file... -FRAME 39538 (100.0%)... Read entire backup file... - -done! -Checking to replace attachment: image/jpeg -Replaced attachment at 1/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 2/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 3/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 4/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 5/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 6/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 7/2046 with file "IMAGE.png" -Checking to replace attachment: image/png -Replaced attachment at 8/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 9/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 10/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 11/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 12/2046 with file "IMAGE.png" -Checking to replace attachment: video/x-matroska -Replaced attachment at 13/2046 with file "VIDEO.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 14/2046 with file "IMAGE.png" -Checking to replace attachment: video/mp4 -Replaced attachment at 15/2046 with file "VIDEO.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 16/2046 with file "IMAGE.png" -Checking to replace attachment: image/jpeg -Replaced attachment at 17/2046 with file "IMAGE.png" -Checking to replace attachment: image/gif -Replaced attachment at 18/2046 with file "GIF.png" -Checking to replace attachment: image/gif -Replaced attachment at 19/2046 with file "GIF.png" - -[...] - -Checking to replace attachment: image/jpeg -Replaced attachment at 2046/2046 with file "IMAGE.png" - -Exporting backup to 'signal-2022-02-14-00-00-01.backup' -Writing HeaderFrame... -Writing DatabaseVersionFrame... -Writing SqlStatementFrame(s)... - Dealing with table 'part'... 2046/2046 entries...done - Dealing with table 'drafts'... 0/0 entries... - Dealing with table 'push'... 0/0 entries... - Dealing with table 'groups'... 1/1 entries...done - Dealing with table 'group_receipts'... 9/9 entries...done - Dealing with table 'sticker'... 31/31 entries...done - Dealing with table 'recipient'... 7/7 entries...done - Dealing with table 'storage_key'... 0/0 entries... - Dealing with table 'remapped_recipients'... 0/0 entries... - Dealing with table 'remapped_threads'... 0/0 entries... - Dealing with table 'mention'... 3/3 entries...done - Dealing with table 'payments'... 0/0 entries... - Dealing with table 'chat_colors'... 0/0 entries... - Dealing with table 'sender_key_shared'... 0/0 entries... - Dealing with table 'pending_retry_receipts'... 0/0 entries... - Dealing with table 'msl_payload'... 93/93 entries...done - Dealing with table 'msl_recipient'... 94/94 entries...done - Dealing with table 'msl_message'... 93/93 entries...done - Dealing with table 'thread'... 6/6 entries...done - Dealing with table 'mms'... 2097/2097 entries...done - Dealing with table 'sms'... 32832/32832 entries...done - Dealing with table 'avatar_picker'... 0/0 entries... - Dealing with table 'identities'... 0/0 entries... - Dealing with table 'group_call_ring'... 0/0 entries... - Dealing with table 'sender_keys'... 0/0 entries... - Dealing with table 'reaction'... 17/17 entries...done - Dealing with table 'notification_profile'... 0/0 entries... - Dealing with table 'notification_profile_schedule'... 0/0 entries... - Dealing with table 'notification_profile_allowed_members'... 0/0 entries... - Dealing with table 'emoji_search'... 0/0 entries... -Writing SharedPrefFrame(s)... -Writing KeyValueFrame(s)... -Writing Avatars... -Writing EndFrame... -Done! -$ ll -h -total 3,0G --rw-r--r-- 1 bepaald bepaald 148 feb 5 21:23 GIF.png --rw-r--r-- 1 bepaald bepaald 195 feb 5 21:23 IMAGE.png --rw-r--r-- 1 bepaald bepaald 3,0G feb 13 15:46 signal-2022-02-14-00-00-00.backup --rw-r--r-- 1 bepaald bepaald 33M feb 13 15:48 signal-2022-02-14-00-00-01.backup --rw-r--r-- 1 bepaald bepaald 189 feb 5 21:23 VIDEO.png -``` - -Note the mime types do not have to be complete, and the longest type will be matched with highest precedence. In the above case, that means all `image/gif` images are replaced with _"GIF.png"_, while all other images are replaced with _"IMAGE.png"_. - -![replace_example](https://user-images.githubusercontent.com/38437099/154285515-0ca20937-dab8-436d-a333-7a614060ed37.png) - -

-
- -###### Option 2 - -To more easily replace individual attachments with other files, one can first [export the decrypted backup to a directory](#dump), and then for each attachment to replace, place the new file in the directory and name it exactly like the attachment to be replaced, changing the extension to '_.new_'. Then call the program with the `--replaceattachments` option (without arguments). - -
- Example (click to show) -

- -``` -$ # dump decrypted backup to directory -$ mkdir RAW126 -$ ./signalbackup-tools signal-2022-01-28-08-11-49.backup 123456789012345678901234567890 -o RAW126/ -signalbackup-tools (./signalbackup-tools) source version 20220111.170852 (OpenSSL) -IV: (hex:) c3 05 25 [...] -SALT: (hex:) 90 38 9e [...] -BACKUPKEY: (hex:) db ff af [...] -CIPHERKEY: (hex:) 69 b5 7d [...] -MACKEY: (hex:) 7c db e4 ed [...] -COUNTER: 3271894439 -Reading backup file... -FRAME 80968 (100.0%)... Read entire backup file... - -done! - -Exporting backup into 'RAW126//' -Writing HeaderFrame... -Writing DatabaseVersionFrame... -Writing Attachments... -Writing Avatars... -Writing SharedPrefFrame(s)... -Writing KeyValueFrame(s)... -Writing StickerFrames... -Writing EndFrame... -Writing database... -Done! -$ # Now place a new attachment in the directory -$ cp ~/IMAGE.png RAW126/Attachment_4653_1643101250724.new -$ # And re-encrypt, note the message saying 'replaced 1 attachment' when reading the attachments. -$ ./signalbackup-tools RAW126/ --replaceattachments -o OUTPUT.backup -op 012345678901234567890123456789 -signalbackup-tools (./signalbackup-tools) source version 20220111.170852 (OpenSSL) -Opening from dir! -Reading database... -Reading HeaderFrame -Reading DatabaseVersionFrame -Reading SharedPreferenceFrame(s) -Reading KeyValueFrame(s) -Reading EndFrame -Reading AvatarFrames: 20/20 -Reading AttachmentFrames - - Replaced 1 attachments -Reading StickerFrames -Done! - -Exporting backup to 'OUTPUT.backup' -Writing HeaderFrame... -Writing DatabaseVersionFrame... -Writing SqlStatementFrame(s)... - Dealing with table 'part'... 4377/4377 entries...done - Dealing with table 'drafts'... 0/0 entries... - Dealing with table 'push'... 0/0 entries... - Dealing with table 'groups'... 25/25 entries...done - Dealing with table 'group_receipts'... 4033/4033 entries...done - Dealing with table 'sticker'... 31/31 entries...done - Dealing with table 'recipient'... 103/103 entries...done - Dealing with table 'storage_key'... 0/0 entries... - Dealing with table 'remapped_recipients'... 1/1 entries...done - Dealing with table 'remapped_threads'... 0/0 entries... - Dealing with table 'mention'... 10/10 entries...done - Dealing with table 'payments'... 0/0 entries... - Dealing with table 'chat_colors'... 0/0 entries... - Dealing with table 'emoji_search'... 0/0 entries... - Dealing with table 'sender_key_shared'... 0/0 entries... - Dealing with table 'pending_retry_receipts'... 0/0 entries... - Dealing with table 'msl_payload'... 184/184 entries...done - Dealing with table 'msl_recipient'... 190/190 entries...done - Dealing with table 'msl_message'... 184/184 entries...done - Dealing with table 'thread'... 38/38 entries...done - Dealing with table 'mms'... 5876/5876 entries...done - Dealing with table 'sms'... 61273/61273 entries...done - Dealing with table 'avatar_picker'... 0/0 entries... - Dealing with table 'identities'... 35/35 entries...done - Dealing with table 'group_call_ring'... 0/0 entries... - Dealing with table 'sender_keys'... 0/0 entries... - Dealing with table 'reaction'... 52/52 entries...done - Dealing with table 'notification_profile'... 0/0 entries... - Dealing with table 'notification_profile_schedule'... 0/0 entries... - Dealing with table 'notification_profile_allowed_members'... 0/0 entries... -Writing SharedPrefFrame(s)... -Writing KeyValueFrame(s)... -Writing Avatars... -Writing EndFrame... -Done! -``` - -

-
- -> [!TIP] -> A handy python script that uses this option was developed to replace attachments with shrunk versions. It is available [here](https://github.com/cycneuramus/signal-backup-shrink). Thanks @cycneuramus! - -**Operations for Signal Desktop** - -While this tool only deals with backups from Signal Android, and there are no plans to change that, a small number of functions that operate on a Signal Desktop database is available. These options primarily exist to facilitate debugging the [import from Desktop](#desktop) function. - -Running with these options does not require an input file to be provided. These options support some of the same modifying options as `--importfromdesktop`, namely: `--desktopdirs`, and `--ignorewal`. - -- `--dumpdesktopdb [OUTPUTFILE]` Save the Desktop database to `[OUTPUTFILE]` without encryption. -- `--rundtsqlquery [QUERY]` Run a query on the Desktop SQL database. Note that the database only resides in memory and any changes are _not_ saved to disk. -- `--rundtprettysqlquery [QUERY]` As above, but tries to make the output a bit nicer to look at. Depending on the size of the query and the size of the output terminal, may make the output more ledgible (or less so). -- `--exportdesktophtml [OUTPUTDIR]` Export the Signal Desktop database to HTML. This function works internally by creating an empty Android backup, importing the desktop into this and then exporting that internal Android backup to HTML. As a result it supports almost all modifying options mentioned in [import from Desktop](#desktop) and [export to HTML](#export-to-html) (excluding `--limittothreads`, and `--includesettings`). It also has the same limitations as both of these functions combined. -- `--exportdesktoptxt [OUTPUTDIR]` Export the Signal Desktop database to plain text. Works as the above function, except the internal Android backup is [exported to TXT](#export-to-txt) instead. -- `--desktopkey [HEXSTRING]` This is a modifying option for all desktop functions. Manually set the cipher key to use for decrypting the Signal Desktop database (see above note). -- `--showdesktopkey` Shows the key used to decrypt the Signal Desktop database. - -**Various** - -This program supports a small number of other options, most of which are of little to no use for everyday users. A select few that may be useful are mentioned here. A more complete list can be found by running with `--help`. - -- `--runsqlquery [QUERY]` Run any query on the SQL database in the backup file. If combined with `-o/--output` any changes made are saved in the new backup file. See also [advanced options](#advanced). -- `--runprettysqlquery [QUERY]` As above, but tries to make make the output a bit nicer to look at. Depending on the size of the query and the size of the output terminal, may make the output more legible (or less so). -- `-l/--logfile [FILE]` All terminal output is saved to file `[FILE]`. -- `--no-truncate` Any SQL query results that are pretty-printed (see `--runprettysqlquery` above) are normally truncated to fit in the output terminal. This option will prevent this truncating. May be useful when redirecting to file (or using the `--logfile` option). -- `--no-showprogress` Disable (most) progress indicators. Especially useful when trying to parse the programs output in a script. -- `-v/--verbose` Run in verbose mode. This will print a _lot_ of text to output, may be useful in case of errors. -- `--listrecipients` Lists all recipients found in the database. -- `--showdbinfo` Prints a list of all tables and their columns in the backups Sqlite database. -- `--scanmissingattachments` If you see _"warning attachment data not found"_ messages, feel free to use this option and provide the -output to the developer. -- `--migrate214to215` Changes in the database prevent v214 and v215 from being compatible for merging. This function attempts to migrate the older database so it can be used as a source for `--importthreads`. See also https://github.com/bepaald/signalbackup-tools/issues/184. - -**Advanced options** - -The program can run any sql queries on the database in the backup file and save the output. If you know the schema of the database and know what you're doing, feel free to run any query and save the output. Examples: - -``` -# delete all sms and mms messages from one thread: -signalbackup-tools [input] [passphrase] --runsqlquery "DELETE * FROM sms WHERE thread_id = 1" --runsqlquery "DELETE * FROM mms WHERE thread_id = 1" --output [output] (--opassphrase [newpassphrase]) -``` - -``` -# list all messages in the sms database where the message body was 'Yes' -$ ./signalbackup-tools [input] [passphrase] --runprettysqlquery "SELECT _id,body,DATETIME(ROUND(date / 1000), 'unixepoch') AS isodate,date FROM sms WHERE body == 'yes' COLLATE NOCASE" -signalbackup-tools source version 20191219.175337 -IV: (hex:) 12 16 72 95 7a 00 68 44 7e cf 7d 20 26 f9 d3 7d (size: 16) -SALT: (hex:) cc 03 85 02 61 97 eb 5b ed 3e 05 00 c4 a8 77 40 28 08 aa 9f e5 a8 00 74 b4 f8 56 aa 24 57 a9 5d (size: 32) -BACKUPKEY: (hex:) 8f ff df 2b 9f 96 73 9a 63 95 0f ea 3f b1 e5 a4 87 12 19 ca 93 31 86 2a 60 3f 41 ef 6d a4 08 44 (size: 32) -CIPHERKEY: (hex:) ce 53 c1 f2 92 4b e3 b8 e1 56 85 61 14 96 82 8b 83 7f 07 21 83 52 1a c2 3f 6b 16 83 3e 33 94 a3 (size: 32) -MACKEY: (hex:) c2 77 af 1e 4b 05 db 62 52 57 af 8a d6 a4 d4 e9 6c 93 53 81 9a e7 6f 12 2c ce 13 8f b3 5e 8d 3a (size: 32) -COUNTER: 2907636 -Reading backup file... -FRAME 4852 (100.0%)... Read entire backup file... - -done! - * Executing query: SELECT _id,body,DATETIME(ROUND(date / 1000), 'unixepoch') AS isodate,date FROM sms WHERE body == 'yes' COLLATE NOCASE ------------------------------------------------------- -| _id | body | isodate | date | ------------------------------------------------------- -| 3235 | Yes | 2017-10-21 17:10:15 | 1508605815286 | -| 9345 | Yes | 2017-12-18 22:18:36 | 1513635516440 | -| 17125 | Yes | 2018-02-02 15:46:16 | 1517586376228 | -| 21300 | Yes | 2018-05-10 21:14:49 | 1525986889325 | -| 26317 | Yes | 2018-10-25 15:16:58 | 1540480618238 | -| 32433 | Yes | 2019-05-10 14:22:25 | 1557498145794 | ------------------------------------------------------- - -# now change a specific message: -[~/programming/signalbackup-tools] $ ./signalbackup-tools [input] [passphrase] --runsqlquery "UPDATE sms SET body = 'No' WHERE _id == 21300" --ouput [output] -``` - -If you also need to edit the attachments, dump the backup to directory first ([as described above](#dump)) and do whatever you want, but realize when editing the .bin file, it will usually require changes to also be made to the .sbf file and the sql database to end up with a valid database. - - -## Future plans - -- ~~merging existing backups (successful tests have been done)~~ _DONE_ -- exporting to other formats (~~csv~~, xml, ~~html~~) _WIP_ -- ~~cropping backup to certain conversations and time spans (successfully done in testing)~~ _DONE_ -- ~~replacing/deleting attachments without changing/deleting messages. For example, replacing with thumbnails or tiny placeholders to save space.~~ _DONE (pretty much hopefully)_ -- ~~importing databases from the desktop app. I have no experience with that yet.~~ _DONE (I think, mostly)_ - -Development will be slow at times. - -## Donate - -If this tool was helpful to you or you appreciate my work and you can spare it, you might consider donating: - -Paypal: [![Paypal](https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif)](https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=U523FZFW3BQBQ¤cy_code=USD&source=url) - -Ko-fi: [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/bepaald) - -BTC: 17RqHi9XBeUAEShbp2RnbmkCSAU2R94tH4 - -Donations will help development in that they will put food in my mouth, and I need food to write code :smile: - -You might also consider helping out the Signal Foundation here: https://support.signal.org/hc/en-us/articles/360007319831-How-can-I-contribute-to-Signal- +NO PRs PLEASE