Installation problems on CentOS 6.7 #364
Comments
|
librelist seems to have problems now and then (I recently moved another mailing list away from there due to that). So, try again later, contact support there, or just create a ticket about it here. |
|
Thank you for clarifying this, Thomas. So I will try here. I have installed Attic on CentOS 6.7. CentOS requires Python 2.6 for its packet manager "yum". So I installed Python 3.4.3 with "make altinstall". Further I installed it as a shared library and put it in /usr/local/Python3/bin. Python 3.4.3 is on the path as python3 and python3.4. I took some of this advice from here: Further, Selinux is off, libacl-devel is installed, OpenSSL is > 1.0, python-msgpack is installed via "pip3.4 install msgpack-python", and finally Attic via "pip3.4 install Attic". Initially I had forgotten about libacl-devel and selinux. I might have done those after the installation of Attic. Then Attic hung on the initialization of my first backup repo. I then ran the test suite by "python3 -m attic.testsuite.run", which gave me the below output. Did I do something wrong? Is the coexistance of python (Python 2.6) and python3 (Python 3.4) on the path a problem for attic? Some of the errors below are supposedly due to permission problems, but this makes no sense to me, as I run the test suite as root (in the temp dir, obviously). In particular:
The dir "/tmp/tmp3ml_3ov1" doesn't exist, but other similar ones are there. Thanks for any ideas! [root@backuper ole]# python3 -m attic.testsuite.run ..............ERROR: test_aes_counter_uniqueness_keyfile (attic.testsuite.archiver.ArchiverTestCase)Traceback (most recent call last): ERROR: test_aes_counter_uniqueness_passphrase (attic.testsuite.archiver.ArchiverTestCase)Traceback (most recent call last): ERROR: test_basic_functionality (attic.testsuite.archiver.ArchiverTestCase)Traceback (most recent call last): ERROR: test_repository_swap_detection (attic.testsuite.archiver.ArchiverTestCase)Traceback (most recent call last): ERROR: test_repository_swap_detection2 (attic.testsuite.archiver.ArchiverTestCase)Traceback (most recent call last): ERROR: test_aes_counter_uniqueness_keyfile (attic.testsuite.archiver.RemoteArchiverTestCase)Traceback (most recent call last): ERROR: test_aes_counter_uniqueness_passphrase (attic.testsuite.archiver.RemoteArchiverTestCase)Traceback (most recent call last): ERROR: test_basic_functionality (attic.testsuite.archiver.RemoteArchiverTestCase)Traceback (most recent call last): ERROR: test_repository_swap_detection (attic.testsuite.archiver.RemoteArchiverTestCase)Traceback (most recent call last): ERROR: test_repository_swap_detection2 (attic.testsuite.archiver.RemoteArchiverTestCase)Traceback (most recent call last): Ran 118 tests in 27.728s FAILED (errors=10, skipped=6) |
otraupe
changed the title
|
Found this: Backups seem to work, too. A lot of 5 M size files are created. |
|
The tests should be run with It is no problem to have both python 2.x and 3.x installed, you just need to use python 3.x for attic. |
|
With fakeroot, it takes much longer, but the result seems to be the same. Very similar to what Whissi posted two weeks ago. Makes me worry, honestly.
This should be ensured by "pip3.4 install attic", right? The "attic" executable resides in "/usr/local/Python3/bin/". |
|
All detailed error reports seem to be related to the "xattr.py". Another dependency missing? |
|
I assume you have libattr and -dev? |
|
[root@backuper ole]# yum list installed | grep attr |
|
I would like to ask the following questions: Which tests are failing exactly, what do they do? My first backup attempt over night finally "failed", although it seems that a lot of data was backuped: |
|
And why is it so slow? Ok, that might have many reasons, but 16 GB took about 45 minutes. I intend to backup multiple TB of data. That's probably not working in the long run. |
|
Tests: it seems like they can not set extended attributes (xattrs) on the files in /tmp. Maybe this is because your /tmp filesystem does not support xattrs or is mounted without enabling that support. "attic: Exiting with failure status due to previous errors" is a warning, you have to read the log output above that to see what exactly did not work. Likely it could not access some file(s) due to permissions or so. That is often not a fatal error, your backup has reached its normal end. |
|
Aside from all hardware, setup and data related factors, some of the slowness may be caused by attic's usage of zlib compression at level 6, by encryption (if you do not have hw accelerated AES) and by internal scheduling. The first two can slow down if your CPU is relatively slow and/or old. If you intend to go multiple TBs, you likely will run into the issue that the target chunksize of attic is ~64KB, so multiple TBs create a lot of chunks, a huge chunk index and huge RAM consumption. I made the chunker configurable in my fork "borgbackup". |
|
Thanks for your help, Thomas! @xattrs: that's what I thought first. But the respective dirs in /tmp don't even exist - probably DUE to the failed tests. Also, I tested for xattr support following this procedure: @the log file: where do I find it? The output in the console is cut to several hundreds (?) of lines. No errors there as far as I can see. |
|
Well, it seems rather clear after analyzing the traceback that there are xattr problems, because the setxattr call in the test setup fails all the time. So, did you check xattr support inside /tmp/? About the log: redirect stdout and stderr to files so you are not limited by the scrollback buffer. The warning/error msg will be in there then. |
|
Yes, just checked it again. @logs: I will do that. |
|
You should test whether setting xattrs on existing files works. Of course the tests clean up their test files after running, that's why you do not see them existing afterwards. |
|
This is the output of the test for working xattrs according to the Samba wiki page I quoted above. Never had any problems with that. [root@backuper tmp]# touch test.txt file: test.txtuser.test="test" [root@backuper tmp]# getfattr -n security.test -d test.txt file: test.txtsecurity.test="test2" |
|
(Can I somehow disable this annoying auto-format stuff, here?) |
|
And: only the faild tests clean-up their files and folders? Because there are a lot of tmp... folders in /tmp: drwx------. 3 root root 4096 Nov 25 16:59 tmp072yfnc5 |
|
Thank you also for your explanations and tips regarding speed. Attic currently runs on a VM (qemu-kvm) with 2 cores of a Xeon E5-2620 v2 @ 2.10GHz, 2 GB RAM, and a separate physical network inferface. HW accelerated AES is out of the question, I guess, but encryption is not selected, if I remember the Attic init correctly. What do you mean by the internal scheduler? However, currently I backup from a Samba share on the host machine, and to a Samba share on a NAS. So this might contribute to the slowness. On the other hand, doing the same on a Windows machine (having backup source and target on Samba shares from different machines; using some arbitrary copy software like the Total Commancer), I achieve a sufficient throughput of ~60 MB/s (on a GB network). So the network architecture shouldn't be much of an issue. Anyways, I plan to migrate the Attic installation to the host system (file server hosting the zfs pool being backuped) once everything is ok (initlaly was concerned about the alternative python3 installation). |
|
Internal scheduling: attic is single threaded. reading, processing, writing, syncing is done in-order. Thus, I/O and I/O wait times are not used otherwise, no parallel processing. Maybe this is not the main cause of slowness for some people (see other factors), but if everything else is fast, this is a limiting factor. |
|
Ok, will see more when attic runs directly on the file server. Curently, I am down to <6 MB/s, which I can't explain by any of these factors. Any more ideas regarding the failed test setup? |
|
You should really keep separate topics in separate issues, but about the speed: are you mounting a remote file system locally and then backing up to or from it? If so, that's almost certainly your problem. To be efficient, you need two copies of attic running, one as close as possible to the source files and one as close as possible to the backup location. |
|
You are totally right, I am sorry. I wasn't paying attention to that as not posting to the mailing list made me feel already out of place. ---------- SPEED DISCUSSION CLOSED ---------- So, how do I find out whether the failing tests pose any thread to my intended use of attic? And I don't actually mean some trial-and-error approach. |
|
Followed the advice here to comment out the culprit line about setting xattrs: Looks better now. Here is my new test output ("Ok", 8 tests skipped, due to encription not enabled? but with actual root rights, it is only 6 skipped): [ole@backuper ~]$ fakeroot python3 -m attic.testsuite.run ..............Ran 118 tests in 157.289s OK (skipped=8) I take it the 8x 's' in the '.......' output are the skipped ones? What's the rest about? |
|
Thomas, as you are responding here and not the original developer of attic (you are talking about your fork), would you say that attic isn't actively developed at the moment? |
|
@otraupe: Read #217. Compare commit log between https://github.com/jborg/attic/commits/master and https://github.com/borgbackup/borg/commits/master Now decide on your own which project you would call more active. |
|
I see, thank you! Am 30.11.2015 um 14:22 schrieb Thomas D.:
|
|
If you use anaconda I've packaged attic 0.16 at https://anaconda.org/unode/attic. |
Hi,
I have problems with the installation and would like to post to the list "attic@librelist.com". I have sent multiple messages to it, but I don't get a response.
Ole
The text was updated successfully, but these errors were encountered: