Smallfiles test

[spuiuk]

Adding a smallfiles test which uses podman.

This will not work at this moment without installing the podman packages and disabling selinux. So setting it to draft for now.

[synarete]

Why merge 'main' into 'smallfiles' (and not rebase)?

[spuiuk]

Why merge 'main' into 'smallfiles' (and not rebase)?

I think that is what you get when you rebase from the github interface.

[spuiuk]

Pushed the latest set of patches. the following changes were done.

1. Use pathlib.Path - Some of the testhelper functions called return a string or expect a string. For such cases, you will see conversions being made to Path() and to str(). I will post patches after this where we move the helper functions to using the pathlib.Path where these awkward conversions should no longer be needed.

2. Change the way test_info and container_tests global variables are set. The earlier way to set the global variables were a result of having to accommodate both pytest and running it as standalone. This is no longer needed after we moved completely to pytest.

3. Changed the smallfiles container to use stream9. Uploaded this to the quay.io repo.

[spuiuk]

/retest all

[spuiuk]

/retest all

[spuiuk]

Output of the smallfiles test from the centos-ci/xfs tests

       "==================================== PASSES ====================================",
        "_________ test_containers[192.168.123.10-share-xfs-default-smallfiles] _________",
        "----------------------------- Captured stdout call -----------------------------",
        "Trying to pull quay.io/samba.org/sit-test-cases:smallfiles...",
        "Getting image source signatures",
        "Copying blob sha256:b7954271d65c4abe8e117cb3f1497cf02139d069933b1223e5b5ad01e094458a",
        "Copying blob sha256:7a80604fbc801b1ca549efebfce436cd52028affa9e073b47f6260dfa853bf57",
        "Copying blob sha256:35f8654d391b38bcded9291f440d06fc3b0c5dccc0d8361c36e2c0bb32b00fc5",
        "Copying blob sha256:1926f8a6af8b3e39edf1650fc135e57b5745658f6b4977b1cf880f732bd29bcc",
        "Copying blob sha256:5218344d3a8991890ee4ad90c322698913a4d059db6bad4fc8cb1bb0c9c3b6c5",
        "Copying blob sha256:04e393290efc212a09612383affe45dadb2d3383f2229a68bf24e563a169bbe0",
        "Copying blob sha256:6476010ca3f151058a7c396ae04bf498d7b9aedda02a23cf20a8fb32a5dacb3a",
        "Copying config sha256:e7b01d2ce37f68d6da26d83dc636b47308c9db1fc9c486c19ebfb1d31409ef32",
        "Writing manifest to image destination",
        "changing --finish to true for op type create",
        "                                 version : 3.2",
        "                           hosts in test : None",
        "                        launch by daemon : False",
        "                   top test directory(s) : ['/testdir']",
        "                               operation : create",
        "                            files/thread : 1024",
        "                                 threads : 4",
        "           record size (KB, 0 = maximum) : 0",
        "                          file size (KB) : 1024",
        "                  file size distribution : fixed",
        "                           files per dir : 100",
        "                            dirs per dir : 10",
        "              threads share directories? : N",
        "                         filename prefix : ",
        "                         filename suffix : ",
        "             hash file number into dir.? : N",
        "                     fsync after modify? : Y",
        "                         incompressible? : N",
        "          pause between files (microsec) : 0",
        "                             auto-pause? : N",
        " delay after cleanup per file (microsec) : 0",
        "             minimum directories per sec : 2",
        "                             total hosts : 1",
        "                    finish all requests? : Y",
        "                              stonewall? : Y",
        "                 measure response times? : N",
        "                            verify read? : Y",
        "                                verbose? : N",
        "                          log to stderr? : N",
        "host = 5db1b9173a5b,thr = 00,elapsed = 44.94384574890137,files = 1024,records = 1024,status = ok",
        "host = 5db1b9173a5b,thr = 01,elapsed = 44.910395860672,files = 1024,records = 1024,status = ok",
        "host = 5db1b9173a5b,thr = 02,elapsed = 44.877068281173706,files = 1024,records = 1024,status = ok",
        "host = 5db1b9173a5b,thr = 03,elapsed = 44.87704110145569,files = 1024,records = 1024,status = ok",
        "total threads = 4",
        "total files = 4096",
        "total IOPS = 91",
        "total data =     4.000 GiB",
        "100.00% of requested files processed, warning threshold is  70.00%",
        "elapsed time =    44.944",
        "files/sec = 91.220737",
        "IOPS = 91.220737",
        "MiB/sec = 91.220737",
        "                                 version : 3.2",
        "                           hosts in test : None",
        "                        launch by daemon : False",
        "                   top test directory(s) : ['/testdir']",
        "                               operation : delete",
        "                            files/thread : 1024",
        "                                 threads : 4",
        "           record size (KB, 0 = maximum) : 0",
        "                          file size (KB) : 1024",
        "                  file size distribution : fixed",
        "                           files per dir : 100",
        "                            dirs per dir : 10",
        "              threads share directories? : N",
        "                         filename prefix : ",
        "                         filename suffix : ",
        "             hash file number into dir.? : N",
        "                     fsync after modify? : Y",
        "                         incompressible? : N",
        "          pause between files (microsec) : 0",
        "                             auto-pause? : N",
        " delay after cleanup per file (microsec) : 0",
        "             minimum directories per sec : 2",
        "                             total hosts : 1",
        "                    finish all requests? : N",
        "                              stonewall? : Y",
        "                 measure response times? : N",
        "                            verify read? : Y",
        "                                verbose? : N",
        "                          log to stderr? : N",
        "host = 5db1b9173a5b,thr = 00,elapsed = 3.500145435333252,files = 1024,records = 0,status = ok",
        "host = 5db1b9173a5b,thr = 01,elapsed = 3.5007212162017822,files = 1024,records = 0,status = ok",
        "host = 5db1b9173a5b,thr = 02,elapsed = 3.4994964599609375,files = 1024,records = 0,status = ok",
        "host = 5db1b9173a5b,thr = 03,elapsed = 3.5004777908325195,files = 1024,records = 0,status = ok",
        "total threads = 4",
        "total files = 4096",
        "100.00% of requested files processed, warning threshold is  70.00%",
        "elapsed time =     3.501",
        "files/sec = 1170.215446",

[spuiuk]

merging based on output from the centos-ci/xfs test only.
(samba-in-kubernetes/sit-environment#53)