How to treat failed Python Safety checks

[jemrobinson]

We currently run Python Safety on each of our conda environments but do not do anything with the output. Here is the output from a recent build, which shows some problematic package versions in each environment.

What should we do with these? The most obvious options are:

• ignore them
• cause any safety check failure to cause the build to fail (maybe better if/when we can reduce the size of our package list)
• flag them up in the analyse_build script but take no further action

py27 conda environment

safety report
checked 233 packages, using default DB
---
-> tensorflow, installed 1.13.1, affected <1.15.0, id 37524
-> tensorflow, installed 1.13.1, affected >=1.0,<1.15.2, id 38038
-> tensorflow, installed 1.13.1, affected >=1.0,<1.15.2, id 37776
-> tensorflow, installed 1.13.1, affected >=1.0,<1.15.2, id 38039
-> sphinx, installed 1.8.5, affected <3.0.4, id 38330
-> pyyaml, installed 5.2, affected <5.3.1, id 38100
-> pylint, installed 1.9.2, affected <2.5.0, id 38224
-> pillow, installed 6.2.1, affected <6.2.2, id 37782
-> pillow, installed 6.2.1, affected <6.2.2, id 37781
-> pillow, installed 6.2.1, affected <6.2.2, id 37780
-> pillow, installed 6.2.1, affected <6.2.2, id 37779
-> pillow, installed 6.2.1, affected >6.0,<6.2.2, id 37772
-> msgpack, installed 0.5.6, affected <0.6.0, id 36700
-> gdal, installed 3.0.2, affected <3.1.0, id 38264

py36 conda environment

+==============================================================================+
| REPORT                                                                       |
| checked 267 packages, using default DB                                       |
+============================+===========+==========================+==========+
| package                    | installed | affected                 | ID       |
+============================+===========+==========================+==========+
| tensorflow                 | 1.13.1    | <1.15.0                  | 37524    |
| tensorflow                 | 1.13.1    | >=1.0,<1.15.2            | 38038    |
| tensorflow                 | 1.13.1    | >=1.0,<1.15.2            | 37776    |
| tensorflow                 | 1.13.1    | >=1.0,<1.15.2            | 38039    |
| pylint                     | 2.4.4     | <2.5.0                   | 38224    |
| msgpack                    | 0.5.6     | <0.6.0                   | 36700    |
| gdal                       | 3.0.2     | <3.1.0                   | 38264    |
+==============================================================================+

py37 conda environment

+==============================================================================+
| REPORT                                                                       |
| checked 266 packages, using default DB                                       |
+============================+===========+==========================+==========+
| package                    | installed | affected                 | ID       |
+============================+===========+==========================+==========+
| tensorflow                 | 1.13.1    | <1.15.0                  | 37524    |
| tensorflow                 | 1.13.1    | >=1.0,<1.15.2            | 38038    |
| tensorflow                 | 1.13.1    | >=1.0,<1.15.2            | 37776    |
| tensorflow                 | 1.13.1    | >=1.0,<1.15.2            | 38039    |
| pylint                     | 2.4.4     | <2.5.0                   | 38224    |
| msgpack                    | 0.5.6     | <0.6.0                   | 36700    |
| gdal                       | 3.0.2     | <3.1.0                   | 38264    |
+==============================================================================+

[JimMadge]

Can we get an impression of the severity of any issues without assessing them ourselves?

It would be nice to know how often we expect to find issues when we have our 'final' list of packages. Because I feel that will change whether failing builds would seem like an occasional inconvenience or a major block.

Maybe a good middle ground would be to produce a more detailed report (with descriptions of the vulnerabilities) and ensure that they are read before committing a new image. I'm not sure how you could enforce that. You could imagine making this report a CI step and having a bot post the report to pull requests. But that wouldn't catch any new problems found after that point.

[jemrobinson]

For newer Python version there is no reason why the safety checks should fail. When we drop Python 2.7 (see #459) we should also ensure that failed safety checks will cause the build to fail.

[JimMadge]

@jemrobinson Is this still relevant, do the conda environments still exist?