4.0 | Revise exit codes for PHPCS and PHPCBF commands

[jrfnl]

Repost from squizlabs/PHP_CodeSniffer#2898 by @gsherwood:

Version 4 is a good time to review the exit codes that both scripts produce to ensure that success cases all use 0 and that all failure cases have codes that make sense. Documentation in the wiki to describe the new exist codes should be an output of this work.

---

Additional info posted by @jrfnl:

Loosely related open issues:

• Disable report for phpcbf squizlabs/PHP_CodeSniffer#1818
• The "phpcs" doesn't use error exit code when invalid standard is given squizlabs/PHP_CodeSniffer#2412 (discoverability of availability of exit code related CLI arguments)

And some related closed issues:

• Return code of phpcbf on all errors fixed squizlabs/PHP_CodeSniffer#2954
• phpcbf returns exit-code 1 even if all errors were fixed squizlabs/PHP_CodeSniffer#3057
• When I run phpcbf with stdin, the exit code is "1" instead of 0 squizlabs/PHP_CodeSniffer#3367
• allow stdin fix to exit sucessfully squizlabs/PHP_CodeSniffer#3541

And the following open PR is also related:

• Add constants for exit return codes squizlabs/PHP_CodeSniffer#3696

---

Related remark posted by @nelson6e65:

Param --runtime-set ignore_warnings_on_exit 1 works on phpcs, but it does not for phpcbf.

---

Question by @dfelton:

Is there any documentation somewhere that lists all non 0 or 1 exit codes that exist, and the meaning of them?

Today was the first time I discovered an exit code of 3 is used for when and invalid standard is given. It'd be nice to be able to read a compiled list of what exit codes there are, and I cannot seem to find any information on the wiki on this topic.

Reply by @jrfnl:

I think this is what you are looking for: squizlabs/PHP_CodeSniffer#930 (comment)

[greg0ire]

0: no errors found
1: errors found
2: fixable errors found
3: processing error

It does not look like all situations are mutually exclusive. For instance, it's possible to have fixable and unfixable errors found during the same run. This should IMO be encoded as 1+2=3. If a processing error happens, 4 should be added, and each new situation should be associated with a power of 2, so that it's possible to tell more about what happened from the exit code alone.

[jrfnl]

@greg0ire I'm sure Greg had some idea in mind on where he wanted to go with this when he created the issue, but the above is all the info which is available.

[mbomb007]

Personally, I would like errors and warnings to have separate exit codes, so that a coding standards error returns 1, whereas warnings with no errors returns 2 or something similar. Otherwise you have to grep the output to determine what type of error it is.

This would be helpful for using PHPCS in GitLab pipelines, because exit code can be used to determine whether a pipeline is allowed to fail. GitLab allows pipelines to specify essentially a "pass with warning" state, and the pipeline stage shows as yellow (warning) instead of green (success) or red (error). I'd like an easy way to determine that from the exit code, such as 0 (success), 1 (error), 2 (warnings, no errors).

[paulshryock]

I'd love for PHPCBF to return exit code 0 when fixable issues are found and fixed and there are no unfixable issues found.

composer coding-standards:fix
> phpcbf
F... 4 / 4 (100%)



PHPCBF RESULT SUMMARY
-------------------------------------------------------------------------------
FILE                                                           FIXED  REMAINING
-------------------------------------------------------------------------------
/Users/User/Desktop/my-org/my-repo/src/php/MyClass.php         1      0
-------------------------------------------------------------------------------
A TOTAL OF 1 ERROR WERE FIXED IN 1 FILE
-------------------------------------------------------------------------------

Time: 128ms; Memory: 6MB


Script phpcbf handling the coding-standards:fix event returned with error code 1

The above example shows 1 error found and fixed. So the outcome is a success; yet it exits 1 as though it failed.