Update documentation with latest fuzzers and arguments

Endava · Mar 13, 2024 · 4cf60e2 · 4cf60e2
1 parent 3d3dd38
commit 4cf60e2
Show file tree

Hide file tree

Showing 6 changed files with 54 additions and 4 deletions.
diff --git a/docs/commands-and-arguments/arguments.md b/docs/commands-and-arguments/arguments.md
@@ -25,7 +25,7 @@ You can get the full list of arguments by running `cats -h`. Below is a short de
 - `--urlParams param1:value1 param2:value2` A comma separated list of 'name:value' pairs of parameters to be replaced inside the URLs. This is useful when you have static parameters in URLs (like 'version' for example).
 - `--functionalFuzzerFile=FILE ` a file used by the `FunctionalFuzzer` that will be used to create user-supplied payloads.
 - `--skipFuzzers=LIST_OF_FIZZERs` a comma separated list of fuzzers that will be skipped for **all** paths. You can either provide full `Fuzzer` names (for example: `--skippedFuzzers=VeryLargeStringsFuzzer`) or partial `Fuzzer` names (for example: `--skipFuzzers=VeryLarge`). `CATS` will check if the `Fuzzer` names contains the string you provide in the arguments value.
-- `--skipFields=field1,field2#subField1` a comma separated list of fields that will be skipped by replacement Fuzzers like EmptyStringsInFields, NullValuesInFields, etc.
+- `--skipFields=field1,field2#subField1` a comma separated list of fields that will be skipped by replacement Fuzzers like EmptyStringsInFields, NullValuesInFields, etc. **When a field starts with `!` any field that starts with that name will be entirely skipped for fuzzing**
 - `--httpMethods=PUT,POST,etc` a comma separated list of HTTP methods that will be used to filter which http methods will be executed for each path within the contract
 - `--securityFuzzerFile=FILE` A file used by the `SecurityFuzzer` that will be used to inject special strings in order to exploit possible vulnerabilities
 - `--printExecutionStatistics` If supplied (no value needed), prints a summary of execution times for each endpoint and HTTP method. By default this will print a summary for each endpoint: max, min and average. If you want detailed reports you must supply `--printExecutionStatistics=detailed`
@@ -85,6 +85,19 @@ You can get the full list of arguments by running `cats -h`. Below is a short de
 - `--tags`  A comma separated list of tags to include. If no tag is supplied, all tags will be considered. To list all available tags run: `cats stats -c api.yml`
 - `--skipTags` A comma separated list of tags to ignore. If no tag is supplied, no tag will be ignored. To list all available tags run: `cats stats -c  api.yml`
 
+:::tip
+When you want to skip fuzzing entirely for a specific JSON object or specific fields you must prefix the field name from the `--skipFields` argument with `!`.
+The following `--skipFields="!address"` will skip fuzzing for all sub-fields of the `address` object. If you also want CATS to not sent the `address` 
+object at all to the service (sometimes some object might not make sense in conjunction with other objects) you must also use the `cats_remove_field`
+within the reference data file.
+
+```yaml
+all:
+  address: cats_remove_field
+```
+
+:::
+
 Next arguments are active only when supplying a custom dictionary via `--words`:
 - `--matchResponseCodes=<matchResponseCodes>[,<matchResponseCodes>...]` A comma separated list of HTTP response codes that will be matched as error. All other response codes will be ignored from the final report. If provided, all Contract Fuzzers will be skipped
 - `--matchResponseLines=<matchResponseLines>[,<matchResponseLines>...]` A comma separated list of number of line counts in the response that will be matched as error. All other response line counts will be ignored from the final report. If provided, all Contract Fuzzers will be skipped

diff --git a/docs/fuzzers/http-fuzzers/custom-http.md b/docs/fuzzers/http-fuzzers/custom-http.md
@@ -0,0 +1,17 @@
+--- 
+hide_table_of_contents: true
+---
+
+# Custom HTTP methods
+
+| Item                                            | Description                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+|:------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| **Full Fuzzer Name**                            | CustomHttpMethodsFuzzer                                                                                                                                                                                                                                                                                                                                                                                                                     |
+| **Log Key**                                     | **CHM**                                                                                                                                                                                                                                                                                                                                                                                                                                     |
+| **Description**                                 | This fuzzer will send happy path requests for custom HTTP methods not specific to REST APIs such as: `diff`, `verify`, `publish`, etc. The expectation is that APIs will reject the request.                                                                                                                                                                                                                                                |
+| **Enabled by default?**                         | Yes                                                                                                                                                                                                                                                                                                                                                                                                                                         |
+| **Expected result**                             | `405`                                                                                                                                                                                                                                                                                                                                                                                                                                       |
+| **Fuzzing logic**                               | Iteratively **sends** a happy path request for each path and the following HTTP methods: `DIFF, VERIFY, PUBLISH, UNPUBLISH, BATCH, VIEW, PURGE, DEBUG, SUBSCRIBE, UNSUBSCRIBE, MERGE, INDEX`                                                                                                                                                                                                                                                |
+| **Conditions when this fuzzer will be skipped** | None                                                                                                                                                                                                                                                                                                                                                                                                                                        |
+| **HTTP methods that will be skipped**           | All REST HTTP methods                                                                                                                                                                                                                                                                                                                                                                                                                       |
+| **Reporting**                                   | Reports `error` if: *1.* response code is `404`; *2.* response code is documented, but not expected; *3.* any unexpected exception. <br/><br/> Reports `warn` if: *1.* response code is expected and documented, but not matches response schema; *2.* response code is expected, but not documented; *3.* response code is `501`. <br/><br/> Reports `success` if: *1.* response code is expected, documented and matches response schema. | 
diff --git a/docs/fuzzers/http-fuzzers/index.md b/docs/fuzzers/http-fuzzers/index.md
@@ -1,7 +1,8 @@
 # HTTP Fuzzers
-CATS has currently 24 registered `HTTP` Fuzzers:
+CATS has currently 25 registered `HTTP` Fuzzers:
 - `BypassAuthenticationFuzzer` - check if an authentication header is supplied; if yes try to make requests without it
 - `CheckDeletedResourcesNotAvailableFuzzer` - checks that resources are not available through `GET` after a successful `DELETE`
+- `CustomHttpMethods` - iterate through a list of hypothetical HTTP methods that are not expected to be implemented by REST APIs
 - `DummyRequestFuzzer` - send a dummy json request {'cats': 'cats'}
 - `EmptyBodyFuzzer` - send a request with a empty string body
 - `EmptyJsonArrayBody` - send a request with a empty json array body

diff --git a/docs/getting-started/cats_response_codes.md b/docs/getting-started/cats_response_codes.md
@@ -0,0 +1,19 @@
+---
+sidebar_position: 18
+description: Mutators
+---
+
+# CATS Response Codes
+
+In order to preserve consistency, CATS has some custom HTTP response codes it uses mainly to signal communication issues.
+They are all in the `9XX` range. The current defined ones are as follows:
+
+- `952` - the server might close the connection earlier than expected; part of the response might be sent, but not all
+- `953` - connection is refused; typically might indicate that the service is not available or there is a typo in the server name
+- `954` - read timeout; typically when reading the response takes longer that the current set timeout; you might want to increase the `--readTimeout` argument
+- `955` - write timeout; typically when writing the request takes longer that the current set timeout; you might want to increase the `--writeTimeout` argument
+- `956` - connection timeout; typically when establishing a connection takes longer that the current set timeout; you might want to increase the `--connectionTimeout` argument
+- `957` - protocol communication exception; typically when the service might write some malformed data into the response; might indicate an issue with the service
+- `999` - when something unexpected happened which is not part of the above errors
+
+If the entire communication is successful i.e. request was fully sent and response was fully consumed the HTTP response code will be used.
diff --git a/docs/getting-started/mutators.md b/docs/getting-started/mutators.md
@@ -1,5 +1,5 @@
 ---
-sidebar_position: 17
+sidebar_position: 18
 description: Mutators
 ---
 

diff --git a/docs/intro.md b/docs/intro.md
@@ -21,7 +21,7 @@ CATS is:
 - **📖 Simple to Learn**: flat learning curve, with intuitive configuration and syntax
 - **⚡️ Fast**: automatic process for writing, running and reporting
 
-By using a simple and minimal syntax, with a flat learning curve, CATS (**C**ontract **A**API **T**esting **S**ecurity) enables you to generate thousands of API tests within minutes with **no coding effort**.
+By using a simple and minimal syntax, with a flat learning curve, CATS (**C**ontract **A**PI **T**esting and **S**ecurity) enables you to generate thousands of API tests within minutes with **no coding effort**.
 All tests are **generated, run and reported automatically** based on a pre-defined set of **100+ Fuzzers**.
 The Fuzzers cover a wide range of boundary testing and negative scenarios from random large Unicode values to well crafted, context dependant values based on the request data types and constraints.
 Even more, you can **leverage** the fact that **CATS** generates request payloads dynamically and **write simple end-to-end functional tests**.