diff --git a/CHANGELOG.md b/CHANGELOG.md index 99cd324..13e5642 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ ## Release history +### 0.2.2 - alpha3 (2021-08-23) + +* Split plugins into common, basic, modifiers and parsers. +* Add Combine modifier. +* Add UrlParser plugin. +* Update documentation with new plugin structure. + ### 0.2.1 - alpha2 (2021-08-03) * Improved the fuzzing module. diff --git a/docs/dev/plugins.rst b/docs/dev/plugins.rst index 7f2ef9a..135b723 100644 --- a/docs/dev/plugins.rst +++ b/docs/dev/plugins.rst @@ -1,5 +1,5 @@ .. _plugins: -.. module:: raider.plugins +.. module:: raider.plugins.common Plugins ======= @@ -11,10 +11,34 @@ between :ref:`Flows `. Below there's a list of predefined Plugins. The users are also encouraged to write their own plugins. +Common +------ + +Plugin +++++++ + +.. autoclass:: Plugin + +Parser +++++++ + +.. autoclass:: Parser + +Empty ++++++ + +.. autoclass:: Empty + + +.. module:: raider.plugins.basic + +Basic +----- + .. _plugin_variable: Variable --------- +++++++++ Use this when the value of the plugin should be extracted from the user data. At the moment only ``username`` and ``password`` are @@ -33,7 +57,7 @@ Example: .. _plugin_prompt: Prompt ------- +++++++ Prompt plugin should be used when the information is not known in advance, for example when receiving the SMS code. @@ -49,27 +73,10 @@ Example: .. _plugin_command: -Command -------- - -Use Command plugin if you want to extract information using a shell -command. - -Example: - -.. code-block:: hylang - - (setv mfa_code (Command - :name "otp" - :command "pass otp personal/app1")) - -.. autoclass:: Command - :members: - .. _plugin_cookie: Cookie ------- +++++++ Use Cookie plugin to extract and set new cookies: @@ -86,7 +93,7 @@ Example: .. _plugin_header: Header ------- +++++++ Use Header plugin to extract and set new headers. It also allows easier setup for basic and bearer authentication using the provided @@ -112,10 +119,29 @@ Example: .. autoclass:: Header :members: + +Command ++++++++ + +Use Command plugin if you want to extract information using a shell +command. + +Example: + +.. code-block:: hylang + + (setv mfa_code (Command + :name "otp" + :command "pass otp personal/app1")) + +.. autoclass:: Command + :members: + + .. _plugin_regex: Regex ------ ++++++ Use Regex plugin if the data you want extracted can be easily identified with a regular expression. The string matched in between @@ -138,7 +164,7 @@ Example: .. _plugin_html: Html ----- +++++ Use the Html plugin when the data you want can be easily extracted by parsing HTML tags. Create a new plugin by giving it a name, the tag @@ -168,19 +194,45 @@ Example: .. _plugin_json: Json ----- +++++ .. autoclass:: Json :members: -.. _plugin_api: +.. module:: raider.plugins.modifiers -Plugin API ----------- +Modifiers +--------- -.. autoclass:: Plugin +Alter ++++++ + +.. autoclass:: Alter + :members: + +Combine ++++++++ + +.. autoclass:: Combine + :members: + + + +.. module:: raider.plugins.parsers + +Parsers +------- +UrlParser ++++++++++ + +.. autoclass:: UrlParser + :members: + + + +.. _plugin_api: Writing custom plugins ---------------------- @@ -240,3 +292,6 @@ And we can create a new variable that will use this class: Now whenever we use the ``mfa_code`` in our requests, its value will be extracted from the password store. + + + diff --git a/docs/index.rst b/docs/index.rst index a2cab4d..ae20176 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,6 +1,16 @@ Welcome to Raider's documentation! ================================== +.. note:: + This documentation and the entire Raider framework is still work in + progress. Many things are not finished, stuff is missing, other stuff + is not working as expected, and so on... Meanwhile, `read the source + code `_ to understand better how + Raider works, `open Github issues + `_ if you find some + mistakes, or come `talk to us in the community forum + `_. + **Raider** is a framework designed to test :term:`authentication` for web applications. While web proxies like `ZAProxy @@ -87,7 +97,7 @@ User guide user/install user/architecture - user/tutorial + user/tutorials user/definitions user/faq diff --git a/docs/user/definitions.rst b/docs/user/definitions.rst index cb7d900..9667885 100644 --- a/docs/user/definitions.rst +++ b/docs/user/definitions.rst @@ -87,7 +87,7 @@ Definitions A piece of code that can be used to generate inputs for outgoing HTTP :term:`Requests `, and/or extract outputs from incoming term:`Responses `. All plugins inherit from - :class:`Plugin ` class. + :class:`Plugin ` class. When used inside a :term:`Request `, Plugins acts as input and replace themselves with the actual value. diff --git a/docs/user/faq.rst b/docs/user/faq.rst index f1a7d7a..cfd881b 100644 --- a/docs/user/faq.rst +++ b/docs/user/faq.rst @@ -24,8 +24,9 @@ files for each new project. Those files contain information describing the authentication process. **Raider** evaluates them, and gives you back a Python object to interact with the application. -Read the :ref:`Architecture ` and :ref:`Tutorial -` pages for more information and examples. +Read the :ref:`Architecture ` and `Tutorials +`_ for more information and +examples. .. _faq_eval: diff --git a/docs/user/tutorial.rst b/docs/user/tutorial.rst deleted file mode 100644 index 8331de1..0000000 --- a/docs/user/tutorial.rst +++ /dev/null @@ -1,620 +0,0 @@ -.. _tutorial: - -Tutorial -======== - -Preparation ------------ - -Before you can use **Raider**, you have to set up the -:term:`authentication` inside :term:`hyfiles`. To do that, you'll -probably need to use a web proxy (`BurpSuite -`_, `ZAProxy -`_, `mitmproxy `_, -etc...) to see the :term:`requests ` the application is -generating, and identify all the important inputs and outputs for each -request. - -After the traffic was captured, there will probably be lots of HTTP -requests that are irrelevant to the authentication. Start by removing -all static files (.png, .js, .pdf, etc...). When you're left with a -fewer requests to deal with, it's time to dive deeper and understand -how the authentication works. - -At this point we assume *you already know* the basics of Python and -Hylang so this documentation will not cover information that can be -found somewhere else. - -This tutorial will show the authentication in use by Reddit at the -time of writing this. It could be different in the future when you're -reading this, if they update the way authentication works or change -the HTML structure, so you will have to do this all by yourself -anyways. - -The easiest way to start this is by going backwards starting with one -authenticated request. This should be some kind of request that only -works when the user is already authenticated. I choose the -"unread_message_count" one for reddit, and the request looks like -this: - -.. code-block:: - - GET https://s.reddit.com/api/v1/sendbird/unread_message_count HTTP/1.1 - User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0 - Accept: application/json - Accept-Language: en-US,en;q=0.5 - Content-Type: application/json - Origin: https://www.reddit.com - DNT: 1 - Authorization: Bearer [REDACTED TOKEN] - Referer: https://www.reddit.com/ - Connection: keep-alive - Host: s.reddit.com - - -As you can see from this, the only information we sent to this URL -from our authentication is the Bearer token. - -We define a new :term:`Flow` that will check for the unread messages -in hy: - -.. code-block:: hylang - - (setv get_unread_messages - (Flow - :name "get_unread_messages" - :request (Request - :method "GET" - :headers [(Header.bearerauth access_token)] - :url "https://s.reddit.com/api/v1/sendbird/unread_message_count"))) - - -In Hy, ``setv`` is used to set up new variables. Here we created the -variable ``get_unread_messages`` that will hold the information about -this Flow. This will be hold in the :ref:`_functions special variable -` which stores the Flows which aren't affecting the -authentication. - -The only required parameters for :class:`Flow ` -objects are the name and the request. The name is a string that is used -for reference purposes, and the request contains the actual HTTP request -definition as a :class:`Request ` object. - -The Request object requires only the method and url. Other parameters -are optional. We translate the original request into **Raider** config -format, and to use the access token we need to define it in the request -header. Since this is a bearer header, we use :class:`Header.bearerauth -` with the ``access_token`` which we will create -later on. - - -Getting the access token ------------------------- - -The next step would be to find out where is this token generated and -how we can extract it. Searching for this token in previous responses, -we can see it was first seen in a request to the main reddit -page. It's located inside the `