Testing an Automation : YAML help

[boringpapa]

On request of Jeff , I'm sharing this here to get additional help.

Using Bots , I'm able to get instant gratification by simulating tickets and know if my code has worked or not . I don't understand the same with Automation yet

Purpose : Closing all tickets from Kotak.com older than 2 days

Request for YAML for testing this code

start:
  # Search for tickets from Kotak.com older than 2 days
  record.search/tickets:
    output: tickets
    inputs:
      record_type: ticket
      record_query@text:
        messages.first:(sender:(host:[kotak.com](http://kotak.com/))) created:"-2 days" status:open limit:50

  # Loop through the search results
  repeat:
    each@csv: {{tickets|keys|join(',')}}
    as: ticket_id
    do:
      # Close the ticket
      record.update/ticket:
        inputs:
          record_type: ticket
          record_id: {{ticket_id}}
          fields:
            status: closed

  # Return the total number of closed tickets
  return:
    closed_tickets@int: {{tickets|length}}

This is a sample I tried which gives me

input:
record_id: 123456
fields:
name: "John Doe"
email: "[johndoe@example.com](mailto:johndoe@example.com)"
status: "Open"
messages:
- sender:
host: kotak.com
created_at: 2022-04-01T09:00:00Z
body: "Hello, I have a question about my account."
- sender:
host: gmail.com
created_at: 2022-04-01T10:00:00Z
body: "Please provide more information about your issue."
- sender:
host: kotak.com
created_at: 2022-04-02T09:00:00Z
body: "I need to reset my password."
expected_output:
closed_tickets: 2

ERROR

__exit: error
error: 'yaml_parse(): parsing error encountered during parsing: did not find expected
  key (line 12, column 1), context while parsing a block mapping (line 1, column 1)'

[jstanden]

Hi @pakiyabhai!

This is a great question, and the docs on cerb.ai need to cover simulating automations better. You actually have much more visibility into what's going on with an automation than you ever did with bot behaviors.

I'm going to assume we're using an automation policy like:

commands:
  record.search:
    deny/type@bool: {{inputs.record_type is not record type ('ticket')}}
    allow@bool: yes
  record.update:
    deny/type@bool: {{inputs.record_type is not record type ('ticket')}}
    allow@bool: yes

An automation will run all commands like normal during simulation unless you implement an on_simulate: event for a command. It can even be empty.

So in your example, the record.search: command will run the actual search for you during simulation. It's perfectly fine because that's not a destructive action (like record.update: might be). You don't have to provide mock data in record.search:on_simulate: for the search results unless you have a good reason to do that (e.g. unit testing).

For instance, in your example we probably don't want to actually close any tickets while simulating. So you'd add an on_simulate: to that command like:

      # Close the ticket
      record.update/ticket:
        inputs:
          record_type: ticket
          record_id: {{ticket_id}}
          fields:
            status: closed
        on_simulate:

When on_simulate: is empty, it's essentially just skipping the command during simulation.

You can always toggle between Simulate and Execute mode on the Run tab of the automation editor:

I think Github also munged your search query, but it would look like:

  # Search for tickets from Kotak.com older than 2 days
  record.search/tickets:
    output: tickets
    inputs:
      record_type: ticket
      record_query@text:
        messages.first:(sender:(host:"kotak.com")) created:"-2 days" status:open limit:50

If you run that without any Inputs: you'll see in the Outputs: panel that the tickets: placeholder has the search results from your live data:

The only time you'd put something in Inputs: is if the automation event expects inputs. For instance, the mail.received event expects message__context: message and message_id: 123 as inputs, with an optional is_new_ticket: yes.

Those would be simulated like:

inputs:
  message__context: message
  message_id: 123
  is_new_ticket: no

If for some reason you wanted to provide mock results for a command during simulation (like your record.search:), you can do it like:

  # Search for tickets from Kotak.com older than 2 days
  record.search/tickets:
    output: tickets
    inputs:
      record_type: ticket
      record_query@text:
        messages.first:(sender:(host:"kotak.com")) created:"-2 days" status:open limit:50
    on_simulate:
      set:
        tickets:
          1:
            created@date: -2 days
            initial_message_sender_host: kotak.com
            initial_message_content: "Hello, I have a question about my account."
            ticket_id: 1
          2:
            created@date: -2 hours
            initial_message_sender_host: kotak.com
            initial_message_content: "I need to reset my password."
            ticket_id: 2

That's useful if you know your automation isn't going to match any live data at this moment. But in most cases, you'd just create some test data that you know will match.

You can also simulate error conditions (including randomly), like:

  # Search for tickets from Kotak.com older than 2 days
  record.search/tickets:
    output: tickets
    inputs:
      record_type: ticket
      record_query@text:
        messages.first:(sender:(host:"kotak.com")) created:"-2 days" status:open limit:50
    on_error:
      error: It broke!
    on_simulate:
      outcome/chaos:
        if@bool: {{random(1,10) <= 2}}
        then:
          simulate.error:
      set:
        # ...      

On the on_simulate: branch we have an outcome/chaos: command that will cause a random failure 20% of the time. That will run the on_error: branch rather than the on_success: branch.

simulate.error: is particularly useful for testing commands like http.request:, where an API request might fail but you have no control over the endpoint.

I'll make sure this finds its way to the docs.

Let me know if anything could be made more clear. 😃