Add save_attachments action and provide enhancements to search_items action.

[TimothyDJones]

1.1.0

• Add save_attachments action.

  • Includes several pack configuration options:
    • attachment_directory: Fully-qualified server path name used to store attachments. Must be readable and writeable by Stackstorm. Defaults to "/opt/stackstorm/packs/msexchange/attachments".
    • attachment_folder_maximum_size: Maximum storage space in MB (default is 50MB) alloted to attachment_directory. Pack maintenance process (see below) manages this.
    • attachment_days_to_keep: Maximum number of days to keep saved attachments (default is 7 days). Also, managed by pack maintenance process.
  • Uses same model as search_items for finding email messages for which to save attachments. Only email messages and file attachments are supported.
  • Action returns a list/array of dictionaries with following attributes:
    • email_subject - Full subject of the email.
    • email_sent - Date that email was sent.
    • sender_email_address - Email address of the sender.
    • attachment_files - List/array of fully-qualified filenames from server of attachments saved. Example:

    [
        {
            "email_subject": "ACCOUNT LIST - resend for testing",
            "email_sent": "2021-06-25 13:54:34+00:00",
            "sender_email_address": "someone@example.com",
            "attachment_files": [
            "/opt/stackstorm/packs/msexchange/attachments/Accounts_06_23_2021.xlsx"
            ]
        }
    ]

  • If attachment filename is not unique in target folder, attempts to generate a unique filename for each attachment via several methods, including date sent, date and time sent, and "random" 8-character string.
  • Attachments can be saved either as BINARY (default) or TEXT format.
  • An attachment_directory_maintenance sensor/trigger/rule combination has been implemented and, by default, runs once daily (polling interval of 86400 seconds) to enforce these rules through the do_attachment_directory_maintenance action. This action can be run manually, as well, if you need to override the pack configuration values; running manually it with no input values uses the pack configuration.
  • Maintenance process removes files by age first and then, if necessary, deletes remaining files starting with largest files until threshold is reached.

• Enhancements to search_items action.

  • Moved search logic from run method in search_items action into _search_items utility method in base/actions.py to allow functionality to be shared by search_items and save_attachments.
  • Added search_start_date parameter specifying the start date for items to search. (End date is always "today".) Date can be entered as free-form text. Most any date format is supported, as dateutil library is used to parse input to valid datetime value.
  • Update search_items action to return additional item attributes, specific to email messages, from item_to_dict helper method. Such attributes can be useful in filtering e-mails based on sender and/or originating domain.
    • sender_email_address - Email address of sender.
    • email_recipient_addresses - List/array of email recipients (from exchangelib to_recipients list only).
  • Added optional folder_name parameter to item_to_dict helper method to include the name of the folder used in the search as attribute of returned dictionary.
  • Update requirements.txt to include python-dateutil (see above) and pytz, which is needed for creating timezone-aware exchangelib EWSDateTime objects for date searches.

[TimothyDJones]

@tonybaloney Hi. Just checking on status of this pull request. I don't anticipate any further changes on my branch in the near future and would like to get it merged into "master" when possible. Thanks!

[blag]

Please remove .vscode/settings.json from your PR.

[TimothyDJones]

Please remove .vscode/settings.json from your PR.

Sorry for the delay in making this update. Should be good now.

[minsis]

Maybe to prep for the upgrade of exchangelib in #16, remove the get_folder_by_name outlined in #20 as it is now deprecated.

[minsis]

Might also be a nice feature to add is_read filter to the items search. This would allow someone the option of ignoring items that are already marked as read.

[amanda11]

I'm happy with this change, @minsis are your comments improvements, or do they need to be addressed for your use-cases before this is merged?

[minsis]

I'm happy with this change, @minsis are your comments improvements, or do they need to be addressed for your use-cases before this is merged?

They're just improvements and not requirements at this time. However, to upgrade exhangelib it will be required to remove get_folder_by_name method as it is deprecated.

[amanda11]

LGTM - changes look good to me, and thanks for the contribution and the enhancements.

It would be good to get some tests into this pack, but I'm not going to hold up this PR for that as we don't have any tests. So putting in a basic test structure for this pack probably should go in as a PR in its own right.