-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Documentation of term-search-basic lambda
- Loading branch information
1 parent
2b766f1
commit b9eaaad
Showing
3 changed files
with
52 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
.. _mp-covenants-term-search-basic: | ||
|
||
mp-covenants-term-search-basic | ||
=============================== | ||
|
||
- Code: `mp-covenants-term-search-basic on Github <https://github.com/UMNLibraries/mp-covenants-term-search-basic>`_ | ||
|
||
This component looks for racial and other terms to flag as potential racial covenants. For each term in covenant_flags, check OCR JSON file received from previous step for existance of term. This is a really simple CNTRL + F style search, no partial matching, regex, fuzzy, etc. Some of the terms are actually exceptions rather than covenant hits, and once they reach the Django stage, will be used to mark this page as exempt from consideration as being considered as a racial covenant. Common examples of exceptions include birth certificates and military discharges, which often contain racial information but are not going to contain a racial covenant.. This is the third Lambda in the Deed Machine initial processing Step Function. | ||
|
||
|
||
Steps of the function | ||
--------------------- | ||
|
||
1. Check event for valid image. | ||
2. For each term, search each line of text for term | ||
3. If term found, add to output JSON along with which line numbers it appears in. | ||
4. Return path to match file. | ||
|
||
Software development requirements | ||
--------------------------------- | ||
|
||
The Lambda components of the Deed Machine are built using Amazon's Serverless Application Model (SAM) and the AWS SAM CLI tool. | ||
|
||
- Pipenv (Can use other virtual environments, but will require fiddling on your part) | ||
- AWS SAM CLI | ||
- Docker | ||
- Python 3 | ||
|
||
Quickstart commands | ||
------------------- | ||
|
||
To build the application: | ||
|
||
.. code-block:: bash | ||
pipenv install | ||
pipenv shell | ||
sam build | ||
To rebuild and deploy the application: | ||
|
||
.. code-block:: bash | ||
sam build && sam deploy | ||
To run tests: | ||
|
||
.. code-block:: bash | ||
pytest |