Beginner's API Hello World Tutorial

Now that you're familiar with the Screenshot Editor. Let's look at the the API that you'll be using to write layers. To do this, we will walk you through the functionality of the Hello World application that you saw running in the Screenshot Editor. First, let's overview the API here: Authoring Layers

Once we've overviewed the API, let's create a layer that will heuristically guess whether elements are text or not. To do this, we first open a python file in the with the name "tag_leaves_as_text.py" in the layers folder (the proctor can help you navigate to this directory). The proctor will delete the contents of this and you will rewrite it :) You can then add this layer to the layer chain by clicking the "Add Layers" button in the screenshot editor and navigating to the file.

Now whenever a screenshot is loaded in the Screenshot Editor, the interpret method in our script will be executed. Let's implement that function now. The function takes interpret_data object as a parameter. interpret_data contains a tree object that represents Prefab's output. We will recursively visit each node in this tree, so let's write a helper function that will let us do that, it should accept args and a current tree node as arguments.

def interpret(interpret_data):
	find_leaves_and_tag_as_text(interpret_data, interpret_data.tree)

In the helper function, we will tag each node and use a heuristic to guess if it is text. Specifically, we will check if the node is a leaf, because these elements typically correspond to textual content. To do this, we check each node to see if it contains any children. If a node is identified as a leaf, then we tag it with "is_text" = True. If the node is not a leaf, tag it as False and we recurse on its children. The tree is readonly, so we cannot directly set the tags on the tree. Instead we use the tree_transformer object to queue the tag operations so that Prefab will execute them after our layer completes.

def interpret(interpret_data):
	find_leaves_and_tag_as_text(interpret_data, interpret_data.tree)
	
	
def find_leaves_and_tag_as_text(interpret_data, currnode):
	if len( currnode.get_children() ) == 0:
		interpret_data.tree_transformer.enqueue_set_tag(currnode, 'is_text', True)
		
	for child in currnode.get_children():
		find_leaves_and_tag_as_text(interpret_data, child)

Now let's test our layer to make sure it worked. Save your script and move back to the Screenshot Editor. Click the "Relaod Layers" button. Click on an element and look at its properties to see if it was labeled as text or not.

Our layer relies on a heuristic to guess if elements are text, which can be incorrect occasionally. And so the next thing we want to do is make sure that we can correct any erroneous tags. To do this, we will import an existing layer that uses human-provided corrections to overwrite the erroneous tags. This layer, "apply_correction_annotations" is already in the layers folder, so all we need to do is add it to our layer chain using the Screenshot Editor using the "Add Layers" button. This layer can be parameterized to use any type of corrections, not just corrections for text. So we need to parameterize this layer to store corrections in our own library specifically for text. To do this, add a parameter using the text boxes below the layer chain. Add a library with the value "text_corrections". This tells the layer where to store the corrections. Click "Submit", and then click "Reload Layers" to reload your Layer Chain.

Some corrections have already been stored for you! But notice one of the icons at the top of the screenshot was labeled as text. Click on this icon and mark it as not text by clicking "Annotate", "is_text = false".

Great! Now everything is labeled as text, but we want to group the text together. To do this, we will import two layers that have already been implemented for you, the "tag_group_next.py" and the "apply_grouping.py" layers. These two layers are used to group related text. Add these layers to your chain, reload the chain, and view the Hello World output.

Hooray, it works! Let's continue this tutorial to write more advanced layers Advanced API Hello World Tutorial