Skip to content

Creating a new experiment type

Jump to bottom
mscholman edited this page Apr 4, 2017 · 21 revisions

Creating new experiment types is one of the most common tasks for an experimenter. Using a simple example (i.e. a story completion experiment), we'll explain how this task can be achieved easily. For users who are new to LingoTurk, we recommend to first follow these steps to create the story completion experiment, before creating the proper experiment type needed for your own experiment.

  1. After clicking on "Create experiment" (on the landing page) you'll see an overview page that lists all currently existing experiment types. If you wish to use an existing template, you can go straight to [instantiating your experiment](https://github.com/FlorianPusse/Lingoturk/wiki/Instantiating-an-experiment) If you wish to create a new experiment type, click on the "Create a new experiment type" tile.

  2. You will be redirected to the experiment creation interface, as displayed below: **(note: interface is now updated, new picture needs to be added)**

    Here are the basic steps you need to follow:

    1. The first thing you have to do is choosing a name for your new experiment type. This name must consist only of alphanumeric characters and must be unique amongst all experiment types. We choose "DEMOStoryCompletion" to be our experiment type name.
    2. The next step is to set the overall experimental condition. First choose whether participants should do all the items in a list, or whether they should be able to see multiple items in a list. **(tba: the difference between these)**
    3. Then choose the list type. If you have several lists and they are all independent, you can choose to have participants do as many lists as they want. However, if one of your lists contains one item in a particular condition and the other contains that same item in another condition, you probably don't want participants to be able to do multiple lists.
    4. The final step in experiment creation is specifying your experimental materials, i.e. the information you need to store in the database. Possible things that you might want to store include itemID, condition, itemText, comprehensionQuestion, etc. Some of these will be displayed to the participant (itemText, comprehensionQuestion), others will be hidden from the participant but stored in the database (itemID, condition). For our example, we have a string that represents the story that should be completed. Additionally, we want to store another string that contains some type information.
    5. After completing this page, we can press the "Submit" button to store our experiment. If you get redirected to a red page that has a button saying ''apply this script now'', click on this button.

Managing your newly created experiment type

There's still more to do in designing your experiment. In this section, we describe what to expect once you've generated the new experiment type. In a nutshell, you've created an empty experiment template with no user interface. Here we describe where to go to make your custom user interface.

  1. Now you've successfully created your new experiment type. That means LingoTurk creates a ''template'' that corresponds to the information you entered before. So LingoTurk has added your new experiment type to the overview page and you are able to instantiate your new experiment type as described here: Instantiating an experiment
  2. Once you've instantiated your experiment, you can view it by going to ''Manage existing experiments'' and clicking on the eye next to your experiment name. When you view the instantiated experiment you will see the instructions that you entered. Click ''Next". You will be asked to enter your workerID. Fill in something random (e.g., ''x'') and click on ''Next''. Then you will see something similar to the image below:

    As you can see, the data that you specified when instantiating the experiment (your story, the itemID) is loaded and automatically displayed in a "raw" way. (On the picture you see a javascript object that contains the data of a single item. Remember the variables we picked when creating the experiment type? Here they are.)

    The next step is to specify which parts of the data should be displayed to the participants and what they have to do in order to complete the experiment. This needs to be done by changing the automatically generated code in the following file: lingoturk/app/views/ExperimentRendering/YourExperiment/YourExperiment_render.scala.html. (Note: We are aware that the need of looking these files up and changing them directly is not optimal. We plan to implement an interface that supports you while doing so)

    Open the file in any text editor. This file contains code that is responsible for loading your experimental materials, submitting them back to the LingoTurk server, and also for the content displayed to the participants. The relevant parts in the file are marked by @ and @ comments:

  3. The {{question}} in the HTML is replaced with the content of the question variable. As we want to display only the story-field to the participants, we can replace complete that statement to be {{question.story}}. When reloading the experiment you'll notice that the content of the panel has been updated to something more meaningful, as shown below:

  4. Since our main goal is to design a story completion experiment, we have to provide an input field for the particpants and we have to make sure that their results are tracked somewhere.

    Thus we add a simple input-element to our code. Let's first take a look at what elements there are in the code. Look for this line in the script: <textarea class="form-control textInput" ng-model="question.answer"></textarea> The class form-control is used for a prettier appearance. We have already set the default of this class, so you will not need to change this. The ng-model="question.answer" attribute is much more meaningful. It takes care of tracking the user's results by storing them in the answer field of the question variable. Values in the answer-field will be sent back automatically to the server later on, which is why you should always store them there (unless you have some more advanced needs that require manual configuration).
    Note: answer-field values can also be of array or object type.

    Putting all that together, we get the following experiment:

  5. In case we want to extract specific variables from our participants' answers, for instance condition or item number, we edit the automatically created file resultsQuery.sql in lingoturk/app/models/Questions/YourExperiment/. In our case, we are not only interested in our participants' answers but also in the corresponding values of story and type (cf. step 2). To do so we change the content of resultsQuery.sql from
    SELECT * FROM DEMOStoryCompletionResults
    to
    SELECT DEMOStoryCompletionResults.*,DEMOStoryCompletion_story, DEMOStoryCompletion_type FROM DEMOStoryCompletionResults JOIN Questions USING (QuestionId)
  6. Congratulations. We successfully implemented our story completion experiment. For common use cases such as the one described here we could also use one of the existing templates instead of creating the design from scratch. For more information about that you should take a look here: Experiment templates
Clone this wiki locally