You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It's commonly the case (especially in pilots) that data that is annotated needs to be re-annotated due to misunderstood instructions or bad worker quality controls. It's also the case that in a coordinated live task, some data may not end up annotated because one of the workers disconnects, leaving the task as incomplete. We'd like to improve the experience for both of these cases
Implementation
For the first, this would involve extending the DataBrowser to allow querying all incomplete or rejected units, and then create a tools.scripts utility function that leverages it to construct a new set of static data to be fed directly to the SharedTaskState in a run script.
For the second, we'd need to modify the generators for the TaskLauncher to remain open for generating re-releases of incomplete tasks as they arise. One early attempt at doing this (and more discussion) can be found here: #224
The text was updated successfully, but these errors were encountered:
Overview
It's commonly the case (especially in pilots) that data that is annotated needs to be re-annotated due to misunderstood instructions or bad worker quality controls. It's also the case that in a coordinated live task, some data may not end up annotated because one of the workers disconnects, leaving the task as incomplete. We'd like to improve the experience for both of these cases
Implementation
For the first, this would involve extending the
DataBrowser
to allow querying all incomplete or rejected units, and then create atools.scripts
utility function that leverages it to construct a new set of static data to be fed directly to theSharedTaskState
in a run script.For the second, we'd need to modify the generators for the
TaskLauncher
to remain open for generating re-releases of incomplete tasks as they arise. One early attempt at doing this (and more discussion) can be found here: #224The text was updated successfully, but these errors were encountered: