Skip to main content

​
Updating Your Project Setup

After launching a batch, there are some aspects of your project setup that can be updated and others that are locked:

​
Taxonomy

Updating your taxonomy after onboarding requires a few followup actions in order to keep your project in a healthy state. Click on ‘Task Setup’ in the sidebar navigation in order to preview your current taxonomy and instructions. In the top right corner, click on ‘Edit Taxonomy’ to get to the taxonomy editor. As a reminder, the changes made in this flow will not update any live tasks and will only be applied to your next batch. Make the necessary changes here (refer to the Taxonomy section of our Project Creation workflow for help with this editor) and hit next. This will bring you to the ‘Propagate Changes’ step, where you can update all of the tasks that are affected by your taxonomy changes. The first section shows you a revision history as it pertains to the affected tasks (note: changes made via the JSON editor will not show up in this section). Review this and confirm that it is accurate. In the next three sections, you will see galleries of instruction examples, training tasks, and quality tasks. Here you can push your new taxonomy version to the tasks individually or all at once. We will try our best to reconcile the task responses according to your changes. While flipping through tasks, confirm that the task responses match what you expect and create/delete any annotations that were reconciled improperly. Note that we are not able to generate any annotations for the new labels that you’ve added to your taxonomy so you will have to complete this yourself. Updating your taxonomy after onboarding requires a few followup actions in order to keep your project in a healthy state. Click on ‘Task Setup’ in the sidebar navigation in order to preview your current taxonomy and instructions. In the top right corner, click on ‘Edit Taxonomy’ to get to the taxonomy editor. As a reminder, the changes made in this flow will not update any live tasks and will only be applied to your next batch. Make the necessary changes here (refer to the Taxonomy section of our Project Creation workflow for help with this editor) and hit next. This will bring you to the ‘Propagate Changes’ step, where you can update all of the tasks that are affected by your taxonomy changes. The first section shows you a revision history as it pertains to the affected tasks (note: changes made via the JSON editor will not show up in this section). Review this and confirm that it is accurate. In the next three sections, you will see galleries of instruction examples, training tasks, and quality tasks. Here you can push your new taxonomy version to the tasks individually or all at once. We will try our best to reconcile the task responses according to your changes. While flipping through tasks, confirm that the task responses match what you expect and create/delete any annotations that were reconciled improperly. Note that we are not able to generate any annotations for the new labels that you’ve added to your taxonomy so you will have to complete this yourself. Once you’re satisfied that all of your examples, training tasks, and quality tasks reflect the new taxonomy, proceed to the final step of updating your instructions. At this point, your new taxonomy has already been saved and any task updates have been propagated. Here we provide suggestions based on your changes (i.e. highlighting a section that was renamed, creating a section for new labels, recommending to delete any obsolete sections). Review our suggestions, revise if necessary, and confirm via the stepper. This will take you back to taxonomy and instructions previews where we conclude this process. Your changes will be applied to the next batch.

​
Instructions

Iterating on your instructions is an important part of the calibration process. We often find that our most successful users revise their instructions after every completed batch (adding edge case examples, tweaking phrasing, clarifying terminology). Writing clear, thorough instructions is the easiest and most direct way to increase your annotation quality. Once your first calibration batch is complete, you can review the instructions feedback for each task on the batch page. This feedback will also be aggregated on the instructions editor view underneath the instruction outline. When you are ready to edit your instructions, head over to the task setup page from the sidebar navigation. Here you can preview both your existing taxonomy and instructions. In the top right corner, click ‘Edit Instructions’ to enter the instructions editor. Make any changes as you see fit, and click ‘Update’. This will provide you with 2 options: updating live tasks with this new version of your instructions or deferring it to subsequent batches. Select the option that matches your intent and click ‘Confirm’.