Labeling Instructions
Data annotation is an integral part of the AI solution development pipeline, regardless of its size and complexity. AI teams use annotation tools to get through this stage. Unfortunately, as the projects' size scales, teams struggle to process a significant amount of raw data that should be annotated and reviewed because of multiple disjointed collaborators involved.
To address this challenge and support you across the workflow, CloudFactory's annotation tool has several built-in features that aim to help developers organize and track the annotation workstream and collaborate with others in real time.
Utilizing these features in the workstream allows all the users to:
- Communicate with each other in real-time;
- Organize and operate with data in batches;
- Add specific labeling instructions for the annotation work;
- Automatically distribute and assign specific data for annotation to a specific expert annotator.
Let's touch on the Labeling Instructions feature.
Labeling Instructions explained
When leading a team of multiple annotators, ensuring everyone is proficient and informed about how to annotate specific data is pivotal. There can be numerous instructions and other documents that explain your particular use case and the taxonomy that is being used.
In CloudFactory, you can use the Labeling Instructions feature to distribute the actual information across the team at any given time to consult and continue their work in the required way.
Adding and Managing the instructions
You can access the Labelling Instruction feature by navigating to 'Project setup' in the project navigation. Here, you can find an overview of all the labeling instructions attached to the project. Additionally, in this panel, you also can add new links. Click on the 'Add new' button to start adding the relevant resources.
A side modal opens where you can add instructions by adding a name, link, and a note. For example, the note field can describe the resource you want to add or point out a critical section. You can add multiple or all sources you think are needed to ensure the tasks in your project will be completed successfully. You can focus on specific labeling instructions, store training documents, or point out edge cases.
Once you have added one or multiple links, a table with an overview of all labeling instructions will appear on this page. Here, you can edit, update, or remove them.
Accessing instructions
Labeling instructions are available for all project roles. You can find them by clicking the Labeling Instruction icon in the topbar of the Annotation Environment. If there are new instructions, a dot will appear on the icon to indicate new updates. When you click the icon, a side modal opens on top of the Annotation Environment with all available resources and their information, like name, link, status, creation date, note, and user.
The status of the Labeling Instruction indicates whether each user has opened the link. Only upon opening the link will the status change to 'Read', and the notification on the icon will disappear.
Notifications and Updates
If the instructions you have added change, but the link remains the same, you can manually trigger an update. To do so, please click the 'Notify users' option of the instructions table on the 'Project Setup' page. This will open a modal where you can add a description or note for the update that all your users will see. For example, this is a great option to clarify a specific instruction.
Every time a new or updated instruction is added, a modal appears when you open the annotation environment to inspire your collaborators to open the Labelling Instructions tab and increase the opportunity to discover and read your valuable instructions. Ensuring everyone in your project has all the required resources and is timely and well-informed is the pivotal key to successful results.
