Review and correct¶
You need: the reviewer dataset role or higher.
Review and correction are separate jobs on purpose — read Review and correction for why.
Setting up a review queue¶
- Open the dataset's review page and configure a queue.
- Pick a sort strategy — the order objects are presented in. Confidence-based orderings put the model's least certain output first, which is where reviewer time changes the most.
- Create the queue. It is a snapshot: a fixed list in a fixed order, so it has a definite end and does not reshuffle underneath you.
Reviewing¶
Work through the queue giving each object a verdict. R rejects in review mode.
Rejected objects resolve as one of:
| Resolution | Meaning |
|---|---|
| fixed | it was wrong, and it has been corrected |
| wont_fix | it was wrong, and it is being left that way deliberately |
wont_fix is a real answer
Recording that a human looked and chose not to change something is different from nobody having looked. That difference is what lets you state the annotation quality of a dataset later.
Correcting¶
The correction pass is where rejected objects get fixed, using the same drawing and editing tools as annotation. Model output stays separable from human edits throughout, so a corrected annotation knows it was corrected and by whom.
Next: Train a model on what you just corrected.