Speaker
Description
Imaging systems now acquire images faster than their contents can be annotated by hand, making cataloguing the bottleneck and leading to increasing costs with collection size. Existing methods only address a part of the problem: unsupervised segmentation and embedding group visually similar objects but don’t attach labels to the group; supervised detectors return interpretable classes while demanding exhaustive per object annotation; vision-language models describe entire scenes without reference to object-level detections. We therefore couple all three into a single pipeline and evaluate on seafloor imagery.
Objects are first segmented and embedded with pretrained foundation models and then clustered without labels. A domain expert reviews a small, fixed number of representative crops from each cluster, and every accepted cluster is promoted to a trained detector. Because that number is fixed, annotation effort per category no longer scales with how many instances the collection happens to contain. Finally, a vision-language model captions each image, and the captions are cross-checked against the detections.
Clustering runs to completion on collections of several hundred images. Detectors bootstrapped from cluster-level review require an order-of-magnitude less annotation effort than a fully supervised baseline trained on exhaustive per-object annotation. A systematic accuracy comparison is left to future work. The captioning stage has so far been assessed qualitatively; quantitative evaluation is in progress. The pipeline was tested on a single imaging domain.