MS COCO

Overview

MS COCO (Common Objects in Context) is Microsoft’s large-scale object detection, segmentation, and captioning dataset. Its emphasis on objects in natural, complex scenes with rich annotations made it far more challenging and useful than previous datasets.

What’s In It

COCO contains 330,000 images with over 2.5 million labeled instances across 80 object categories. Annotations include bounding boxes, instance segmentation masks, keypoint annotations for people, and five human-written captions per image.

How It’s Used

COCO is the standard evaluation benchmark for object detection, instance segmentation, and image captioning. Its detection metrics are the field standard. The captioning annotations are used for vision-language pre-training.

Controversies

COCO’s 80 categories represent a Western-centric view of common objects. The dataset has demographic imbalances in person annotations and captions can contain stereotypical descriptions.