Detectron Model Zoo and Baselines

Introduction

This file documents a large collection of baselines trained with Detectron, primarily in late December 2017. We refer to these results as the 12_2017_baselines. All configurations for these baselines are located in the configs/12_2017_baselines directory. The tables below provide results and useful statistics about training and inference. Links to the trained models as well as their output are provided. Unless noted differently below (see "Notes" under each table), the following common settings are used for all training and inference runs.

Common Settings and Notes

• All baselines were run on Big Basin servers with 8 NVIDIA Tesla P100 GPU accelerators (with 16GB GPU memory, CUDA 8.0, and cuDNN 6.0.21).
• All baselines were trained using 8 GPU data parallel sync SGD with a minibatch size of either 8 or 16 images (see the im/gpu column).
• For training, only horizontal flipping data augmentation was used.
• For inference, no test-time augmentations (e.g., multiple scales, flipping) were used.
• All models were trained on the union of coco_2014_train and coco_2014_valminusminival, which is exactly equivalent to the recently defined coco_2017_train dataset.
• All models were tested on the coco_2014_minival dataset, which is exactly equivalent to the recently defined coco_2017_val dataset.
• Inference times are often expressed as "X Y", in which X is time taken in reasonably well-optimized GPU code and Y is time taken in unoptimized CPU code. (The CPU code time could be reduced substantially with additional engineering.)
• Inference results for boxes, masks, and keypoints ("kps") are provided in the COCO json format.
• The model id column is provided for ease of reference.
• To check downloaded file integrity: for any download URL on this page, simply append .md5sum to the URL to download the file's md5 hash.
• All models and results below are on the COCO dataset.
• Baseline models and results for the Cityscapes dataset are coming soon!

Training Schedules

We use three training schedules, indicated by the lr schd column in the tables below.

• 1x: For minibatch size 16, this schedule starts at a LR of 0.02 and is decreased by a factor of * 0.1 after 60k and 80k iterations and finally terminates at 90k iterations. This schedules results in 12.17 epochs over the 118,287 images in coco_2014_train union coco_2014_valminusminival (or equivalently, coco_2017_train).
• 2x: Twice as long as the 1x schedule with the LR change points scaled proportionally.
• s1x ("stretched 1x"): This schedule scales the 1x schedule by roughly 1.44x, but also extends the duration of the first learning rate. With a minibatch size of 16, it reduces the LR by * 0.1 at 100k and 120k iterations, finally ending after 130k iterations.

All training schedules also use a 500 iteration linear learning rate warm up. When changing the minibatch size between 8 and 16 images, we adjust the number of SGD iterations and the base learning rate according to the principles outlined in our paper Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour.

License

All models available for download through this document are licensed under the Creative Commons Attribution-ShareAlike 3.0 license.

ImageNet Pretrained Models

The backbone models pretrained on ImageNet are available in the format used by Detectron. Unless otherwise noted, these models are trained on the standard ImageNet-1k dataset.

• R-50.pkl: converted copy of MSRA's original ResNet-50 model
• R-101.pkl: converted copy of MSRA's original ResNet-101 model
• X-101-64x4d.pkl: converted copy of FB's original ResNeXt-101-64x4d model trained with Torch7
• X-101-32x8d.pkl: ResNeXt-101-32x8d model trained with Caffe2 at FB
• X-152-32x8d-IN5k.pkl: ResNeXt-152-32x8d model trained on ImageNet-5k with Caffe2 at FB (see our ResNeXt paper for details on ImageNet-5k)

Log Files

Training and inference logs are available for most models in the model zoo.

Proposal, Box, and Mask Detection Baselines

RPN Proposal Baselines

Notes:

• Inference time only includes RPN proposal generation.
• "prop. AR" is proposal average recall at 1000 proposals per image.
• Proposal download links ("props"): "1" is coco_2014_train; "2" is coco_2014_valminusminival; and "3" is coco_2014_minival.

Fast & Mask R-CNN Baselines Using Precomputed RPN Proposals

Notes:

• Each row uses precomputed RPN proposals from the corresponding table row above that uses the same backbone.
• Inference time excludes proposal generation.

End-to-End Faster & Mask R-CNN Baselines

Notes:

• For these models, RPN and the detector are trained jointly and end-to-end.
• Inference time is fully image-to-detections, including proposal generation.

RetinaNet Baselines

Notes: none

Mask R-CNN with Bells & Whistles

Notes:

• A deeper backbone architecture is used: ResNeXt-152-32x8d-FPN
• The backbone ResNeXt-152-32x8d model was trained on ImageNet-5k (not the usual ImageNet-1k)
• Training uses multi-scale jitter over scales {640, 672, 704, 736, 768, 800}
• Row 1: test-time augmentations are multi-scale testing over {400, 500, 600, 700, 900, 1000, 1100, 1200} and horizontal flipping (on each scale)
• Row 2: same model as row 1, but without any test-time augmentation (i.e., same as the common baseline configuration)
• Like the other results, this is a single model result (it is not an ensemble of models)

Keypoint Detection Baselines

Common Settings for Keypoint Detection Baselines (That Differ from Boxes and Masks)

Our keypoint detection baselines differ from our box and mask baselines in a couple of details:

• Due to less training data for the keypoint detection task compared with boxes and masks, we enable multi-scale jitter during training for all keypoint detection models. (Testing is still without any test-time augmentations by default.)
• Models are trained only on images from coco_2014_train union coco_2014_valminusminival that contain at least one person with keypoint annotations (all other images are discarded from the training set).
• Metrics are reported for the person class only (still run on the entire coco_2014_minival dataset).

Person-Specific RPN Baselines

Notes:

• Metrics are for the person category only.
• Inference time only includes RPN proposal generation.
• "prop. AR" is proposal average recall at 1000 proposals per image.
• Proposal download links ("props"): "1" is coco_2014_train; "2" is coco_2014_valminusminival; and "3" is coco_2014_minival. These include all images, not just the ones with valid keypoint annotations.

Keypoint-Only Mask R-CNN Baselines Using Precomputed RPN Proposals

Notes:

• Metrics are for the person category only.
• Each row uses precomputed RPN proposals from the corresponding table row above that uses the same backbone.
• Inference time excludes proposal generation.

End-to-End Keypoint-Only Mask R-CNN Baselines

Notes:

• Metrics are for the person category only.
• For these models, RPN and the detector are trained jointly and end-to-end.
• Inference time is fully image-to-detections, including proposal generation.