Region network proposals with Faster R-CNN

Deep Learning for Images with PyTorch

Michal Oleszak

Machine Learning Engineer

Regions and anchor boxes

  • Region: a smaller area of the image that could contain objects of interest, grouped by visual characteristics

region proposals

Deep Learning for Images with PyTorch

Regions and anchor boxes

  • Region: a smaller area of the image that could contain objects of interest, grouped by visual characteristics

region proposals

  • Anchor box: predefined bounding box templates of different sizes and shapes
Deep Learning for Images with PyTorch

Faster R-CNN model

Faster R-CNN: an advanced version of R-CNN

rcnn layers

  • Backbone (convolutional layers)
1 Edward Raff. 2022. Inside Deep Learning.
Deep Learning for Images with PyTorch

Faster R-CNN model

Faster R-CNN: an advanced version of R-CNN rcnn layers

  • Backbone (convolutional layers)
  • Region proposal network (RPN) for bounding box proposals
1 Edward Raff. 2022. Inside Deep Learning.
Deep Learning for Images with PyTorch

Faster R-CNN model

Faster R-CNN: an advanced version of R-CNN

rcnn layers

  • Convolution layers (backbone): feature maps
  • Region proposal network (RPN): bounding box proposals
  • Classifier and regressor to produce predictions
1 Edward Raff. 2022. Inside Deep Learning.
Deep Learning for Images with PyTorch

Region proposal network (RPN)

region proposal network architecture

Deep Learning for Images with PyTorch

Region proposal network (RPN)

region proposal network architecture

  • Anchor generator:
    • Generate a set of anchor boxes of different sizes and aspect ratios
Deep Learning for Images with PyTorch

Region proposal network (RPN)

region proposal network architecture

  • Anchor generator:
    • Generate a set of anchor boxes of different sizes and aspect ratios
  • Classifier and regressor:
    • Predict if the box contains an object and provide coordinates
Deep Learning for Images with PyTorch

Region proposal network (RPN)

region proposal network architecture

  • Anchor generator:
    • Generate a set of anchor boxes of different sizes and aspect ratios
  • Classifier and regressor:
    • Predict if the box contains an object and provide coordinates
  • Region of interest (RoI) pooling:
    • Resize the RPN proposal to a fixed size for fully connected layers
Deep Learning for Images with PyTorch

RPN in PyTorch

from torchvision.models.detection.rpn import AnchorGenerator


anchor_generator = AnchorGenerator( sizes=((32, 64, 128),), aspect_ratios=((0.5, 1.0, 2.0),), )
from torchvision.ops import MultiScaleRoIAlign


roi_pooler = MultiScaleRoIAlign( featmap_names=["0"], output_size=7, sampling_ratio=2, )
Deep Learning for Images with PyTorch

Fast R-CNN loss functions

  • RPN classification loss:
    • region contains object or not
    • binary cross-entropy
    • rpn_cls_criterion = nn.BCEWithLogitsLoss()

 

  • RPN box regression loss:
    • bounding box coordinates
    • mean squared error
    • rpn_reg_criterion = nn.MSELoss()
  • R-CNN classification loss:
    • multiple object classes
    • cross-entropy
    • rcnn_cls_criterion = nn.CrossEntropyLoss()

 

  • R-CNN box regression loss:
    • bounding box coordinates
    • mean squared error
    • rcnn_reg_criterion = nn.MSELoss()
Deep Learning for Images with PyTorch

Faster R-CNN in PyTorch

from torchvision.models.detection import FasterRCNN


backbone = torchvision.models.mobilenet_v2(weights="DEFAULT").features
backbone.out_channels = 1280
model = FasterRCNN( backbone=backbone, num_classes=num_classes, rpn_anchor_generator=anchor_generator, box_roi_pool=roi_pooler, )
Deep Learning for Images with PyTorch

Faster R-CNN in PyTorch

Load pre-trained Faster R-CNN

from torchvision.models.detection.faster_rcnn import FastRCNNPredictor

model = torchvision.models.detection.fasterrcnn_resnet50_fpn(weights="DEFAULT")

Define number of classes and classifier input sise

num_classes = 2

in_features = model.roi_heads.box_predictor.cls_score.in_features

Replace model's classifier with a one with the desired number of classes

model.roi_heads.box_predictor = FastRCNNPredictor(in_features, num_classes)
Deep Learning for Images with PyTorch

Let's practice!

Deep Learning for Images with PyTorch

Preparing Video For Download...