华为云AI|物体检测-Faster R-CNN

观前提示:请将页面调成白色主题(点击右上角水彩调色盘)

物体检测是计算机视觉中的一个重要的研究领域,在人流检测,行人跟踪,自动驾驶,医学影像等领域有着广泛的应用。不同于简单的图像分类,物体检测旨在对图像中的目标进行精确识别,包括物体的位置和分类,因此能够应用于更多高层视觉处理的场景。例如在自动驾驶领域,需要辨识摄像头拍摄的图像中的车辆、行人、交通指示牌及其位置,以便进一步根据这些数据决定驾驶策略。上一期学习案例中,我们聚焦于YOLO算法,YOLO(You Only Look Once)是一种one-stage物体检测算法,在本期案例中,我们介绍一种two-stage算法——Faster R-CNN,将目标区域检测和类别识别分为两个任务进行物体检测。

点击跳转至Faster-RCNN模型简介

进入环境

进入ModelArts

点击如下链接:https://www.huaweicloud.com/product/modelarts.html , 进入ModelArts主页。点击“立即使用”按钮,输入用户名和密码登录,进入ModelArts使用页面

创建ModelArts notebook

下面,我们在ModelArts中创建一个notebook开发环境,ModelArts notebook提供网页版的Python开发环境,可以方便的编写、运行代码,并查看运行结果。

第一步:在ModelArts服务主界面依次点击“开发环境”、“创建”create_nb_create_button

第二步:填写notebook所需的参数:

项目建议填写方式
名称自定义环境名称
工作环境Python3
资源池选择”公共资源池”即可
类型GPU
规格[限时免费]体验规格GPU版
存储配置EVS
磁盘规格5GB

第三步:配置好notebook参数后,点击下一步,进入notebook信息预览。确认无误后,点击“立即创建“

第四步:创建完成后,返回开发环境主界面,等待Notebook创建完毕后,打开Notebook,进行下一步操作modelarts_notebook_index

在ModelArts中创建开发环境

接下来,我们创建一个实际的开发环境,用于后续的实验步骤。

第一步:点击下图所示的“打开”按钮,进入刚刚创建的Notebook,enter_dev_env

第二步:创建一个Python3环境的的Notebook。点击右上角的”New”,然后选择Pytorch-1.0.0开发环境。

第三步:点击左上方的文件名”Untitled”,并输入一个与本实验相关的名称,

在Notebook中编写并执行代码

在Notebook中,我们输入一个简单的打印语句,然后点击上方的运行按钮,可以查看语句执行的结果:”,run_helloworld

开发环境准备好啦,接下来可以愉快地写代码啦!”

数据准备

首先,我们将需要的代码和数据下载到Notebook。

本案例我们使用PASCAL VOC 2007数据集训练模型,共20个类别的物体。In [1]:

import os
from modelarts.session import Session
sess = Session()
if sess.region_name == 'cn-north-1':
    bucket_path="modelarts-labs/notebook/DL_object_detection_faster/fasterrcnn.tar.gz"
elif sess.region_name == 'cn-north-4':
    bucket_path="modelarts-labs-bj4/notebook/DL_object_detection_faster/fasterrcnn.tar.gz"
else:
    print("请更换地区到北京一或北京四")
if not os.path.exists('./experiments'):
    sess.download_data(bucket_path=bucket_path, path="./fasterrcnn.tar.gz")
if os.path.exists('./fasterrcnn.tar.gz'):
    # 解压压缩包
    os.system("tar -xf ./fasterrcnn.tar.gz")
    # 清理压缩包
    os.system("rm -r ./fasterrcnn.tar.gz")

安装依赖并引用

In [2]:

!pip install pycocotools==2.0.0
!pip install torchvision==0.4.0
!pip install protobuf==3.9.0
Collecting pycocotools==2.0.0
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/96/84/9a07b1095fd8555ba3f3d519517c8743c2554a245f9476e5e39869f948d2/pycocotools-2.0.0.tar.gz (1.5MB)
    100% |████████████████████████████████| 1.5MB 54.9MB/s eta 0:00:01
Building wheels for collected packages: pycocotools
  Running setup.py bdist_wheel for pycocotools ... done
  Stored in directory: /home/ma-user/.cache/pip/wheels/94/39/5f/52a87f45927330522a105995b30ee880c64da963c6d879c819
Successfully built pycocotools
Installing collected packages: pycocotools
Successfully installed pycocotools-2.0.0
You are using pip version 9.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting torchvision==0.4.0
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/06/e6/a564eba563f7ff53aa7318ff6aaa5bd8385cbda39ed55ba471e95af27d19/torchvision-0.4.0-cp36-cp36m-manylinux1_x86_64.whl (8.8MB)
    100% |████████████████████████████████| 8.8MB 90.7MB/s eta 0:00:01           | 2.7MB 80.7MB/s eta 0:00:01
Requirement already satisfied: numpy in /home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages (from torchvision==0.4.0)
Requirement already satisfied: six in /home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages (from torchvision==0.4.0)
Requirement already satisfied: pillow>=4.1.1 in /home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages (from torchvision==0.4.0)
Collecting torch==1.2.0 (from torchvision==0.4.0)
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/30/57/d5cceb0799c06733eefce80c395459f28970ebb9e896846ce96ab579a3f1/torch-1.2.0-cp36-cp36m-manylinux1_x86_64.whl (748.8MB)
    89% |████████████████████████████▋   | 669.3MB 108.2MB/s eta 0:00:01    100% |████████████████████████████████| 748.9MB 58.2MB/s ta 0:00:011% |████████████████████████████▊   | 673.3MB 86.1MB/s eta 0:00:01 0:00:02  90% |█████████████████████████████   | 680.3MB 57.0MB/s eta 0:00:02█████▏  | 682.4MB 86.9MB/s eta 0:00:01�██████▎  | 684.7MB 104.6MB/s eta 0:00:01█████▍  | 686.5MB 79.7MB/s eta 0:00:01��█████▌  | 689.1MB 92.9MB/s eta 0:00:01.1MB 82.5MB/s eta 0:00:01█████▋  | 693.3MB 91.1MB/s eta 0:00:01█████▉  | 697.9MB 51.3MB/s eta 0:00:01.3MB 73.7MB/s eta 0:00:013.2MB 101.2MB/s eta 0:00:01��███████████████████████████▏ | 705.9MB 43.8MB/s eta 0:00:01�████████▎ | 708.1MB 108.8MB/s eta 0:00:01████████▍ | 711.7MB 104.5MB/s eta 0:00:01.0MB/s eta 0:00:01████████████████████████████▋ | 715.4MB 105.9MB/s eta 0:00:01�████████████████████████████▊ | 719.5MB 96.1MB/s eta 0:00:01███████▉ | 721.7MB 62.6MB/s eta 0:00:01�██████████████████████████ | 723.9MB 33.8MB/s eta 0:00:01�████████ | 726.3MB 84.3MB/s eta 0:00:01��█████████████████▏| 729.0MB 86.4MB/s eta 0:00:01�| 736.4MB 85.2MB/s eta 0:00:01�█████████████████▋| 738.7MB 100.0MB/s eta 0:00:01��████▊| 741.2MB 106.9MB/s eta 0:00:01�████▊| 743.6MB 110.3MB/s eta 0:00:01��██████████████████▉| 746.1MB 105.4MB/s eta 0:00:01████| 748.7MB 69.5MB/s eta 0:00:01
Installing collected packages: torch, torchvision
  Found existing installation: torch 1.0.0
    Uninstalling torch-1.0.0:
      Successfully uninstalled torch-1.0.0
  Found existing installation: torchvision 0.2.1
    Uninstalling torchvision-0.2.1:
      Successfully uninstalled torchvision-0.2.1
Successfully installed torch-1.2.0 torchvision-0.4.0
You are using pip version 9.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting protobuf==3.9.0
  Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/dc/0e/e7cdff89745986c984ba58e6ff6541bc5c388dd9ab9d7d312b3b1532584a/protobuf-3.9.0-cp36-cp36m-manylinux1_x86_64.whl (1.2MB)
    100% |████████████████████████████████| 1.2MB 65.7MB/s eta 0:00:01
Requirement already satisfied: six>=1.9 in /home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages (from protobuf==3.9.0)
Requirement already satisfied: setuptools in /home/ma-user/anaconda3/envs/Pytorch-1.0.0/lib/python3.6/site-packages (from protobuf==3.9.0)
Installing collected packages: protobuf
  Found existing installation: protobuf 3.5.1
    Uninstalling protobuf-3.5.1:
      Successfully uninstalled protobuf-3.5.1
Successfully installed protobuf-3.9.0
You are using pip version 9.0.1, however version 20.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

In [3]:

import tools._init_paths
%matplotlib inline
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import tensorboardX as tb
from datasets.factory import get_imdb
from model.train_val import get_training_roidb, train_net
from model.config import cfg, cfg_from_file, cfg_from_list, get_output_dir, get_output_tb_dir

In [4]:

import roi_data_layer.roidb as rdl_roidb
from roi_data_layer.layer import RoIDataLayer
import utils.timer
import pickle
import torch
import torch.optim as optim
from nets.vgg16 import vgg16
import numpy as np
import os
import sys
import glob
import time

神经网络搭建

模型训练超参设置

为了减少训练时间,我们在预训练模型的基础上进行训练。这里,我们使用VGG16作为FasterRCNN的主干网络。In [5]:

imdb_name = "voc_2007_trainval"
imdbval_name = "voc_2007_test"
# 使用的预训练模型位置
weight = "./data/imagenet_weights/vgg16.pth"
# 训练迭代次数
max_iters = 100
# cfg模型文件位置
cfg_file = './experiments/cfgs/vgg16.yml'
set_cfgs = None
if cfg_file is not None:
    cfg_from_file(cfg_file)
if set_cfgs is not None:
    cfg_from_list(set_cfgs)
print('Using config:')
print(cfg)

定义读取数据集函数

数据集的标注格式是PASCAL VOC格式。In [6]:

def combined_roidb(imdb_names):
    
    def get_roidb(imdb_name):
        # 加载数据集
        imdb = get_imdb(imdb_name)
        print('Loaded dataset `{:s}` for training'.format(imdb.name))
        # 使用ground truth作为数据集策略
        imdb.set_proposal_method(cfg.TRAIN.PROPOSAL_METHOD)
        print('Set proposal method: {:s}'.format(cfg.TRAIN.PROPOSAL_METHOD))
        roidb = get_training_roidb(imdb)
        return roidb
    roidbs = [get_roidb(s) for s in imdb_names.split('+')]
    roidb = roidbs[0]
    if len(roidbs) > 1:
        for r in roidbs[1:]:
            roidb.extend(r)
        tmp = get_imdb(imdb_names.split('+')[1])
        imdb = datasets.imdb.imdb(imdb_names, tmp.classes)
    else:
        imdb = get_imdb(imdb_names)
    return imdb, roidb

设置模型训练参数

In [7]:

np.random.seed(cfg.RNG_SEED)
# 加载训练数据集
imdb, roidb = combined_roidb(imdb_name)
print('{:d} roidb entries'.format(len(roidb)))
# 设置输出路径
output_dir = get_output_dir(imdb,None)
print('Output will be saved to `{:s}`'.format(output_dir))
# 设置日志保存路径
tb_dir = get_output_tb_dir(imdb, None)
print('TensorFlow summaries will be saved to `{:s}`'.format(tb_dir))
# 加载验证数据集
orgflip = cfg.TRAIN.USE_FLIPPED
cfg.TRAIN.USE_FLIPPED = False
_, valroidb = combined_roidb(imdbval_name)
print('{:d} validation roidb entries'.format(len(valroidb)))
cfg.TRAIN.USE_FLIPPED = orgflip
# 创建backbone网络
# 在案例中使用的是VGG16模型,可以尝试其他不同的模型结构,例如Resnet等
net = vgg16()
Using config:
{'TRAIN': {'LEARNING_RATE': 0.001, 'MOMENTUM': 0.9, 'WEIGHT_DECAY': 0.0001, 'GAMMA': 0.1, 'STEPSIZE': [30000], 'DISPLAY': 10, 'DOUBLE_BIAS': True, 'TRUNCATED': False, 'BIAS_DECAY': False, 'USE_GT': False, 'ASPECT_GROUPING': False, 'SNAPSHOT_KEPT': 3, 'SUMMARY_INTERVAL': 180, 'SCALES': [600], 'MAX_SIZE': 1000, 'IMS_PER_BATCH': 1, 'BATCH_SIZE': 128, 'FG_FRACTION': 0.25, 'FG_THRESH': 0.5, 'BG_THRESH_HI': 0.5, 'BG_THRESH_LO': 0.1, 'USE_FLIPPED': True, 'BBOX_REG': True, 'BBOX_THRESH': 0.5, 'SNAPSHOT_ITERS': 5000, 'SNAPSHOT_PREFIX': 'res101_faster_rcnn', 'BBOX_NORMALIZE_TARGETS': True, 'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': True, 'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0], 'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2], 'PROPOSAL_METHOD': 'gt', 'HAS_RPN': True, 'RPN_POSITIVE_OVERLAP': 0.7, 'RPN_NEGATIVE_OVERLAP': 0.3, 'RPN_CLOBBER_POSITIVES': False, 'RPN_FG_FRACTION': 0.5, 'RPN_BATCHSIZE': 256, 'RPN_NMS_THRESH': 0.7, 'RPN_PRE_NMS_TOP_N': 12000, 'RPN_POST_NMS_TOP_N': 2000, 'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0], 'RPN_POSITIVE_WEIGHT': -1.0, 'USE_ALL_GT': True}, 'TEST': {'SCALES': [600], 'MAX_SIZE': 1000, 'NMS': 0.3, 'SVM': False, 'BBOX_REG': True, 'HAS_RPN': False, 'PROPOSAL_METHOD': 'gt', 'RPN_NMS_THRESH': 0.7, 'RPN_PRE_NMS_TOP_N': 6000, 'RPN_POST_NMS_TOP_N': 300, 'MODE': 'nms', 'RPN_TOP_N': 5000}, 'RESNET': {'MAX_POOL': False, 'FIXED_BLOCKS': 1}, 'MOBILENET': {'REGU_DEPTH': False, 'FIXED_LAYERS': 5, 'WEIGHT_DECAY': 4e-05, 'DEPTH_MULTIPLIER': 1.0}, 'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]), 'RNG_SEED': 3, 'ROOT_DIR': '/home/ma-user/work', 'DATA_DIR': '/home/ma-user/work/data', 'MATLAB': 'matlab', 'EXP_DIR': 'default', 'USE_GPU_NMS': True, 'POOLING_MODE': 'align', 'POOLING_SIZE': 7, 'ANCHOR_SCALES': [8, 16, 32], 'ANCHOR_RATIOS': [0.5, 1, 2], 'RPN_CHANNELS': 512}
Loaded dataset `voc_2007_trainval` for training
Set proposal method: gt
Appending horizontally-flipped training examples...
wrote gt roidb to /home/ma-user/work/data/cache/voc_2007_trainval_gt_roidb.pkl
done
Preparing training data...
done
10022 roidb entries
Output will be saved to `/home/ma-user/work/output/default/voc_2007_trainval/default`
TensorFlow summaries will be saved to `/home/ma-user/work/tensorboard/default/voc_2007_trainval/default`
Loaded dataset `voc_2007_test` for training
Set proposal method: gt
Preparing training data...
wrote gt roidb to /home/ma-user/work/data/cache/voc_2007_test_gt_roidb.pkl
done
4952 validation roidb entries

In [8]:

from model.train_val import filter_roidb, SolverWrapper
# 对ROI进行筛选,将无效的ROI数据筛选掉
roidb = filter_roidb(roidb)
valroidb = filter_roidb(valroidb)
sw = SolverWrapper(
    net,
    imdb,
    roidb,
    valroidb,
    output_dir,
    tb_dir,
    pretrained_model=weight)
print('Solving...')
Filtered 0 roidb entries: 10022 -> 10022
Filtered 0 roidb entries: 4952 -> 4952
Solving...

In [9]:

# 显示所有模型属性
sw.__dict__.keys()

Out[9]:

dict_keys(['net', 'imdb', 'roidb', 'valroidb', 'output_dir', 'tbdir', 'tbvaldir', 'pretrained_model'])

In [10]:

# sw.net为主干网络
print(sw.net)
vgg16()

定义神经网络结构

使用PyTorch搭建神经网络。

部分实现细节可以去相应的文件夹查看源码。In [11]:

# 构建网络结构,模型加入ROI数据层
sw.data_layer = RoIDataLayer(sw.roidb, sw.imdb.num_classes)
sw.data_layer_val = RoIDataLayer(sw.valroidb, sw.imdb.num_classes, random=True)
# 构建网络结构,在VGG16基础上加入ROI和Classifier部分
lr, train_op = sw.construct_graph()
# 加载之前的snapshot
lsf, nfiles, sfiles = sw.find_previous()
# snapshot 为训练提供了断点训练,如果有snapshot将加载进来,继续训练
if lsf == 0:
    lr, last_snapshot_iter, stepsizes, np_paths, ss_paths = sw.initialize()
else:
    lr, last_snapshot_iter, stepsizes, np_paths, ss_paths = sw.restore(str(sfiles[-1]), str(nfiles[-1]))
iter = last_snapshot_iter + 1
last_summary_time = time.time()
# 在之前的训练基础上继续进行训练
stepsizes.append(max_iters)
stepsizes.reverse()
next_stepsize = stepsizes.pop()
# 将net切换成训练模式
print("网络结构:")
sw.net.train()
sw.net.to(sw.net._device)
Loading initial model weights from ./data/imagenet_weights/vgg16.pth
Loaded.
网络结构:

Out[11]:

vgg16(
  (vgg): VGG(
    (features): Sequential(
      (0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): ReLU(inplace=True)
      (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (3): ReLU(inplace=True)
      (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
      (5): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (6): ReLU(inplace=True)
      (7): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (8): ReLU(inplace=True)
      (9): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
      (10): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (11): ReLU(inplace=True)
      (12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (13): ReLU(inplace=True)
      (14): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (15): ReLU(inplace=True)
      (16): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
      (17): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (18): ReLU(inplace=True)
      (19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (20): ReLU(inplace=True)
      (21): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (22): ReLU(inplace=True)
      (23): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
      (24): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (25): ReLU(inplace=True)
      (26): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (27): ReLU(inplace=True)
      (28): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (29): ReLU(inplace=True)
      (30): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
    )
    (avgpool): AdaptiveAvgPool2d(output_size=(7, 7))
    (classifier): Sequential(
      (0): Linear(in_features=25088, out_features=4096, bias=True)
      (1): ReLU(inplace=True)
      (2): Dropout(p=0.5, inplace=False)
      (3): Linear(in_features=4096, out_features=4096, bias=True)
      (4): ReLU(inplace=True)
      (5): Dropout(p=0.5, inplace=False)
    )
  )
  (rpn_net): Conv2d(512, 512, kernel_size=[3, 3], stride=(1, 1), padding=(1, 1))
  (rpn_cls_score_net): Conv2d(512, 18, kernel_size=[1, 1], stride=(1, 1))
  (rpn_bbox_pred_net): Conv2d(512, 36, kernel_size=[1, 1], stride=(1, 1))
  (cls_score_net): Linear(in_features=4096, out_features=21, bias=True)
  (bbox_pred_net): Linear(in_features=4096, out_features=84, bias=True)
)

开始训练

In [12]:

while iter < max_iters + 1:
    if iter == next_stepsize + 1:
        # 加入snapshot节点
        sw.snapshot(iter)
        lr *= cfg.TRAIN.GAMMA
        scale_lr(sw.optimizer, cfg.TRAIN.GAMMA)
        next_stepsize = stepsizes.pop()
    utils.timer.timer.tic()
    # 数据通过ROI数据层,进行前向计算
    blobs = sw.data_layer.forward()
    now = time.time()
    if iter == 1 or now - last_summary_time > cfg.TRAIN.SUMMARY_INTERVAL:
        # 计算loss函数
        # 根据loss函数对模型进行训练
        rpn_loss_cls, rpn_loss_box, loss_cls, loss_box, total_loss, summary = \
          sw.net.train_step_with_summary(blobs, sw.optimizer)
        for _sum in summary:
            sw.writer.add_summary(_sum, float(iter))
        # 进行数据层验证计算
        blobs_val = sw.data_layer_val.forward()
        summary_val = sw.net.get_summary(blobs_val)
        for _sum in summary_val:
            sw.valwriter.add_summary(_sum, float(iter))
        last_summary_time = now
    else:
        rpn_loss_cls, rpn_loss_box, loss_cls, loss_box, total_loss = \
          sw.net.train_step(blobs, sw.optimizer)
    utils.timer.timer.toc()
    if iter % (cfg.TRAIN.DISPLAY) == 0:
        print('iter: %d / %d, total loss: %.6f\n >>> rpn_loss_cls: %.6f\n '
              '>>> rpn_loss_box: %.6f\n >>> loss_cls: %.6f\n >>> loss_box: %.6f\n >>> lr: %f' % \
              (iter, max_iters, total_loss, rpn_loss_cls, rpn_loss_box, loss_cls, loss_box, lr))
        print('speed: {:.3f}s / iter'.format(
            utils.timer.timer.average_time()))
    # 进行snapshot存储
    if iter % cfg.TRAIN.SNAPSHOT_ITERS == 0:
        last_snapshot_iter = iter
        ss_path, np_path = sw.snapshot(iter)
        np_paths.append(np_path)
        ss_paths.append(ss_path)
        # 删掉多余的snapshot
        if len(np_paths) > cfg.TRAIN.SNAPSHOT_KEPT:
            sw.remove_snapshot(np_paths, ss_paths)
    iter += 1
if last_snapshot_iter != iter - 1:
    sw.snapshot(iter - 1)
sw.writer.close()
sw.valwriter.close()
iter: 10 / 100, total loss: 1.025015
 >>> rpn_loss_cls: 0.387036
 >>> rpn_loss_box: 0.004538
 >>> loss_cls: 0.366100
 >>> loss_box: 0.267340
 >>> lr: 0.001000
speed: 0.666s / iter
iter: 20 / 100, total loss: 2.173192
 >>> rpn_loss_cls: 0.363755
 >>> rpn_loss_box: 0.094425
 >>> loss_cls: 1.005797
 >>> loss_box: 0.709214
 >>> lr: 0.001000
speed: 0.396s / iter
iter: 30 / 100, total loss: 1.212159
 >>> rpn_loss_cls: 0.144665
 >>> rpn_loss_box: 0.043567
 >>> loss_cls: 0.626252
 >>> loss_box: 0.397675
 >>> lr: 0.001000
speed: 0.305s / iter
iter: 40 / 100, total loss: 0.848530
 >>> rpn_loss_cls: 0.756678
 >>> rpn_loss_box: 0.079122
 >>> loss_cls: 0.012730
 >>> loss_box: 0.000000
 >>> lr: 0.001000
speed: 0.259s / iter
iter: 50 / 100, total loss: 1.012320
 >>> rpn_loss_cls: 0.318012
 >>> rpn_loss_box: 0.029802
 >>> loss_cls: 0.507314
 >>> loss_box: 0.157193
 >>> lr: 0.001000
speed: 0.232s / iter
iter: 60 / 100, total loss: 0.994835
 >>> rpn_loss_cls: 0.353266
 >>> rpn_loss_box: 0.043218
 >>> loss_cls: 0.404328
 >>> loss_box: 0.194023
 >>> lr: 0.001000
speed: 0.214s / iter
iter: 70 / 100, total loss: 0.686826
 >>> rpn_loss_cls: 0.259330
 >>> rpn_loss_box: 0.006708
 >>> loss_cls: 0.420788
 >>> loss_box: 0.000000
 >>> lr: 0.001000
speed: 0.200s / iter
iter: 80 / 100, total loss: 1.976414
 >>> rpn_loss_cls: 0.153041
 >>> rpn_loss_box: 0.103065
 >>> loss_cls: 1.080667
 >>> loss_box: 0.639641
 >>> lr: 0.001000
speed: 0.190s / iter
iter: 90 / 100, total loss: 2.763836
 >>> rpn_loss_cls: 0.700790
 >>> rpn_loss_box: 0.201123
 >>> loss_cls: 1.155626
 >>> loss_box: 0.706297
 >>> lr: 0.001000
speed: 0.182s / iter
iter: 100 / 100, total loss: 1.721430
 >>> rpn_loss_cls: 0.281938
 >>> rpn_loss_box: 0.036765
 >>> loss_cls: 0.779848
 >>> loss_box: 0.622879
 >>> lr: 0.001000
speed: 0.176s / iter
Wrote snapshot to: /home/ma-user/work/output/default/voc_2007_trainval/default/res101_faster_rcnn_iter_100.pth

测试部分

在这部分中,我们利用训练得到的模型进行推理测试。In [13]:

%matplotlib inline
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
# 将路径转入lib
import tools._init_paths
from model.config import cfg
from model.test import im_detect
from torchvision.ops import nms
from utils.timer import Timer
import matplotlib.pyplot as plt
import numpy as np
import os, cv2
import argparse
from nets.vgg16 import vgg16
from nets.resnet_v1 import resnetv1
from model.bbox_transform import clip_boxes, bbox_transform_inv
import torch

参数定义

In [14]:

# PASCAL VOC类别设置
CLASSES = ('__background__',
           'aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor')
# 网络模型文件名定义
NETS = {'vgg16': ('vgg16_faster_rcnn_iter_%d.pth',),'res101': ('res101_faster_rcnn_iter_%d.pth',)}
# 数据集文件名定义
DATASETS= {'pascal_voc': ('voc_2007_trainval',),'pascal_voc_0712': ('voc_2007_trainval+voc_2012_trainval',)}

结果绘制

将预测的标签和边界框绘制在原图上。In [15]:

def vis_detections(im, class_dets, thresh=0.5):
    """Draw detected bounding boxes."""
    im = im[:, :, (2, 1, 0)]
    fig, ax = plt.subplots(figsize=(12, 12))
    ax.imshow(im, aspect='equal')
    for class_name in class_dets:
        dets = class_dets[class_name]
        inds = np.where(dets[:, -1] >= thresh)[0]
        if len(inds) == 0:
            continue
        
        for i in inds:
            bbox = dets[i, :4]
            score = dets[i, -1]
            ax.add_patch(
                plt.Rectangle((bbox[0], bbox[1]),
                              bbox[2] - bbox[0],
                              bbox[3] - bbox[1], fill=False,
                              edgecolor='red', linewidth=3.5)
                )
            ax.text(bbox[0], bbox[1] - 2,
                    '{:s} {:.3f}'.format(class_name, score),
                    bbox=dict(facecolor='blue', alpha=0.5),
                    fontsize=14, color='white')
        plt.axis('off')
        plt.tight_layout()
        plt.draw()

准备测试图片

我们将测试图片传到test文件夹下,我们准备了两张图片进行测试,大家也可以通过notebook的upload按钮上传自己的测试数据。注意,测试数据需要是图片,并且放在test文件夹下。In [16]:

test_file = "./test"

模型推理

这里我们加载一个预先训练好的模型,也可以选择案例中训练的模型。In [17]:

import cv2
from utils.timer import Timer
from model.test import im_detect
from torchvision.ops import nms
cfg.TEST.HAS_RPN = True  # Use RPN for proposals
# 模型存储位置
# 这里我们加载一个已经训练110000迭代之后的模型,可以选择自己的训练模型位置
saved_model = "./models/vgg16-voc0712/vgg16_faster_rcnn_iter_110000.pth"
print('trying to load weights from ', saved_model)
# 加载backbone
net = vgg16()
# 构建网络
net.create_architecture(21, tag='default', anchor_scales=[8, 16, 32])
# 加载权重文件
net.load_state_dict(torch.load(saved_model, map_location=lambda storage, loc: storage))
net.eval()
# 选择推理设备
net.to(net._device)
print('Loaded network {:s}'.format(saved_model))
for file in os.listdir(test_file):
    if file.startswith("._") == False:
        file_path = os.path.join(test_file, file)
        print(file_path)
        # 打开测试图片文件
        im = cv2.imread(file_path)
        # 定义计时器
        timer = Timer()
        timer.tic()
        # 检测得到图片ROI
        scores, boxes = im_detect(net, im)
        print(scores.shape, boxes.shape)
        timer.toc()
        print('Detection took {:.3f}s for {:d} object proposals'.format(timer.total_time(), boxes.shape[0]))
        # 定义阈值
        CONF_THRESH = 0.7
        NMS_THRESH = 0.3
        cls_dets = {}
        # NMS 非极大值抑制操作,过滤边界框
        for cls_ind, cls in enumerate(CLASSES[1:]):
            cls_ind += 1 # 跳过 background
            cls_boxes = boxes[:, 4*cls_ind:4*(cls_ind + 1)]
            cls_scores = scores[:, cls_ind]
            dets = np.hstack((cls_boxes,
                              cls_scores[:, np.newaxis])).astype(np.float32)
            keep = nms(torch.from_numpy(cls_boxes), torch.from_numpy(cls_scores), NMS_THRESH)
            dets = dets[keep.numpy(), :]
            if len(dets) > 0:
                if cls in cls_dets:
                    cls_dets[cls] = np.vstack([cls_dets[cls], dets]) 
                else:
                    cls_dets[cls] = dets
        vis_detections(im, cls_dets, thresh=CONF_THRESH)
        plt.show()
trying to load weights from  ./models/vgg16-voc0712/vgg16_faster_rcnn_iter_110000.pth
Loaded network ./models/vgg16-voc0712/vgg16_faster_rcnn_iter_110000.pth
./test/test_image_0.jpg
(300, 21) (300, 84)
Detection took 0.062s for 300 object proposals
./test/test_image_1.jpg
(300, 21) (300, 84)
Detection took 0.054s for 300 object proposals
© 版权声明
THE END
一发入魂
点赞0打赏
分享
评论 抢沙发

请登录后发表评论