观前提示：请将页面调成白色主题（点击右上角水彩调色盘）
物体检测是计算机视觉中的一个重要的研究领域，在人流检测，行人跟踪，自动驾驶，医学影像等领域有着广泛的应用。不同于简单的图像分类，物体检测旨在对图像中的目标进行精确识别，包括物体的位置和分类，因此能够应用于更多高层视觉处理的场景。例如在自动驾驶领域，需要辨识摄像头拍摄的图像中的车辆、行人、交通指示牌及其位置，以便进一步根据这些数据决定驾驶策略。本期学习案例，我们将聚焦于YOLO算法，YOLO(You Only Look Once)是一种one-stage物体检测算法。
进入ModelArts
点击如下链接：https://www.huaweicloud.com/product/modelarts.html ， 进入ModelArts主页。点击“立即使用”按钮，输入用户名和密码登录，进入ModelArts使用页面。
创建ModelArts notebook
下面，我们在ModelArts中创建一个notebook开发环境，ModelArts notebook提供网页版的Python开发环境，可以方便的编写、运行代码，并查看运行结果。
第一步：在ModelArts服务主界面依次点击“开发环境”、“创建”
第二步：填写notebook所需的参数：
| 项目 | 建议填写方式 |
|---|---|
| 名称 | 自定义环境名称 |
| 工作环境 | Python3 |
| 资源池 | 选择”公共资源池”即可 |
| 类型 | GPU |
| 规格 | GPU:1*p100, CPU:8核64GiB |
| 存储配置 | EVS |
| 磁盘规格 | 5GB |
第三步：配置好notebook参数后，点击下一步，进入notebook信息预览。确认无误后，点击“立即创建”
第四步：创建完成后，返回开发环境主界面，等待Notebook创建完毕后，打开Notebook，进行下一步操作。
在ModelArts中创建开发环境
接下来，我们创建一个实际的开发环境，用于后续的实验步骤。
第一步：点击下图所示的“打开”按钮，进入刚刚创建的Notebook
第二步：创建一个Python3环境的的Notebook。点击右上角的”New”，然后选择TensorFlow 1.13.1开发环境。
第三步：点击左上方的文件名”Untitled”，并输入一个与本实验相关的名称，如”yolo_v3″
在Notebook中编写并执行代码
在Notebook中，我们输入一个简单的打印语句，然后点击上方的运行按钮，可以查看语句执行的结果：
开发环境准备好啦，接下来可以愉快地写代码啦！
数据和代码下载
运行下面代码，进行数据和代码的下载和解压
本案例使用coco数据，共80个类别。 In [1]:
from modelarts.session import Session
sess = Session()
if sess.region_name == 'cn-north-1':
bucket_path="modelarts-labs/notebook/DL_object_detection_yolo/yolov3.tar.gz"
elif sess.region_name == 'cn-north-4':
bucket_path="modelarts-labs-bj4/notebook/DL_object_detection_yolo/yolov3.tar.gz"
else:
print("请更换地区到北京一或北京四")
sess.download_data(bucket_path=bucket_path, path="./yolov3.tar.gz")
# 解压文件
!tar -xf ./yolov3.tar.gz
# 清理压缩包
!rm -r ./yolov3.tar.gz
Successfully download file modelarts-labs-bj4/notebook/DL_object_detection_yolo/yolov3.tar.gz from OBS to local ./yolov3.tar.gz
准备数据
文件路径定义 In [2]:
from train import get_classes, get_anchors
# 数据文件路径
data_path = "./coco/coco_data"
# coco类型定义文件存储位置
classes_path = './model_data/coco_classes.txt'
# coco数据anchor值文件存储位置
anchors_path = './model_data/yolo_anchors.txt'
# coco数据标注信息文件存储位置
annotation_path = './coco/coco_train.txt'
# 预训练权重文件存储位置
weights_path = "./model_data/yolo.h5"
# 模型文件存储位置
save_path = "./result/models/"
classes = get_classes(classes_path)
anchors = get_anchors(anchors_path)
# 获取类型数量和anchor数量变量
num_classes = len(classes)
num_anchors = len(anchors)
Using TensorFlow backend.
读取标注数据 In [3]:
import numpy as np
# 训练集与验证集划分比例
val_split = 0.1
with open(annotation_path) as f:
lines = f.readlines()
np.random.seed(10101)
np.random.shuffle(lines)
np.random.seed(None)
num_val = int(len(lines)*val_split)
num_train = len(lines) - num_val
数据读取函数，构建数据生成器。每次读取一个批次的数据至内存训练，并做数据增强。 In [4]:
def data_generator(annotation_lines, batch_size, input_shape, data_path,anchors, num_classes):
n = len(annotation_lines)
i = 0
while True:
image_data = []
box_data = []
for b in range(batch_size):
if i==0:
np.random.shuffle(annotation_lines)
image, box = get_random_data(annotation_lines[i], input_shape, data_path,random=True) # 随机挑选一个批次的数据
image_data.append(image)
box_data.append(box)
i = (i+1) % n
image_data = np.array(image_data)
box_data = np.array(box_data)
y_true = preprocess_true_boxes(box_data, input_shape, anchors, num_classes) # 对标注框预处理，过滤异常标注框
yield [image_data, *y_true], np.zeros(batch_size)
def data_generator_wrapper(annotation_lines, batch_size, input_shape, data_path,anchors, num_classes):
n = len(annotation_lines)
if n==0 or batch_size<=0: return None
return data_generator(annotation_lines, batch_size, input_shape, data_path,anchors, num_classes)
模型训练
本案例使用Keras深度学习框架搭建YOLOv3神经网络。
可以进入相应的文件夹路径查看源码实现。
构建神经网络
可以在./yolo3/model.py文件中查看细节 In [5]:
import keras.backend as K
from yolo3.model import preprocess_true_boxes, yolo_body, yolo_loss
from keras.layers import Input, Lambda
from keras.models import Model
# 初始化session
K.clear_session()
# 图像输入尺寸
input_shape = (416, 416)
image_input = Input(shape=(None, None, 3))
h, w = input_shape
# 设置多尺度检测的下采样尺寸
y_true = [Input(shape=(h//{0:32, 1:16, 2:8}[l], w//{0:32, 1:16, 2:8}[l], num_anchors//3, num_classes+5))
for l in range(3)]
# 构建YOLO模型结构
model_body = yolo_body(image_input, num_anchors//3, num_classes)
# 将YOLO权重文件加载进来，如果希望不加载预训练权重，从头开始训练的话，可以删除这句代码
model_body.load_weights(weights_path, by_name=True, skip_mismatch=True)
# 定义YOLO损失函数
model_loss = Lambda(yolo_loss, output_shape=(1,), name='yolo_loss',
arguments={'anchors': anchors, 'num_classes': num_classes, 'ignore_thresh': 0.5})([*model_body.output, *y_true])
# 构建Model，为训练做准备
model = Model([model_body.input, *y_true], model_loss)
WARNING:tensorflow:From /home/ma-user/anaconda3/envs/TensorFlow-1.13.1/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer.
In [6]:
# 打印模型各层结构
model.summary()
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) (None, None, None, 3 0
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, None, None, 3 864 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, None, None, 3 128 conv2d_1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, None, None, 3 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
zero_padding2d_1 (ZeroPadding2D (None, None, None, 3 0 leaky_re_lu_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, None, None, 6 18432 zero_padding2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, None, None, 6 256 conv2d_2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU) (None, None, None, 6 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, None, None, 3 2048 leaky_re_lu_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, None, None, 3 128 conv2d_3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU) (None, None, None, 3 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, None, None, 6 18432 leaky_re_lu_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, None, None, 6 256 conv2d_4[0][0]
__________________________________________________________________________________________________
leaky_re_lu_4 (LeakyReLU) (None, None, None, 6 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
add_1 (Add) (None, None, None, 6 0 leaky_re_lu_2[0][0]
leaky_re_lu_4[0][0]
__________________________________________________________________________________________________
zero_padding2d_2 (ZeroPadding2D (None, None, None, 6 0 add_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, None, None, 1 73728 zero_padding2d_2[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, None, 1 512 conv2d_5[0][0]
__________________________________________________________________________________________________
leaky_re_lu_5 (LeakyReLU) (None, None, None, 1 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, None, None, 6 8192 leaky_re_lu_5[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, None, 6 256 conv2d_6[0][0]
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU) (None, None, None, 6 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, None, None, 1 73728 leaky_re_lu_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, None, 1 512 conv2d_7[0][0]
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU) (None, None, None, 1 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
add_2 (Add) (None, None, None, 1 0 leaky_re_lu_5[0][0]
leaky_re_lu_7[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, None, None, 6 8192 add_2[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, None, 6 256 conv2d_8[0][0]
__________________________________________________________________________________________________
leaky_re_lu_8 (LeakyReLU) (None, None, None, 6 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, None, None, 1 73728 leaky_re_lu_8[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, None, 1 512 conv2d_9[0][0]
__________________________________________________________________________________________________
leaky_re_lu_9 (LeakyReLU) (None, None, None, 1 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
add_3 (Add) (None, None, None, 1 0 add_2[0][0]
leaky_re_lu_9[0][0]
__________________________________________________________________________________________________
zero_padding2d_3 (ZeroPadding2D (None, None, None, 1 0 add_3[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, None, None, 2 294912 zero_padding2d_3[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, None, 2 1024 conv2d_10[0][0]
__________________________________________________________________________________________________
leaky_re_lu_10 (LeakyReLU) (None, None, None, 2 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, None, None, 1 32768 leaky_re_lu_10[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, None, 1 512 conv2d_11[0][0]
__________________________________________________________________________________________________
leaky_re_lu_11 (LeakyReLU) (None, None, None, 1 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_11[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, None, 2 1024 conv2d_12[0][0]
__________________________________________________________________________________________________
leaky_re_lu_12 (LeakyReLU) (None, None, None, 2 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
add_4 (Add) (None, None, None, 2 0 leaky_re_lu_10[0][0]
leaky_re_lu_12[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, None, None, 1 32768 add_4[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, None, 1 512 conv2d_13[0][0]
__________________________________________________________________________________________________
leaky_re_lu_13 (LeakyReLU) (None, None, None, 1 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_13[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, None, 2 1024 conv2d_14[0][0]
__________________________________________________________________________________________________
leaky_re_lu_14 (LeakyReLU) (None, None, None, 2 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
add_5 (Add) (None, None, None, 2 0 add_4[0][0]
leaky_re_lu_14[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, None, None, 1 32768 add_5[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, None, None, 1 512 conv2d_15[0][0]
__________________________________________________________________________________________________
leaky_re_lu_15 (LeakyReLU) (None, None, None, 1 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_15[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, None, None, 2 1024 conv2d_16[0][0]
__________________________________________________________________________________________________
leaky_re_lu_16 (LeakyReLU) (None, None, None, 2 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
add_6 (Add) (None, None, None, 2 0 add_5[0][0]
leaky_re_lu_16[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, None, None, 1 32768 add_6[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, None, None, 1 512 conv2d_17[0][0]
__________________________________________________________________________________________________
leaky_re_lu_17 (LeakyReLU) (None, None, None, 1 0 batch_normalization_17[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_17[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, None, None, 2 1024 conv2d_18[0][0]
__________________________________________________________________________________________________
leaky_re_lu_18 (LeakyReLU) (None, None, None, 2 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
add_7 (Add) (None, None, None, 2 0 add_6[0][0]
leaky_re_lu_18[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, None, None, 1 32768 add_7[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, None, None, 1 512 conv2d_19[0][0]
__________________________________________________________________________________________________
leaky_re_lu_19 (LeakyReLU) (None, None, None, 1 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_19[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, None, None, 2 1024 conv2d_20[0][0]
__________________________________________________________________________________________________
leaky_re_lu_20 (LeakyReLU) (None, None, None, 2 0 batch_normalization_20[0][0]
__________________________________________________________________________________________________
add_8 (Add) (None, None, None, 2 0 add_7[0][0]
leaky_re_lu_20[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, None, None, 1 32768 add_8[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, None, None, 1 512 conv2d_21[0][0]
__________________________________________________________________________________________________
leaky_re_lu_21 (LeakyReLU) (None, None, None, 1 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_21[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, None, None, 2 1024 conv2d_22[0][0]
__________________________________________________________________________________________________
leaky_re_lu_22 (LeakyReLU) (None, None, None, 2 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
add_9 (Add) (None, None, None, 2 0 add_8[0][0]
leaky_re_lu_22[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, None, None, 1 32768 add_9[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, None, None, 1 512 conv2d_23[0][0]
__________________________________________________________________________________________________
leaky_re_lu_23 (LeakyReLU) (None, None, None, 1 0 batch_normalization_23[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_23[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, None, None, 2 1024 conv2d_24[0][0]
__________________________________________________________________________________________________
leaky_re_lu_24 (LeakyReLU) (None, None, None, 2 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
add_10 (Add) (None, None, None, 2 0 add_9[0][0]
leaky_re_lu_24[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, None, None, 1 32768 add_10[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, None, None, 1 512 conv2d_25[0][0]
__________________________________________________________________________________________________
leaky_re_lu_25 (LeakyReLU) (None, None, None, 1 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_25[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, None, None, 2 1024 conv2d_26[0][0]
__________________________________________________________________________________________________
leaky_re_lu_26 (LeakyReLU) (None, None, None, 2 0 batch_normalization_26[0][0]
__________________________________________________________________________________________________
add_11 (Add) (None, None, None, 2 0 add_10[0][0]
leaky_re_lu_26[0][0]
__________________________________________________________________________________________________
zero_padding2d_4 (ZeroPadding2D (None, None, None, 2 0 add_11[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, None, None, 5 1179648 zero_padding2d_4[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, None, None, 5 2048 conv2d_27[0][0]
__________________________________________________________________________________________________
leaky_re_lu_27 (LeakyReLU) (None, None, None, 5 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_27[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, None, None, 2 1024 conv2d_28[0][0]
__________________________________________________________________________________________________
leaky_re_lu_28 (LeakyReLU) (None, None, None, 2 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_28[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, None, None, 5 2048 conv2d_29[0][0]
__________________________________________________________________________________________________
leaky_re_lu_29 (LeakyReLU) (None, None, None, 5 0 batch_normalization_29[0][0]
__________________________________________________________________________________________________
add_12 (Add) (None, None, None, 5 0 leaky_re_lu_27[0][0]
leaky_re_lu_29[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, None, None, 2 131072 add_12[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, None, None, 2 1024 conv2d_30[0][0]
__________________________________________________________________________________________________
leaky_re_lu_30 (LeakyReLU) (None, None, None, 2 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_30[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, None, None, 5 2048 conv2d_31[0][0]
__________________________________________________________________________________________________
leaky_re_lu_31 (LeakyReLU) (None, None, None, 5 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
add_13 (Add) (None, None, None, 5 0 add_12[0][0]
leaky_re_lu_31[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, None, None, 2 131072 add_13[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, None, None, 2 1024 conv2d_32[0][0]
__________________________________________________________________________________________________
leaky_re_lu_32 (LeakyReLU) (None, None, None, 2 0 batch_normalization_32[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_32[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, None, None, 5 2048 conv2d_33[0][0]
__________________________________________________________________________________________________
leaky_re_lu_33 (LeakyReLU) (None, None, None, 5 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
add_14 (Add) (None, None, None, 5 0 add_13[0][0]
leaky_re_lu_33[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, None, None, 2 131072 add_14[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, None, None, 2 1024 conv2d_34[0][0]
__________________________________________________________________________________________________
leaky_re_lu_34 (LeakyReLU) (None, None, None, 2 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_34[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, None, None, 5 2048 conv2d_35[0][0]
__________________________________________________________________________________________________
leaky_re_lu_35 (LeakyReLU) (None, None, None, 5 0 batch_normalization_35[0][0]
__________________________________________________________________________________________________
add_15 (Add) (None, None, None, 5 0 add_14[0][0]
leaky_re_lu_35[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, None, None, 2 131072 add_15[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, None, None, 2 1024 conv2d_36[0][0]
__________________________________________________________________________________________________
leaky_re_lu_36 (LeakyReLU) (None, None, None, 2 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_36[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, None, None, 5 2048 conv2d_37[0][0]
__________________________________________________________________________________________________
leaky_re_lu_37 (LeakyReLU) (None, None, None, 5 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
add_16 (Add) (None, None, None, 5 0 add_15[0][0]
leaky_re_lu_37[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, None, None, 2 131072 add_16[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, None, None, 2 1024 conv2d_38[0][0]
__________________________________________________________________________________________________
leaky_re_lu_38 (LeakyReLU) (None, None, None, 2 0 batch_normalization_38[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_38[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, None, None, 5 2048 conv2d_39[0][0]
__________________________________________________________________________________________________
leaky_re_lu_39 (LeakyReLU) (None, None, None, 5 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
add_17 (Add) (None, None, None, 5 0 add_16[0][0]
leaky_re_lu_39[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, None, None, 2 131072 add_17[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, None, None, 2 1024 conv2d_40[0][0]
__________________________________________________________________________________________________
leaky_re_lu_40 (LeakyReLU) (None, None, None, 2 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_40[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, None, None, 5 2048 conv2d_41[0][0]
__________________________________________________________________________________________________
leaky_re_lu_41 (LeakyReLU) (None, None, None, 5 0 batch_normalization_41[0][0]
__________________________________________________________________________________________________
add_18 (Add) (None, None, None, 5 0 add_17[0][0]
leaky_re_lu_41[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, None, None, 2 131072 add_18[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, None, None, 2 1024 conv2d_42[0][0]
__________________________________________________________________________________________________
leaky_re_lu_42 (LeakyReLU) (None, None, None, 2 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_42[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, None, None, 5 2048 conv2d_43[0][0]
__________________________________________________________________________________________________
leaky_re_lu_43 (LeakyReLU) (None, None, None, 5 0 batch_normalization_43[0][0]
__________________________________________________________________________________________________
add_19 (Add) (None, None, None, 5 0 add_18[0][0]
leaky_re_lu_43[0][0]
__________________________________________________________________________________________________
zero_padding2d_5 (ZeroPadding2D (None, None, None, 5 0 add_19[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, None, None, 1 4718592 zero_padding2d_5[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, None, None, 1 4096 conv2d_44[0][0]
__________________________________________________________________________________________________
leaky_re_lu_44 (LeakyReLU) (None, None, None, 1 0 batch_normalization_44[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, None, None, 5 524288 leaky_re_lu_44[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, None, None, 5 2048 conv2d_45[0][0]
__________________________________________________________________________________________________
leaky_re_lu_45 (LeakyReLU) (None, None, None, 5 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_45[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, None, None, 1 4096 conv2d_46[0][0]
__________________________________________________________________________________________________
leaky_re_lu_46 (LeakyReLU) (None, None, None, 1 0 batch_normalization_46[0][0]
__________________________________________________________________________________________________
add_20 (Add) (None, None, None, 1 0 leaky_re_lu_44[0][0]
leaky_re_lu_46[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, None, None, 5 524288 add_20[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, None, None, 5 2048 conv2d_47[0][0]
__________________________________________________________________________________________________
leaky_re_lu_47 (LeakyReLU) (None, None, None, 5 0 batch_normalization_47[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_47[0][0]
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, None, None, 1 4096 conv2d_48[0][0]
__________________________________________________________________________________________________
leaky_re_lu_48 (LeakyReLU) (None, None, None, 1 0 batch_normalization_48[0][0]
__________________________________________________________________________________________________
add_21 (Add) (None, None, None, 1 0 add_20[0][0]
leaky_re_lu_48[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, None, None, 5 524288 add_21[0][0]
__________________________________________________________________________________________________
batch_normalization_49 (BatchNo (None, None, None, 5 2048 conv2d_49[0][0]
__________________________________________________________________________________________________
leaky_re_lu_49 (LeakyReLU) (None, None, None, 5 0 batch_normalization_49[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_49[0][0]
__________________________________________________________________________________________________
batch_normalization_50 (BatchNo (None, None, None, 1 4096 conv2d_50[0][0]
__________________________________________________________________________________________________
leaky_re_lu_50 (LeakyReLU) (None, None, None, 1 0 batch_normalization_50[0][0]
__________________________________________________________________________________________________
add_22 (Add) (None, None, None, 1 0 add_21[0][0]
leaky_re_lu_50[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, None, None, 5 524288 add_22[0][0]
__________________________________________________________________________________________________
batch_normalization_51 (BatchNo (None, None, None, 5 2048 conv2d_51[0][0]
__________________________________________________________________________________________________
leaky_re_lu_51 (LeakyReLU) (None, None, None, 5 0 batch_normalization_51[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_51[0][0]
__________________________________________________________________________________________________
batch_normalization_52 (BatchNo (None, None, None, 1 4096 conv2d_52[0][0]
__________________________________________________________________________________________________
leaky_re_lu_52 (LeakyReLU) (None, None, None, 1 0 batch_normalization_52[0][0]
__________________________________________________________________________________________________
add_23 (Add) (None, None, None, 1 0 add_22[0][0]
leaky_re_lu_52[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, None, None, 5 524288 add_23[0][0]
__________________________________________________________________________________________________
batch_normalization_53 (BatchNo (None, None, None, 5 2048 conv2d_53[0][0]
__________________________________________________________________________________________________
leaky_re_lu_53 (LeakyReLU) (None, None, None, 5 0 batch_normalization_53[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_53[0][0]
__________________________________________________________________________________________________
batch_normalization_54 (BatchNo (None, None, None, 1 4096 conv2d_54[0][0]
__________________________________________________________________________________________________
leaky_re_lu_54 (LeakyReLU) (None, None, None, 1 0 batch_normalization_54[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, None, None, 5 524288 leaky_re_lu_54[0][0]
__________________________________________________________________________________________________
batch_normalization_55 (BatchNo (None, None, None, 5 2048 conv2d_55[0][0]
__________________________________________________________________________________________________
leaky_re_lu_55 (LeakyReLU) (None, None, None, 5 0 batch_normalization_55[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_55[0][0]
__________________________________________________________________________________________________
batch_normalization_56 (BatchNo (None, None, None, 1 4096 conv2d_56[0][0]
__________________________________________________________________________________________________
leaky_re_lu_56 (LeakyReLU) (None, None, None, 1 0 batch_normalization_56[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (None, None, None, 5 524288 leaky_re_lu_56[0][0]
__________________________________________________________________________________________________
batch_normalization_57 (BatchNo (None, None, None, 5 2048 conv2d_57[0][0]
__________________________________________________________________________________________________
leaky_re_lu_57 (LeakyReLU) (None, None, None, 5 0 batch_normalization_57[0][0]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_57[0][0]
__________________________________________________________________________________________________
batch_normalization_59 (BatchNo (None, None, None, 2 1024 conv2d_60[0][0]
__________________________________________________________________________________________________
leaky_re_lu_59 (LeakyReLU) (None, None, None, 2 0 batch_normalization_59[0][0]
__________________________________________________________________________________________________
up_sampling2d_1 (UpSampling2D) (None, None, None, 2 0 leaky_re_lu_59[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, None, None, 7 0 up_sampling2d_1[0][0]
add_19[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (None, None, None, 2 196608 concatenate_1[0][0]
__________________________________________________________________________________________________
batch_normalization_60 (BatchNo (None, None, None, 2 1024 conv2d_61[0][0]
__________________________________________________________________________________________________
leaky_re_lu_60 (LeakyReLU) (None, None, None, 2 0 batch_normalization_60[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_60[0][0]
__________________________________________________________________________________________________
batch_normalization_61 (BatchNo (None, None, None, 5 2048 conv2d_62[0][0]
__________________________________________________________________________________________________
leaky_re_lu_61 (LeakyReLU) (None, None, None, 5 0 batch_normalization_61[0][0]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_61[0][0]
__________________________________________________________________________________________________
batch_normalization_62 (BatchNo (None, None, None, 2 1024 conv2d_63[0][0]
__________________________________________________________________________________________________
leaky_re_lu_62 (LeakyReLU) (None, None, None, 2 0 batch_normalization_62[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_62[0][0]
__________________________________________________________________________________________________
batch_normalization_63 (BatchNo (None, None, None, 5 2048 conv2d_64[0][0]
__________________________________________________________________________________________________
leaky_re_lu_63 (LeakyReLU) (None, None, None, 5 0 batch_normalization_63[0][0]
__________________________________________________________________________________________________
conv2d_65 (Conv2D) (None, None, None, 2 131072 leaky_re_lu_63[0][0]
__________________________________________________________________________________________________
batch_normalization_64 (BatchNo (None, None, None, 2 1024 conv2d_65[0][0]
__________________________________________________________________________________________________
leaky_re_lu_64 (LeakyReLU) (None, None, None, 2 0 batch_normalization_64[0][0]
__________________________________________________________________________________________________
conv2d_68 (Conv2D) (None, None, None, 1 32768 leaky_re_lu_64[0][0]
__________________________________________________________________________________________________
batch_normalization_66 (BatchNo (None, None, None, 1 512 conv2d_68[0][0]
__________________________________________________________________________________________________
leaky_re_lu_66 (LeakyReLU) (None, None, None, 1 0 batch_normalization_66[0][0]
__________________________________________________________________________________________________
up_sampling2d_2 (UpSampling2D) (None, None, None, 1 0 leaky_re_lu_66[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, None, None, 3 0 up_sampling2d_2[0][0]
add_11[0][0]
__________________________________________________________________________________________________
conv2d_69 (Conv2D) (None, None, None, 1 49152 concatenate_2[0][0]
__________________________________________________________________________________________________
batch_normalization_67 (BatchNo (None, None, None, 1 512 conv2d_69[0][0]
__________________________________________________________________________________________________
leaky_re_lu_67 (LeakyReLU) (None, None, None, 1 0 batch_normalization_67[0][0]
__________________________________________________________________________________________________
conv2d_70 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_67[0][0]
__________________________________________________________________________________________________
batch_normalization_68 (BatchNo (None, None, None, 2 1024 conv2d_70[0][0]
__________________________________________________________________________________________________
leaky_re_lu_68 (LeakyReLU) (None, None, None, 2 0 batch_normalization_68[0][0]
__________________________________________________________________________________________________
conv2d_71 (Conv2D) (None, None, None, 1 32768 leaky_re_lu_68[0][0]
__________________________________________________________________________________________________
batch_normalization_69 (BatchNo (None, None, None, 1 512 conv2d_71[0][0]
__________________________________________________________________________________________________
leaky_re_lu_69 (LeakyReLU) (None, None, None, 1 0 batch_normalization_69[0][0]
__________________________________________________________________________________________________
conv2d_72 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_69[0][0]
__________________________________________________________________________________________________
batch_normalization_70 (BatchNo (None, None, None, 2 1024 conv2d_72[0][0]
__________________________________________________________________________________________________
leaky_re_lu_70 (LeakyReLU) (None, None, None, 2 0 batch_normalization_70[0][0]
__________________________________________________________________________________________________
conv2d_73 (Conv2D) (None, None, None, 1 32768 leaky_re_lu_70[0][0]
__________________________________________________________________________________________________
batch_normalization_71 (BatchNo (None, None, None, 1 512 conv2d_73[0][0]
__________________________________________________________________________________________________
leaky_re_lu_71 (LeakyReLU) (None, None, None, 1 0 batch_normalization_71[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (None, None, None, 1 4718592 leaky_re_lu_57[0][0]
__________________________________________________________________________________________________
conv2d_66 (Conv2D) (None, None, None, 5 1179648 leaky_re_lu_64[0][0]
__________________________________________________________________________________________________
conv2d_74 (Conv2D) (None, None, None, 2 294912 leaky_re_lu_71[0][0]
__________________________________________________________________________________________________
batch_normalization_58 (BatchNo (None, None, None, 1 4096 conv2d_58[0][0]
__________________________________________________________________________________________________
batch_normalization_65 (BatchNo (None, None, None, 5 2048 conv2d_66[0][0]
__________________________________________________________________________________________________
batch_normalization_72 (BatchNo (None, None, None, 2 1024 conv2d_74[0][0]
__________________________________________________________________________________________________
leaky_re_lu_58 (LeakyReLU) (None, None, None, 1 0 batch_normalization_58[0][0]
__________________________________________________________________________________________________
leaky_re_lu_65 (LeakyReLU) (None, None, None, 5 0 batch_normalization_65[0][0]
__________________________________________________________________________________________________
leaky_re_lu_72 (LeakyReLU) (None, None, None, 2 0 batch_normalization_72[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (None, None, None, 2 261375 leaky_re_lu_58[0][0]
__________________________________________________________________________________________________
conv2d_67 (Conv2D) (None, None, None, 2 130815 leaky_re_lu_65[0][0]
__________________________________________________________________________________________________
conv2d_75 (Conv2D) (None, None, None, 2 65535 leaky_re_lu_72[0][0]
__________________________________________________________________________________________________
input_2 (InputLayer) (None, 13, 13, 3, 85 0
__________________________________________________________________________________________________
input_3 (InputLayer) (None, 26, 26, 3, 85 0
__________________________________________________________________________________________________
input_4 (InputLayer) (None, 52, 52, 3, 85 0
__________________________________________________________________________________________________
yolo_loss (Lambda) (None, 1) 0 conv2d_59[0][0]
conv2d_67[0][0]
conv2d_75[0][0]
input_2[0][0]
input_3[0][0]
input_4[0][0]
==================================================================================================
Total params: 62,001,757
Trainable params: 61,949,149
Non-trainable params: 52,608
__________________________________________________________________________________________________
训练回调函数定义 In [7]:
from keras.callbacks import ReduceLROnPlateau, EarlyStopping
# 定义回调方法
reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=3, verbose=1) # 学习率衰减策略
early_stopping = EarlyStopping(monitor='val_loss', min_delta=0, patience=10, verbose=1) # 早停策略
开始训练
In [8]:
from keras.optimizers import Adam
from yolo3.utils import get_random_data
# 设置所有的层可训练
for i in range(len(model.layers)):
model.layers[i].trainable = True
# 选择Adam优化器，设置学习率
learning_rate = 1e-4
model.compile(optimizer=Adam(lr=learning_rate), loss={'yolo_loss': lambda y_true, y_pred: y_pred})
# 设置批大小和训练轮数
batch_size = 16
max_epochs = 2
print('Train on {} samples, val on {} samples, with batch size {}.'.format(num_train, num_val, batch_size))
# 开始训练
model.fit_generator(data_generator_wrapper(lines[:num_train], batch_size, input_shape, data_path,anchors, num_classes),
steps_per_epoch=max(1, num_train//batch_size),
validation_data=data_generator_wrapper(lines[num_train:], batch_size, input_shape, data_path,anchors, num_classes),
validation_steps=max(1, num_val//batch_size),
epochs=max_epochs,
initial_epoch=0,
callbacks=[reduce_lr, early_stopping])
Train on 1346 samples, val on 149 samples, with batch size 16. Epoch 1/2 84/84 [==============================] - 108s 1s/step - loss: 13.2619 - val_loss: 12.6032 Epoch 2/2 84/84 [==============================] - 89s 1s/step - loss: 11.8585 - val_loss: 12.2370
Out[8]:
<keras.callbacks.History at 0x7fdd6ba012e8>
保存模型 In [9]:
import os
os.makedirs(save_path)
# 保存模型
model.save_weights(os.path.join(save_path, 'trained_weights_final.h5'))
模型测试
打开一张测试图片 In [10]:
from PIL import Image
import numpy as np
# 测试文件路径
test_file_path = './test.jpg'
# 打开测试文件
image = Image.open(test_file_path)
image_ori = np.array(image)
image_ori.shape
Out[10]:
(640, 481, 3)
图片预处理 In [11]:
from yolo3.utils import letterbox_image
new_image_size = (image.width - (image.width % 32), image.height - (image.height % 32))
boxed_image = letterbox_image(image, new_image_size)
image_data = np.array(boxed_image, dtype='float32')
image_data /= 255.
image_data = np.expand_dims(image_data, 0)
image_data.shape
Out[11]:
(1, 640, 480, 3)
In [12]:
import keras.backend as K
sess = K.get_session()
构建模型 In [13]:
from yolo3.model import yolo_body
from keras.layers import Input
# coco数据anchor值文件存储位置
anchor_path = "./model_data/yolo_anchors.txt"
with open(anchor_path) as f:
anchors = f.readline()
anchors = [float(x) for x in anchors.split(',')]
anchors = np.array(anchors).reshape(-1, 2)
yolo_model = yolo_body(Input(shape=(None,None,3)), len(anchors)//3, num_classes)
加载模型权重，或将模型路径替换成上一步训练得出的模型路径 In [14]:
# 模型权重存储路径
weights_path = "./model_data/yolo.h5"
yolo_model.load_weights(weights_path)
定义IOU以及score：
- IOU： 将交并比大于IOU的边界框作为冗余框去除
- score：将预测分数大于score的边界框筛选出来
In [15]:
iou = 0.45
score = 0.8
构建输出[boxes, scores, classes] In [16]:
from yolo3.model import yolo_eval
input_image_shape = K.placeholder(shape=(2, ))
boxes, scores, classes = yolo_eval(
yolo_model.output,
anchors,
num_classes,
input_image_shape,
score_threshold=score,
iou_threshold=iou)
进行预测 In [17]:
out_boxes, out_scores, out_classes = sess.run(
[boxes, scores, classes],
feed_dict={
yolo_model.input: image_data,
input_image_shape: [image.size[1], image.size[0]],
K.learning_phase(): 0
})
In [18]:
class_coco = get_classes(classes_path)
out_coco = []
for i in out_classes:
out_coco.append(class_coco[i])
In [19]:
print(out_boxes)
print(out_scores)
print(out_coco)
[[152.6994 166.27255 649.0503 459.93747 ] [ 68.62152 21.843102 465.6621 452.6878 ]] [0.9838943 0.999688 ] ['person', 'umbrella']
将预测结果绘制在图片上 In [20]:
from PIL import Image, ImageFont, ImageDraw
font = ImageFont.truetype(font='font/FiraMono-Medium.otf',
size=np.floor(3e-2 * image.size[1] + 0.5).astype('int32'))
thickness = (image.size[0] + image.size[1]) // 300
for i, c in reversed(list(enumerate(out_coco))):
predicted_class = c
box = out_boxes[i]
score = out_scores[i]
label = '{} {:.2f}'.format(predicted_class, score)
draw = ImageDraw.Draw(image)
label_size = draw.textsize(label, font)
top, left, bottom, right = box
top = max(0, np.floor(top + 0.5).astype('int32'))
left = max(0, np.floor(left + 0.5).astype('int32'))
bottom = min(image.size[1], np.floor(bottom + 0.5).astype('int32'))
right = min(image.size[0], np.floor(right + 0.5).astype('int32'))
print(label, (left, top), (right, bottom))
if top - label_size[1] >= 0:
text_origin = np.array([left, top - label_size[1]])
else:
text_origin = np.array([left, top + 1])
for i in range(thickness):
draw.rectangle(
[left + i, top + i, right - i, bottom - i],
outline=225)
draw.rectangle(
[tuple(text_origin), tuple(text_origin + label_size)],
fill=225)
draw.text(text_origin, label, fill=(0, 0, 0), font=font)
del draw
umbrella 1.00 (22, 69) (453, 466) person 0.98 (166, 153) (460, 640)
In [21]:
image
Out[21]:
© 版权声明
文章版权归作者所有，未经允许请勿转载。
THE END
一发入魂
请登录后发表评论
注册
社交帐号登录