2020年12月18日
OpenCV Dnn调用OpenVINO security_barrier_camera模型检测并识别车牌
Posted in openCV • Python By On 2020年12月18日,2279 Views在OpenVINO+Win10安装及环境配置攻略一文中有一个Security Barrier Camera Demo 应用程序,它调用了3个模型,可以检测车辆,车牌,车辆颜色以及类型,并且识别车牌。本文分享如何用OpenCV的DNN来调用Security Barrier Camera Demo 中的模型实现检测并识别车牌。
1. 运行环境
python3.6
opencv-contrib-python 4.4.0.44
openvino_2021.1.110
2. Security Barrier Camera Demo
Security Barrier Camera Demo展示了车辆和车牌检测模型,然后是在检测结果之上进行车辆属性识别和车牌识别,使用的检测模型如下:
vehicle-license-plate-detection-barrier-0106:车辆和车牌的检测模型;
vehicle-attributes-recognition-barrier-0039:在第一个模型的结果之上执行,当第一个模型检测到车辆时,当前模型可检测车辆属性,例如,车辆类型(汽车/公共汽车/卡车/货车)和颜色。经测试,该模型准确度不高。查阅过几篇应用此模型的博客,检测结果与我的测试结果类似,当然也有可能时我使用不当,若有高手指点,感激不尽。
license-plate-recognition-barrier-0001:在第一个模型的结果之上执行,当第一个模型检测到车牌时,该模型可识别当前车牌。
3. OpenCV调用OpenVINO security_barrier_camera模型
下面我们开始讲解代码testOpenVINO.py。我们先来看变量定义部分。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
import numpy as np import cv2 import imutils from imutils import paths bShowColor = False TEST_PATH = "testImgs" FRAME_WIDTH = 0 FRAME_HEIGHT = 0 vColor = (0, 255, 0)#vehicle bounding-rect and information color pColor = (0, 0, 255)#plate bounding-rect and information color rectThinkness = 2 items = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "<Anhui>", "<Beijing>", "<Chongqing>", "<Fujian>", "<Gansu>", "<Guangdong>", "<Guangxi>", "<Guizhou>", "<Hainan>", "<Hebei>", "<Heilongjiang>", "<Henan>", "<HongKong>", "<Hubei>", "<Hunan>", "<InnerMongolia>", "<Jiangsu>", "<Jiangxi>", "<Jilin>", "<Liaoning>", "<Macau>", "<Ningxia>", "<Qinghai>", "<Shaanxi>", "<Shandong>", "<Shanghai>", "<Shanxi>", "<Sichuan>", "<Tianjin>", "<Tibet>", "<Xinjiang>", "<Yunnan>", "<Zhejiang>", "<police>", "A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z"] seq_ind = np.ones([88, 1], dtype=np.float32) seq_ind[0, 0] = 0 #https://docs.openvinotoolkit.org/2019_R1/_license_plate_recognition_barrier_0001_description_license_plate_recognition_barrier_0001.html lpr_net = cv2.dnn.Net_readFromModelOptimizer('./license-plate-recognition-barrier-0001/license-plate-recognition-barrier-0001.xml', './license-plate-recognition-barrier-0001/license-plate-recognition-barrier-0001.bin') lpr_net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE) lpr_net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU) CAR_COLORS = ["white", "gray", "yellow", "red", "green", "blue", "black"] CAR_TYPES = ["car", "bus", "truck", "van"] #https://docs.openvinotoolkit.org/2019_R1/_vehicle_attributes_recognition_barrier_0039_description_vehicle_attributes_recognition_barrier_0039.html attr_net = cv2.dnn.Net_readFromModelOptimizer('./vehicle-attributes-recognition-barrier-0039/vehicle-attributes-recognition-barrier-0039.xml', './vehicle-attributes-recognition-barrier-0039/vehicle-attributes-recognition-barrier-0039.bin') attr_net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE) attr_net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU) #https://docs.openvinotoolkit.org/2019_R1/_vehicle_license_plate_detection_barrier_0106_description_vehicle_license_plate_detection_barrier_0106.html pd_net = cv2.dnn.readNet('./vehicle-license-plate-detection-barrier-0106/vehicle-license-plate-detection-barrier-0106.xml', './vehicle-license-plate-detection-barrier-0106/vehicle-license-plate-detection-barrier-0106.bin') pd_net.setPreferableBackend(cv2.dnn.DNN_BACKEND_INFERENCE_ENGINE) pd_net.setPreferableTarget(cv2.dnn.DNN_TARGET_CPU) |
第1~4行导入必要库;
第6行设定是否需要在输出结果中显示车型和车的颜色检测结果(vehicle-attributes-recognition-barrier-0039),因为车型和车的颜色检测结果准确度不高,因此默认不显示。
第8行设定测试图片文件夹,本文分享的代码会读取testOpenVINO.py同级文件夹testImgs中的图片,并逐一处理。如需要更改测试图片路径,更改此行代码即可。
第10~11行,记录当前处理图片的宽与高;
第13行,vehicle-license-plate-detection-barrier-0106检测到的车辆将用绿色矩形在当前被检测图片中标注;
第14行,vehicle-license-plate-detection-barrier-0106检测到的车牌将用红色矩形在当前被检测图片中标注;
第15行,标注车辆与车牌时矩形线条的粗细设定值;
第17行,请参考license-plate-recognition-barrier-0001,items用于将该模型的输出结果解析为相应的字符;
第31-32行,模型license-plate-recognition-barrier-0001使用时需要的辅助变量;
第35-38行,定义模型license-plate-recognition-barrier-0001的检测网络并设定相关变量;
第41-42行,用于解析模型vehicle-attributes-recognition-barrier-0039的检测结果;
第44-47行,定义模型vehicle-attributes-recognition-barrier-0039的检测网络并设定相关变量;
第50-53行,定义模型vehicle-license-plate-detection-barrier-0106的检测网络并设定相关变量;
接下来看主体子函数plateRecognition。该函数先用vehicle-license-plate-detection-barrier-0106对图片进行检测,其检测结果分为两类,classId=1时,当前返回结果为车辆,此时调用vehicle-attributes-recognition-barrier-0039对车辆属性进行检测;classId=2时,当前返回结果为车牌,此时调用license-plate-recognition-barrier-0001对车牌进行识别。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 |
def plateRecognition(frame): #name: "input" , shape: [1x3x300x300] - An input image in the format [BxCxHxW], where: pd_blob = cv2.dnn.blobFromImage(frame, size=(300, 300), ddepth=cv2.CV_8U) pd_net.setInput(pd_blob) out_pb = pd_net.forward() # An every detection is a vector [imageId, classId, conf, x, y, X, Y] for detection in out_pb.reshape(-1, 7): conf = detection[2] if conf < 0.4: continue classId = int(detection[1]) if classId == 1:#car xmin = int(detection[3] * FRAME_WIDTH) ymin = int(detection[4] * FRAME_HEIGHT) xmax = int(detection[5] * FRAME_WIDTH) ymax = int(detection[6] * FRAME_HEIGHT) rectW = xmax - xmin if rectW < 72: # Minimal weight in vehicle-attributes-recognition-barrier-0039 is 72 continue attrImg = frame[ymin:ymax+1, xmin:xmax+1] attr_blob = cv2.dnn.blobFromImage(attrImg, size=(72, 72), ddepth=cv2.CV_8U) attr_net.setInput(attr_blob,'input') out_color = attr_net.forward("color") out_type = attr_net.forward("type") carColor = "Color: " + CAR_COLORS[np.argmax(out_color.flatten())] carType = "Type: " + CAR_TYPES[np.argmax(out_type.flatten())] cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), vColor,rectThinkness) if bShowColor: drawText(frame,rectW*0.002,xmin,ymin,vColor,carColor +" " + carType) elif classId == 2:#plate xmin = int(detection[3] * FRAME_WIDTH) ymin = int(detection[4] * FRAME_HEIGHT) xmax = int(detection[5] * FRAME_WIDTH) ymax = int(detection[6] * FRAME_HEIGHT) xmin = max(0, xmin - 5) ymin = max(0, ymin - 5) xmax = min(xmax + 5, FRAME_WIDTH - 1) ymax = min(ymax + 5, FRAME_HEIGHT - 1) rectW = xmax - xmin if rectW < 93: # Minimal weight in plate-recognition-barrier-0001 is 94 continue # Crop a license plate. Do some offsets to better fit a plate. lpImg = frame[ymin:ymax+1, xmin:xmax+1] blob = cv2.dnn.blobFromImage(lpImg, size=(94, 24), ddepth=cv2.CV_8U) lpr_net.setInput(blob, 'data') lpr_net.setInput(seq_ind, 'seq_ind') out_lpr = lpr_net.forward() content = '' for idx in np.int0(out_lpr.reshape(-1)): if idx == -1: break content += items[idx] cv2.rectangle(frame, (xmin, ymin), (xmax, ymax), pColor,rectThinkness) drawText(frame,rectW*0.008,xmin,ymin,pColor,content) showImg = imutils.resize(frame, height=600) cv2.imshow("showImg",showImg) |
第4-6行根据模型要求设定模型输入并进行模型推理;
第9行开始遍历vehicle-license-plate-detection-barrier-0106每一个检测结果;
第10-12行,检测当前结果的置信度,仅处理置信度不小于0.4的检测结果;
第15行,若当前检测结果类别为车辆,则接下来执行vehicle-attributes-recognition-barrier-0039车辆属性检测;
第16-19行,获取当前车辆位置的左上角和右下角坐标;
第21-23行,根据vehicle-attributes-recognition-barrier-0039模型对输入的要求对检测车辆宽度进行限定;
第25-28行,根据模型要求设定模型输入;
第30-31行,模型推理。vehicle-attributes-recognition-barrier-0039的模型推理值得关注。大部分模型的输出只有一个,但该模型输出有两个,因此在进行推理时我们需要分别设置相应的参数。
1 2 |
name: "color", shape: [1, 7, 1, 1] - Softmax output across seven color classes [white, gray, yellow, red, green, blue, black] name: "type", shape: [1, 4, 1, 1] - Softmax output across four type classes [car, bus, truck, van] |
第33-34行,将输出结果由多维数组转换为一维数组,并获取最大值的序号,再从我们定义的车辆属性变量CAR_COLORS和CAR_TYPES获取最大值对应的属性;
第36行,在检测图像中标注车辆的位置;
第38-39行,在检测图像中标注车辆属性的检测结果;
第42行,若vehicle-license-plate-detection-barrier-0106检测结果类别为车牌,则接下来执行license-plate-recognition-barrier-0001对车牌进行识别;
第43-51行,获取当前车辆位置的左上角和右下角坐标,并对车牌坐标进行适当扩充;
第53-55行,根据license-plate-recognition-barrier-0001模型对输入的要求对检测车牌宽度进行限定;
第58-62行,根据模型要求设定模型输入并进行推理;这里也是值得我们注意的,license-plate-recognition-barrier-0001有两个输入,因此我们在使用函数setInput时,需要设定参数;
1 2 3 4 |
Inputs name: "data" , shape: [1x3x24x94] - An input image in following format [1xCxHxW]. Expected color order is BGR. name: "seq_ind" , shape: [88,1] - An auxiliary blob that is needed for correct decoding. Set this to [0, 1, 1, ..., 1]. |
第64-68行,对license-plate-recognition-barrier-0001检测结果进行解析,获取车牌识别结果;
第70-71行,在检测图像中标注车牌位置以及车牌识别结果;
第73-74行,显示检测结果。为了观察方便,显示结果时将图像高度统一设定为600。
4. 运行testOpenVINO.py
4.1 打开command.exe,切换至OpenVINO安装目录下的setupvars.bat文件所在路径,运行setupvars.bat。
4.2 在当前command.exe窗口中切换至testOpenVINO.py所在路径运行
1 2 3 4 5 6 7 8 9 10 |
C:\Users\username>cd C:\Program Files (x86)\Intel\openvino_2021.1.110\bin C:\Program Files (x86)\Intel\openvino_2021.1.110\bin>setupvars.bat Python 3.6.5 [setupvars.bat] OpenVINO environment initialized C:\Program Files (x86)\Intel\openvino_2021.1.110\bin>D: D:\>cd D:\Python\OpenVINOMODEL D:\Python\OpenVINOMODEL>python testOpenVINO.py [E:] [BSL] found 0 ioexpander device <span style="font-family:Microsoft YaHei;"></span> |
5. 总结
本文分享了在python中用OpenCV的DNN来调用Open Model Zoo中的预训练模型中的车辆与车牌检测和识别,在理解了这一部分的使用方法后,我们就可以根据需求来调用Open Model Zoo其他有趣的预训练模型,增加更多奇怪的知识。
本文到此结束,感谢支持,欢迎关注。