基于视频帧间局部相关信息的光流估计网络

徐煦, 马鹏飞, 司建军, 高国军

PDF(1443 KB)
PDF(1443 KB)
辽宁工程技术大学学报(自然科学版) ›› 2025, Vol. 44 ›› Issue (01) : 120-128. DOI: 10.11956/j.issn.1008-0562.20230491
图像识别与处理

基于视频帧间局部相关信息的光流估计网络

作者信息 +

Optical flow estimation via fusing sequence image intensity correlation information

Author information +
History +

摘要

为解决光流估计网络在目标边缘分割、运动速度和运动方向不准确的问题,提出基于视频帧间局部相关信息的光流估计网络。运用特征编码器从图像中提取出编码特征,通过上下文网络获取图像的上下文特征。采用下采样处理减小特征尺寸提高计算效率。根据连续两帧光流图像位移较小的特性,提出一种分区计算视觉相似度的方法,构建更为精细的4D相关体。采用残差滤波器和相似卷积块的方法,分别针对相关体和光流信息进行操作,更有效地保留局部微小位移信息。研究结果表明:采用基于视频帧间局部相关信息的光流估计网络进行计算,端点误差分别实现了8.0%和5.7%的优化,显著提升了光流估计的准确性,对复杂场景下光流信息提取更准确。研究结果可为自动驾驶、智能安防等领域提供参考。

Abstract

To address the challenges associated with inaccurate target edge segmentation, motion speed, and motion direction, this paper introduces an optical flow estimation network that leverages local correlation information between video frames. Initially, the network employs a feature encoder to extract encoding features from the image and capture contextual information through a context network. Subsequently, the feature size is reduced through downsampling to enhance computational efficiency. Given the minute displacement of the optical flow image across consecutive frames, a partition-based visual similarity computation method is proposed to construct a more refined 4D correlation volume. Residual filters and similar convolution blocks are utilized for processing the correlation volume and optical flow information, respectively, ensuring the preservation of local small displacement details. The research results show that the optical flow estimation network based on the local correlation information between video frames has achieved optimizations of 8.0% and 5.7% respectively in the optical flow estimation evaluation metric (endpoint error, EPE). This significantly improves the accuracy of optical flow estimation and effectively alleviates the problem of inaccurate optical flow information extraction in complex scenarios. The research conclusions provide references for fields such as autonomous driving and intelligent security.

关键词

计算机视觉 / 光流估计 / 深度学习 / 区域匹配 / 迭代更新

Key words

computer vision / optical flow estimation / deep learning / regional matching / iterative update

中图分类号

TP391.4

引用本文

导出引用
徐煦 , 马鹏飞 , 司建军 , . 基于视频帧间局部相关信息的光流估计网络. 辽宁工程技术大学学报(自然科学版). 2025, 44(01): 120-128 https://doi.org/10.11956/j.issn.1008-0562.20230491
XU Xu, MA Pengfei, SI Jianjun, et al. Optical flow estimation via fusing sequence image intensity correlation information[J]. Journal of Liaoning Technical University (Natural Science). 2025, 44(01): 120-128 https://doi.org/10.11956/j.issn.1008-0562.20230491

参考文献

1
焦红虹,周浩,方淇.基于光流场的时间分段网络行为识别[J].云南大学学报(自然科学版),2019,41(1):36-45.
JIAO Honghong, ZHOU Hao, FANG Qi.The Temporal Segment Network based optical flow for action recognition[J].Journal of Yunnan University(Natural Sciences Edition),2019,41(1):36-45.
2
杨华,王姣,张维君 等.基于光流估计的轻量级视频插帧算法[J].沈阳航空航天大学学报,2022,39(6):57-64.
YANG Hua, WANG Jiao, ZHANG Weijun,et al.Lightweight video frame interpolation algorithm based on optical flow estimation[J].Journal of Shenyang Aerospace University,2022,39(6):57-64.
3
李志慧,胡永利,赵永华,等.基于车载的运动行人区域估计方法[J].吉林大学学报(工学版),2018,48(3):694-703.
LI Zhihui, HU Yongli, ZHAO Yonghua,et al.Locating moving pedestrian from running vehicle[J]. Journal of Jilin University (Engineering and Technology Edition),2018,48(3):694-703.
4
HORN B K P, SCHUNCK B G.Determining optical flow[J].Artificial Intelligence,1981,17(1/2/3):185-203.
5
BLACK M J, ANANDAN P.A framework for the robust estimation of optical flow[C]//1993(4th) International Conference on Computer Vision.May 11-14,1993,Berlin,Germany.IEEE,1993:231-236.
6
ZACH C, POCK T, BISCHOF H.A duality based approach for realtime TV-L1 optical flow[C]//Pattern Recognition.Berlin,Heidelberg:Springer Berlin Heidelberg,2007:214-223.
7
WEINZAEPFEL P, REVAUD J, HARCHAOUI Z,et al.DeepFlow:large displacement optical flow with deep matching[C]//2013 IEEE International Conference on Computer Vision.December 1-8,2013, Sydney,NSW,Australia.IEEE,2013:1385-1392.
8
SIMONYAN K, ZISSERMAN A, SIMONYAN K,et al.Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 1.December 8-13,2014,Montreal,Canada.ACM,2014: 568-576.
9
ILG E, MAYER N, SAIKIA T,et al.FlowNet 2.0:evolution of optical flow estimation with deep networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.July 21-26,2017,Honolulu,HI,USA. IEEE,2017:1647-1655.
10
RANJAN A, BLACK M J.Optical flow estimation using a spatial pyramid network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.July 21-26,2017,Honolulu,HI,USA.IEEE,2017: 2720-2729.
11
SUN D Q, YANG X D, LIU M Y,et al.PWC-net:cnns for optical flow using pyramid,warping,and cost volume[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:8934-8943.
12
YANG G, RAMANAN D.Volumetric Correspondence Networks for Optical Flow[C]//Annual Conference on Neural Information Processing Systems,December 2019.Vancouver,BC,Canada:NeurIPS,2019:793-803.
13
HUI T W, TANG X O, LOY C C.LiteFlowNet:a lightweight convolutional neural network for optical flow estimation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:8981-8989.
14
TEED Z, DENG J.RAFT:recurrent all-pairs field transforms for optical flow[C]//Computer Vision-ECCV 2020.Cham:Springer International Publishing,2020:402-419.
15
GEIGER A, LENZ P, STILLER C,et al.Vision meets robotics:The KITTI dataset[J].International Journal of Robotics Research,2013,32 (11):1231-1237.
16
BUTLER D J, WULFF J, STANLEY G B,et al.A naturalistic open source movie for optical flow evaluation[C]//Computer Vision-ECCV 2012.Berlin,Heidelberg:Springer Berlin Heidelberg,2012:611-625.
17
张水发,张文生,丁欢,等.融合光流速度与背景建模的目标检测方法[J].中国图象图形学报,2011,16(2):236-243.
ZHANG Shuifa, ZHANG Wensheng, DING Huan,et al.Background modeling and object detecting based on optical flow velocity field[J].Journal of Image and Graphics,2011,16(2):236-243.
18
许广富,曾继超,刘锡祥.融合光流法和特征匹配的视觉里程计[J].激光与光电子学进展,2020,57(20):270-278.
XU Guangfu, ZENG Jichao, LIU Xixiang.Visual odometer based on optical flow method and feature matching[J].Laser & Optoelectronics Progress,2020,57(20):270-278.
19
安峰,戴军,韩振,等.引入注意力机制的自监督光流计算[J].图学学报,2022,43(5):841-848.
AN Feng, DAI Jun, HAN Zhen,et al.Self-supervised optical flow estimation with attention module[J].Journal of Graphics,2022,43(5):841-848.
20
WANG J Y, ZHONG Y R, DAI Y C,et al.Displacement-invariant matching cost learning for accurate optical flow estimation[EB/OL].(2020-10-28)[2023-11-17].

基金

国家自然科学基金项目(61601213)

评论

PDF(1443 KB)

Accesses

Citation

Detail

段落导航
相关文章

/