Optical flow estimation via fusing sequence image intensity correlation information

XU Xu, MA Pengfei, SI Jianjun, GAO Guojun

PDF(1443 KB)
PDF(1443 KB)
Journal of Liaoning Technical University (Natural Science) ›› 2025, Vol. 44 ›› Issue (01) : 120-128. DOI: 10.11956/j.issn.1008-0562.20230491

Optical flow estimation via fusing sequence image intensity correlation information

Author information +
History +

Abstract

To address the challenges associated with inaccurate target edge segmentation, motion speed, and motion direction, this paper introduces an optical flow estimation network that leverages local correlation information between video frames. Initially, the network employs a feature encoder to extract encoding features from the image and capture contextual information through a context network. Subsequently, the feature size is reduced through downsampling to enhance computational efficiency. Given the minute displacement of the optical flow image across consecutive frames, a partition-based visual similarity computation method is proposed to construct a more refined 4D correlation volume. Residual filters and similar convolution blocks are utilized for processing the correlation volume and optical flow information, respectively, ensuring the preservation of local small displacement details. The research results show that the optical flow estimation network based on the local correlation information between video frames has achieved optimizations of 8.0% and 5.7% respectively in the optical flow estimation evaluation metric (endpoint error, EPE). This significantly improves the accuracy of optical flow estimation and effectively alleviates the problem of inaccurate optical flow information extraction in complex scenarios. The research conclusions provide references for fields such as autonomous driving and intelligent security.

Key words

computer vision / optical flow estimation / deep learning / regional matching / iterative update

Cite this article

Download Citations
XU Xu , MA Pengfei , SI Jianjun , et al. Optical flow estimation via fusing sequence image intensity correlation information. Journal of Liaoning Technical University (Natural Science). 2025, 44(01): 120-128 https://doi.org/10.11956/j.issn.1008-0562.20230491

References

1
焦红虹,周浩,方淇.基于光流场的时间分段网络行为识别[J].云南大学学报(自然科学版),2019,41(1):36-45.
JIAO Honghong, ZHOU Hao, FANG Qi.The Temporal Segment Network based optical flow for action recognition[J].Journal of Yunnan University(Natural Sciences Edition),2019,41(1):36-45.
2
杨华,王姣,张维君 等.基于光流估计的轻量级视频插帧算法[J].沈阳航空航天大学学报,2022,39(6):57-64.
YANG Hua, WANG Jiao, ZHANG Weijun,et al.Lightweight video frame interpolation algorithm based on optical flow estimation[J].Journal of Shenyang Aerospace University,2022,39(6):57-64.
3
李志慧,胡永利,赵永华,等.基于车载的运动行人区域估计方法[J].吉林大学学报(工学版),2018,48(3):694-703.
LI Zhihui, HU Yongli, ZHAO Yonghua,et al.Locating moving pedestrian from running vehicle[J]. Journal of Jilin University (Engineering and Technology Edition),2018,48(3):694-703.
4
HORN B K P, SCHUNCK B G.Determining optical flow[J].Artificial Intelligence,1981,17(1/2/3):185-203.
5
BLACK M J, ANANDAN P.A framework for the robust estimation of optical flow[C]//1993(4th) International Conference on Computer Vision.May 11-14,1993,Berlin,Germany.IEEE,1993:231-236.
6
ZACH C, POCK T, BISCHOF H.A duality based approach for realtime TV-L1 optical flow[C]//Pattern Recognition.Berlin,Heidelberg:Springer Berlin Heidelberg,2007:214-223.
7
WEINZAEPFEL P, REVAUD J, HARCHAOUI Z,et al.DeepFlow:large displacement optical flow with deep matching[C]//2013 IEEE International Conference on Computer Vision.December 1-8,2013, Sydney,NSW,Australia.IEEE,2013:1385-1392.
8
SIMONYAN K, ZISSERMAN A, SIMONYAN K,et al.Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems-Volume 1.December 8-13,2014,Montreal,Canada.ACM,2014: 568-576.
9
ILG E, MAYER N, SAIKIA T,et al.FlowNet 2.0:evolution of optical flow estimation with deep networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.July 21-26,2017,Honolulu,HI,USA. IEEE,2017:1647-1655.
10
RANJAN A, BLACK M J.Optical flow estimation using a spatial pyramid network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition.July 21-26,2017,Honolulu,HI,USA.IEEE,2017: 2720-2729.
11
SUN D Q, YANG X D, LIU M Y,et al.PWC-net:cnns for optical flow using pyramid,warping,and cost volume[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:8934-8943.
12
YANG G, RAMANAN D.Volumetric Correspondence Networks for Optical Flow[C]//Annual Conference on Neural Information Processing Systems,December 2019.Vancouver,BC,Canada:NeurIPS,2019:793-803.
13
HUI T W, TANG X O, LOY C C.LiteFlowNet:a lightweight convolutional neural network for optical flow estimation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. June 18-23,2018,Salt Lake City,UT,USA.IEEE,2018:8981-8989.
14
TEED Z, DENG J.RAFT:recurrent all-pairs field transforms for optical flow[C]//Computer Vision-ECCV 2020.Cham:Springer International Publishing,2020:402-419.
15
GEIGER A, LENZ P, STILLER C,et al.Vision meets robotics:The KITTI dataset[J].International Journal of Robotics Research,2013,32 (11):1231-1237.
16
BUTLER D J, WULFF J, STANLEY G B,et al.A naturalistic open source movie for optical flow evaluation[C]//Computer Vision-ECCV 2012.Berlin,Heidelberg:Springer Berlin Heidelberg,2012:611-625.
17
张水发,张文生,丁欢,等.融合光流速度与背景建模的目标检测方法[J].中国图象图形学报,2011,16(2):236-243.
ZHANG Shuifa, ZHANG Wensheng, DING Huan,et al.Background modeling and object detecting based on optical flow velocity field[J].Journal of Image and Graphics,2011,16(2):236-243.
18
许广富,曾继超,刘锡祥.融合光流法和特征匹配的视觉里程计[J].激光与光电子学进展,2020,57(20):270-278.
XU Guangfu, ZENG Jichao, LIU Xixiang.Visual odometer based on optical flow method and feature matching[J].Laser & Optoelectronics Progress,2020,57(20):270-278.
19
安峰,戴军,韩振,等.引入注意力机制的自监督光流计算[J].图学学报,2022,43(5):841-848.
AN Feng, DAI Jun, HAN Zhen,et al.Self-supervised optical flow estimation with attention module[J].Journal of Graphics,2022,43(5):841-848.
20
WANG J Y, ZHONG Y R, DAI Y C,et al.Displacement-invariant matching cost learning for accurate optical flow estimation[EB/OL].(2020-10-28)[2023-11-17].

Comments

PDF(1443 KB)

Accesses

Citation

Detail

Sections
Recommended

/