FlowNet swMATH ID: 31203 Software Authors: E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox Description: FlowNet 2.0: Evolution of optical flow estimationwith deep networks. The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a sub-network specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50 Homepage: https://arxiv.org/abs/1612.01925 Source Code: https://github.com/lmb-freiburg/flownet2 Keywords: Computer Vision; Pattern Recognition; arXiv_cs.CV; Optical Flow Estimation; Deep Networks; FlowNet Related Software: Adam; EpicFlow; PWC-Net; KITTI; DeepFlow; TensorFlow; ImageNet; PointNet; UCF101; PyTorch; Key.Net; U-Net; MVSNet; BRISK; SOSNet; L2-Net; Theia; R2D2; SuperGlue; Pfinder Cited in: 17 Publications Standard Articles 1 Publication describing the Software Year FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, T. Brox 2016 all top 5 Cited by 56 Authors 2 de Bézenac, Emmanuel 2 Gallinari, Patrick 2 Pajot, Arthur 1 Aviles-Rivero, Angelica I. 1 Ayed, Ibrahim 1 Black, Michael J. 1 Bouwmans, Thierry 1 Buades, Antoni 1 Chapel, Marie-Neige 1 Corona, Veronica 1 Debroux, Noémie 1 Duran, Joan 1 Fan, Aoxiang 1 Fukagata, Koji 1 Fukami, Kai 1 Galliani, Silvano 1 Gua, Banji 1 Ha, Namkoo 1 Hackel, Timo 1 Hernández Huelin, Mónica María 1 Hoffmann, David T. 1 Huang, Wei 1 Innamorati, Carlo 1 Jang, Hyunsung 1 Jiang, Junjun 1 Jiang, Xingyu 1 Kim, Youngjung 1 Kuzmin, A. I. 1 Liu, Yan 1 Lv, Bingxue 1 Ma, Ding 1 Ma, Jiayi 1 Mitra, Niloy J. 1 Navarro, Julia 1 Oh, Changjae 1 Ranjan, Anurag 1 Ritschel, Tobias K. S. 1 Romero, Javier 1 Schindler, Konrad 1 Schmoderer, Timothée 1 Schönlieb, Carola-Bibiane 1 Sohn, Kwanghoon 1 Song, Taeyong 1 Szeliski, Richard 1 Taira, Kunihiko 1 Tang, Siyu 1 Tzionas, Dimitrios 1 Usvyatsov, Mikhail 1 Wang, Song 1 Wang, Yuheng 1 Wang, Zengfu 1 Wegner, Jan D. 1 Weyrich, Tim 1 Yan, Junchi 1 Yao, Rui 1 Zhao, Jiaqi all top 5 Cited in 13 Serials 5 International Journal of Computer Vision 1 Inverse Problems 1 Journal of Fluid Mechanics 1 Programming and Computer Software 1 Machine Learning 1 Journal of Mathematical Imaging and Vision 1 International Journal of Wavelets, Multiresolution and Information Processing 1 Journal of Statistical Mechanics: Theory and Experiment 1 Mathematical Biosciences and Engineering 1 SIAM Journal on Imaging Sciences 1 Algorithms 1 Computer Science Review 1 Texts in Computer Science all top 5 Cited in 6 Fields 15 Computer science (68-XX) 5 Information and communication theory, circuits (94-XX) 2 Calculus of variations and optimal control; optimization (49-XX) 2 Numerical analysis (65-XX) 1 Fluid mechanics (76-XX) 1 Geophysics (86-XX) Citations by Year