Face2Face swMATH ID: 42555 Software Authors: Justus Thies, Michael Zollhöfer, Marc Stamminger, Christian Theobalt, Matthias Nießner Description: Face2Face: Real-time Face Capture and Reenactment of RGB Videos. We present Face2Face, a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time. Homepage: https://arxiv.org/abs/2007.14808 Related Software: PyTorch; ImageNet; EfficientNet; PWC-Net; LoFTR; Inception-v4; UCF101; TransGAN; StarGAN; InfoGAN; DensePose; ArcFace; Adam; Caffe; SegStereo; AnchorNet; Soft scissors; MVSNet; NIMA; Fashion-MNIST Cited in: 3 Documents all top 5 Cited by 8 Authors 1 Amerini, Irene 1 Anagnostopoulos, Aris 1 Bao, Michael 1 Celsi, Lorenzo Ricciardi 1 Fedkiw, Ronald P. 1 Hyde, David A. B. 1 Maiano, Luca 1 Szeliski, Richard Cited in 3 Serials 1 Journal of Computational Physics 1 Foundations and Trends in Computer Graphics and Vision 1 Texts in Computer Science Cited in 3 Fields 3 Computer science (68-XX) 1 Numerical analysis (65-XX) 1 Operations research, mathematical programming (90-XX) Citations by Year