Skip to the content.

Abstract: We propose RIFE, a Real-time Intermediate Flow Estimation algorithm, with applications to Video Frame Interpolation (VFI). Most existing flow estimation methods first estimate the bi-directional optical flows, and then linearly combine them to approximate intermediate flows, leading to artifacts on motion boundaries. We design a neural network named IFNet, that can directly estimate the intermediate flows from images. When interpolating videos, we can warp the frames according to the estimated intermediate flows and employ a fusion process to compute final results. Based on our proposed leakage distillation loss, RIFE can be trained in an end-to-end fashion. Experiments demonstrate that our method is significantly faster than existing VFI methods and can achieve state-of-the-art performance on public benchmarks.

Image demos

16x interpolation results using only two images

Demo Demo

Video demos

We are consistenly working on improving the models generalization to videos with various appearance.

24 FPS -> 96 FPS

User Genereated demos

2d Animation 御坂大哥想让我表白 - 魔女之旅 | ablyh - 超电磁炮 | 赫萝与罗伦斯的旅途 - 绫波丽 | 没有鼠鼠的雏子Official - 千恋万花 |

3d Animation 没有鼠鼠的雏子Official - 原神魈 | 今天我练出腹肌了吗 - 最终幻想14 | 娜不列颠 - 冰雪奇缘2 | 今天我练出腹肌了吗 - 仙剑奇侠传6 |

MV Navetek - 邓丽君 | 生米阿怪 - 周深 |

Film Life in a Day 2020 |

Want to generate your own videos?

[colab]: Try our colab notebook, upload your own videos or images, and run! Or your can visit our github repo

[Apps]: You can refer to Waifu2x-Extension-GUI, Flowframes and RIFE-ncnn-vulkan. A Chinese version: Squirrel-RIFE

Contact

You can reach us at: Zhewei Huang @ huangzhewei@megvii.com, Tianyuan zhang @ tianyuanzhang@pku.edu.cn

How to cite

@article{huang2020rife,
  title={RIFE: Real-Time Intermediate Flow Estimation for Video Frame Interpolation},
  author={Huang, Zhewei and Zhang, Tianyuan and Heng, Wen and Shi, Boxin and Zhou, Shuchang},
  journal={arXiv preprint arXiv:2011.06294},
  year={2020}
}