Title: 3D Gaussian Splatting: Past, Present and Future

Speaker: George Drettakis, Senior Researcher, Inria, Team Leader GRAPHDECO , Inria Centre d'Université Côte d'Azur, Sophia-Antipolis, France

Location: “Stelios Orphanoudakis” Meeting Room (Foundation for Research and Technology-Hellas)
Date: Thursday September 26, 2024
Time: 17:00-18:30

Host: Yannis Tzitzikas

Abstract:
Neural rendering has advanced at outstanding speed in recent years, with the advent of Neural Radiance Fields (NeRFs), typically based on volumetric ray-marching. Last year, our group developed an alternative approach, 3D Gaussian Splatting, that has better performance for training, display speed and visual quality and has seen widespread adoption both academically and industrially. In this talk, we describe the 20+ year process leading to the development of this method and discuss some future directions. We will start with a short historical perspective of our work on image-based and neural rendering over the years, outlining several developments that guided our thinking over the years. We then discuss a sequence of three point-based rasterization methods for novel view synthesis -- developed in the context of G. Kopanas' Ph.D. and the ERC Advanced Grant FUNGRAPH -- that culminated with 3D Gaussian Splatting. We will emphasize how we progressively overcame the challenges as the research progressed. We first discuss differentiable point splatting and how we extended in our first approach that enhances points with neural features, optimizing geometry to correct reconstruction errors. We briefly review our second method that handles highly reflective objects, where we use multi-layer perceptrons (MLP), to learn the motion of reflections and to perform the final rendering of captured scenes. We then discuss 3D Gaussian Splatting, that provides the high-quality real-time rendering for novel view synthesis using a novel 3D scene representation based on 3D Gaussians and fast GPU rasterization. We will conclude with a discussion of future directions for 3D Gaussian splatting with examples from recent work.

Bio:
George Drettakis graduated in Computer Science from the University of Crete, Greece, and obtained an M.Sc. and a Ph.D., (1994) at the University of Toronto, with E. Fiume. After an ERCIM postdoc in Grenoble, Barcelona and Bonn, he obtained a Inria researcher position in Grenoble in 1995, and his "Habilitation" at the University of Grenoble (1999). He then founded the REVES research group at INRIA Sophia-Antipolis, and now heads the follow-up group GRAPHDECO. He is a INRIA Senior Researcher (full professor equivalent). He received the Eurographics (EG) Outstanding Technical Contributions award in 2007, the EG Distinguished Career Award in 2024 and is an EG fellow. He has received two  prestigious ERC Advanced Grants in 2018 and in 2024. He was associate editor for ACM Trans. on Graphics, technical papers chair of SIGGRAPH Asia 2010, co-chair of Eurographics IPC 2002 & 2008, chairσ the ACM SIGGRAPH Papers Advisory Group and the EG working group on Rendering (EGSR). He has worked on many different topics in computer graphics, with an emphasis on rendering. He initially concentrated on lighting and shadow computation and subsequently worked on 3D audio, perceptually-driven algorithms, virtual reality and 3D interaction. He has worked on textures, weathering and perception for graphics and in recent years focused on novel-view synthesis, relighting as well as material acquisition often using deep learning methodologies.