Research Activities
PhD Thesis: 4D modeling from multiple cameras.
Nowadays mutli-camera setups allow the acquisition of both color image streams and 3D models streams. Thus permitting the study of complex scenes. These scenes can be composed of any number of non-rigid objects moving freely. One of the main limitations of such data is its lack of temporal coherence between two consecutive observations. The work presented in this thesis consider this issue and propose novel methods to recover this temporal coherence. First we present a new approach that computes at each frame a dense motion field over the surface of the scene (i.e. Scene Flow), gathering both photometric and geometric information. We then extend this approach to hybrid multi-camera setups composed of color and depth sensor (such as the kinect sensor). Second, we introduce "Progressive Shape Models", a new method that allows to gather topology information over a complete sequence of 3D models and incrementally build a complete and coherent surface template.In French.
Learning Shape Topology
In this work we address the problem of recovering both the topology and the geometry of a deformable shape using temporal mesh sequences. The interest arises in multi-camera applications when unknown natural dynamic scenes are captured. While several approaches allow recovery of shape models from static scenes, few consider dynamic scenes with evolving topology and without prior knowledge. In this nonetheless generic situation, a single time observation is not necessarily sufficient to infer the correct topology of the observed shape and evidences must be accumulated over time in order to learn the topology and to enable temporally consistent modelling. This appears to be a new problem for which no formal solution exists. We propose a principled approach based on the assumption that the observed objects have a fixed topology. Under this assumption, we can progressively learn the topology meanwhile capturing the deformation of the dynamic scene. The approach has been successfully experimented on several standard 4D datasets.
Scene Flow
In this work we study how to incorporate, in an efficient way, various constraints when estimating dense motion information over 3D surfaces from temporal variations of the intensity function in several images. Our primary motivation is to provide robust motion cues that can be directly used by an application, e.g. interactive applications, or that can be fed into more advanced tasks such as surface tracking or segmentation. The approach is however not limited to a specific scenario and applies to any application that can benefit from low level motion information. We explore the combination of heterogeneous photometric and geometric cues in an unified framework. We fuse 2D and 3D, sparse and dense information to constrain the estimation of the instantaneous 3D scene flow on a surface. This project uses the output of any multi-camera system, such as Grimage, which gives for each time step a set of calibrated images. Our framework merges constraints coming from 2D normal flow, sparse 2D features matching in the images, sparse 3D features matching on the surfaces and a regularization term to provide an accurate and smooth motion field over the surface.
Master thesis - Plane detection in images
Binocular stereovision is a computer vision technique that uses two images of the same scene taken by sensors placed at different positions. It is used in many ways to recover the depth of a scene. After matching elements (pixels, regions, etc.) between the two images, it is possible to recover certain geometric properties of the scene. Detecting the planes of a scene from image pairs is a problem that has been extensively addressed. Several methods have been proposed, each trying to use a different approach (RANSAC, optical flow, manual pre-segmentation, etc.). Each approach has its advantages and disadvantages depending on the type of scene.This Master's thesis is part of the TSIGANE 1 project, which aims, on one hand, to enable onboard vision-based localization within a 3D geographic information system (GIS) and, on the other hand, to enhance the GIS with acquired images. Detecting planes would allow orientation in the world based on GIS information by detecting building walls, for example. A preliminary position would be provided by a GPS, and then matching the model with the scene reconstructed by the sensors could offer greater precision while incorporating textures into the GIS.
This work first provides a state-of-the-art overview of plane detection methods. Then, we will discuss two approaches for the simultaneous estimation of inter-image homographies. The first method is based on the use of generalized principal component analysis, while the second assumes urban scenes and uses the boundaries between planes to segment the correspondences between images. The work includes a detailed study of the first method and its adaptation to another type of data, line correspondences. It also involves the development of the second method, which approaches the problem in a novel way. We will explain how each approach works and present the results obtained along with their evaluation.
Teaching
- Spring semester 2016 - C Project, Under-Graduate practical course at INPG - Ensimag.
- Spring semester 2012 - Algorithms and C programming, Under-Graduate course at INPG - Phelma.
- Spring semester 2012 - Computer Graphics, Post-Graduate practical course at Université Joseph Fourier.
- Spring semester 2011 - Algorithms and data structures, Under-Graduate practical course at INPG - Ensimag.
- Spring semester 2011 - Computer Graphics, Post-Graduate practical course at Université Joseph Fourier.
- Spring semester 2010 - Computer Graphics, Post-Graduate practical course at Université Joseph Fourier.
Publications
Enhanced Laparoscopic Vision Improves Detection of Intraoperative Adverse Events During Laparoscopy
Innovation and Research in BioMedical engineering - 2020
Stereo-vision three-dimensional reconstruction of curvilinear structures imaged with a TEM
Ultramicroscopy - 2018
Tilt-less 3-D electron imaging and reconstruction of complex curvilinear structures
Nature - Scientific reports - 2017
Three-dimensional electron imaging and reconstruction of dislocations from a single acquisition
Microscopy Conference - 2017
Instruments Localisation and Identification for Laparoscopic Surgeries
MICAI Workshop - 2016
Three-dimensional electron imaging of dislocations from a single sample tilt
European Microscopy Congress - 2016
Progressive Shape Models
CVPR - Computer Vision and Patern Recognition - 2012
Flot de scène
Traitement du Signal, - 2012
Modèles progressifs de forme
CORESA - COmpression et REprésentation des Signaux Audiovisuels - 2012
Flot de scène à partir d'images couleur et de cartes de profondeur
RFIA - Reconnaissance des Formes et Intelligence Artificielle - 2012
Surface Flow from Visual Cues
VMV - Vision, Modeling and Visualization Workshop - 2011
Scene Flow from Depth and Color Images
BMVC - British Machine Vision Conference - 2011
Flot de surface à partir d'indices visuels
ORASIS - Congrès des jeunes chercheurs en vision par ordinateur - 2011