Jean-Philippe Mercier, PhDApplied Scientist

Je mène et livre des projets d'IA appliquée en vision par ordinateur, en robotique, en IA générative, en optimisation et en développement logiciel sur mesure. Selon le projet, je suis le contributeur technique principal, le responsable d'une petite équipe technique ou le superviseur de développeurs et de chercheurs.

Systèmes d'IA et logiciels bâtis pour RDDC Valcartier, en poste chez Aerex Avionique, combinant vision par ordinateur, développement logiciel, intégration de capteurs et matériel expérimental. Travaux de détection, de suivi et de classification d'objets, avec des systèmes de perception visible, infrarouge et hyperspectrale. Modèles exécutés sur serveurs, PC et systèmes embarqués dans des prototypes de recherche. Coauteur d'un article OTAN.

Thèse « Deep learning for object detection in robotic grasping contexts », sous la supervision de Philippe Giguère (Université Laval) et la codirection d'Abdeslam Boularias (Rutgers University).

Saisie robotisée par démonstration : reconnaissance et appariement d'objets par nuages de points 3D, puis contrôle d'un bras Kinova Jaco pour reproduire la prise. Stage de fin de maîtrise chez Robotiq, où le projet a inspiré un produit de l'entreprise. Prix d'excellence Pierre Marchand (2014).

N. Lapointe, K. Laurent, K. Voyer, J.-P. Mercier, M. R. Karimi Dastjerdi, A. Grenier, C. Croeser, I. Van Der Wat

B. Debaque, H. Perreault, J.-P. Mercier, M.-A. Drouin et al.

Fusing thermal and visible images is a recurring challenge in computer vision, especially when the images of the two modalities are not well registered. This registration problem is traditionally solved by matching descriptors and depends on the richness and discriminating power of the representation. Ensuring that detected features are dense and uniformly distributed is not necessarily guaranteed. More recently, machine learning methods addressed the issue of visible to visible matching, but few address the multi-modality setting. In this paper, we propose to address the special case of thermal-visible image registration with small baseline parallax correction. Our deep homography model is evaluated on an open thermal and visible dataset with two training settings, unsupervised and supervised. Results demonstrate the feasibility of the approach, and performances comparison to state-of-the-art models is evaluated.

A. Coluccia et al. (dont J.-P. Mercier)

Adopting effective techniques to automatically detect and identify small drones is a very compelling need for a number of different stakeholders in both the public and private sectors. This work presents three different original approaches that competed in a grand challenge on the “Drone vs. Bird” detection problem. The goal is to detect one or more drones appearing at some time point in video sequences where birds and other distractor objects may be also present, together with motion in background or foreground. Algorithms should raise an alarm and provide a position estimate only when a drone is present, while not issuing alarms on birds, nor being confused by the rest of the scene. In particular, three original approaches based on different deep learning strategies are proposed and compared on a real-world dataset provided by a consortium of universities and research centers, under the 2020 edition of the Drone vs. Bird Detection Challenge. Results show that there is a range in difficulty among different test sequences, depending on the size and the shape visibility of the drone in the sequence, while sequences recorded by a moving camera and very distant drones are the most challenging ones. The performance comparison reveals that the different approaches perform somewhat complementary, in terms of correct detection rate, false alarm rate, and average precision.

J.-P. Mercier, M. Garon, P. Giguère, J.-F. Lalonde

Much of the focus in the object detection literature has been on the problem of identifying the bounding box of a particular class of object in an image. Yet, in contexts such as robotics and augmented reality, it is often necessary to find a specific object instance, a unique toy or a custom industrial part for example, rather than a generic object class. Here, applications can require a rapid shift from one object instance to another, thus requiring fast turnaround which affords little-to-no training time. What is more, gathering a dataset and training a model for every new object instance to be detected can be an expensive and time-consuming process. In this context, we propose a generic 2D object instance detection approach that uses example viewpoints of the target object at test time to retrieve its 2D location in RGB images, without requiring any additional training (i.e. fine-tuning) step. To this end, we present an end-to-end architecture that extracts global and local information of the object from its viewpoints. The global information is used to tune early filters in the backbone while local viewpoints are correlated with the input image. Our method offers an improvement of almost 30 mAP over the previous template matching methods on the challenging Occluded Linemod dataset (overall mAP of 50.7). Our experiments also show that our single generic model (not trained on any of the test objects) yields detection results that are on par with approaches that are trained specifically on the target objects.

G. Gagné, M. Breton, Y. de Villers, J.-P. Mercier, V. Paquin, J. Giesbrecht

J.-P. Mercier

Dans la dernière décennie, les approches basées sur les réseaux de neurones convolutionnels sont devenus les standards pour la plupart des tâches en vision numérique. Alors qu'une grande partie des méthodes classiques de vision étaient basées sur des règles et algorithmes, les réseaux de neurones sont optimisés directement à partir de données d'entraînement qui sont étiquetées pour la tâche voulue. En pratique, il peut être difficile d'obtenir une quantité suffisante de données d'entraînement ou d'interpréter les prédictions faites par les réseaux. Également, le processus d'entraînement doit être recommencé pour chaque nouvelle tâche ou ensemble d'objets. Au final, bien que très performantes, les solutions basées sur des réseaux de neurones peuvent être difficiles à mettre en place. Dans cette thèse, nous proposons des stratégies visant à contourner ou solutionner en partie ces limitations en contexte de détection d'instances d'objets. Premièrement, nous proposons d'utiliser une approche en cascade consistant à utiliser un réseau de neurone comme pré-filtrage d'une méthode standard de « template matching ». Cette façon de faire nous permet d'améliorer les performances de la méthode de « template matching » tout en gardant son interprétabilité. Deuxièmement, nous proposons une autre approche en cascade. Dans ce cas, nous proposons d'utiliser un réseau faiblement supervisé pour générer des images de probabilité afin d'inférer la position de chaque objet. Cela permet de simplifier le processus d'entraînement et diminuer le nombre d'images d'entraînement nécessaires pour obtenir de bonnes performances. Finalement, nous proposons une architecture de réseau de neurones ainsi qu'une procédure d'entraînement permettant de généraliser un détecteur d'objets à des objets qui ne sont pas vus par le réseau lors de l'entraînement. Notre approche supprime donc la nécessité de réentraîner le réseau de neurones pour chaque nouvel objet.

J.-P. Mercier, C. Mitash, P. Giguère, A. Boularias

This work proposes a process for efficiently training a point-wise object detector that enables localizing objects and computing their 6D poses in cluttered and occluded scenes. Accurate pose estimation is typically a requirement for robust robotic grasping and manipulation of objects placed in cluttered, tight environments, such as a shelf with multiple objects. To minimize the human labor required for annotation, the proposed object detector is first trained in simulation by using automatically annotated synthetic images. We then show that the performance of the detector can be substantially improved by using a small set of weakly annotated real images, where a human provides only a list of objects present in each image without indicating the location of the objects. To close the gap between real and synthetic images, we adopt a domain adaptation approach through adversarial training. The detector resulting from this training process can be used to localize objects by using its per-object activation maps. In this work, we use the activation maps to guide the search of 6D poses of objects. Our proposed approach is evaluated on several publicly available datasets for pose estimation. We also evaluated our model on classification and localization in unsupervised and semi-supervised settings. The results clearly indicate that this approach could provide an efficient way toward fully automating the training process of computer vision models used in robotics.

V. Paquin, M.-A. Lacasse, Y. Drolet-Mihelic, J.-P. Mercier

A robotic arm mounted camera system allows an end-user to begin using the camera for object recognition without involving a robotics specialist. Automated object model calibration is performed under conditions of variable robotic arm pose dependent feature recognition of an object. The user can then teach the system to perform tasks on the object using the calibrated model. The camera's body can have parallel top and bottom sides and adapted to be fastened to a robotic arm end and to an end effector with its image sensor and optics extending sideways in the body, and it can include an illumination source for lighting a field of view.

J.-P. Mercier, L. Trottier, P. Giguère, B. Chaib-draa

Pick-and-place is an important task in robotic manipulation. In industry, template-matching approaches are often used to provide the level of precision required to locate an object to be picked. However, if a robotic workstation is to handle numerous objects, brute-force template-matching becomes expensive, and is subject to notoriously hard-to-tune thresholds. In this paper, we explore the use of Deep Learning methods to speed up traditional methods such as template matching. In particular, we employed a Single Shot Detection (SSD) and a Residual Network (ResNet) for object detection and classification. Classification scores allows the re-ranking of objects so that template matching is performed in order of likelihood. Tests on a dataset containing 10 industrial objects demonstrated the validity of our approach, by getting an average ranking of 1.37 for the object of interest. Moreover, we tested our approach on the standard Pose dataset which contains 15 objects and got an average ranking of 1.99. Because SSD and ResNet operates essentially in constant time in a Graphics Processor Unit, our approach is able to reach near-constant time execution. We also compared the F1 scores of LINE-2D, a state-of-the-art template matching method, using different strategies (including our own) and the results show that our method is competitive to a brute-force template matching approach. Coupled with near-constant time execution, it therefore opens up the possibility for performing template matching for databases containing hundreds of objects.

F.-M. De Rainville, J.-P. Mercier, C. Gagné, P. Giguère, D. Laurendeau

This paper proposes a complete system for robotic sensor placement in initially unknown arbitrary three-dimensional environments. The system uses a novel approach for computing the quality of acquisition of a mobile sensor group in such environments. The quality of acquisition is based on a geometric model of a camera which allows accurate sensor models and simple occlusion computation. The proposed system combines this new metric with a global derivative-free optimization algorithm to find simultaneously the number of sensors and their configuration to sense accordingly the environment. The presented framework compares favourably with current techniques working in two-dimensional environments. Furthermore, simulation and experimental results demonstrate the ability of the system to cope with full three-dimensional environments, a domain still unexplored by previous methods.