Deep Learning · Computer Vision · Robotics

Jean-Philippe Mercier, PhDApplied Scientist

I build perception systems that help robots understand, localize and interact with the real world

About

I am a co-founder and AI scientist at NQB, where I deliver custom AI projects in vision, AI agents, and operations research for clients in defence, healthcare, manufacturing, and SaaS. I hold a PhD in computer science from Université Laval and work at the intersection of deep learning, computer vision, and robotics. My research has focused on object instance detection, 6D pose estimation in cluttered scenes, domain adaptation between simulation and real data, as well as 3D perception and autonomous sensor placement. Before NQB, I co-developed at Robotiq an industrial pick-and-place product for Universal Robots — a technology on which I am a co-inventor of the patent — and contributed at Aerex Avionics to nearly ten defence contracts in visible, infrared, and hyperspectral imagery. My work connects learning models to the real-world constraints of robots and sensors.

« From pixels to physical interaction. »

Perception Modalities09

RGB & RGB-D
Thermal & Visible Imaging
Infrared & Hyperspectral Imaging
3D Perception & Point Clouds
Laser Ranging & Leddar
AR Tags & Calibration
3D Models & CAD
Video & Object Tracking
Electro-Optical Sensors

Technical Expertise09

Deep Learning
Object & Instance Detection
6D Pose Estimation
Domain Adaptation
Sim-to-Real & Synthetic Data
Weak Supervision
AI Agents & Automation
Operations Research
ROS · PyTorch · C++

Experience

2021 — Present

Co-founder & AI Scientist

NQB.AI

Delivery of custom AI projects in vision, NLP, generative AI, signal processing, and operations research for clients in defence, healthcare, manufacturing, and SaaS. Depending on the mandate: sole contributor, technical team lead, or supervisor of developers and researchers.

PythonPyTorchComputer VisionAI Agents

2020 — 2021

Computer Vision & AI Specialist

Aerex Avionics · DRDC

Contributed to nearly ten defence contracts for the Department of National Defence: object detection, tracking, and classification in visible, infrared, and hyperspectral imagery. Deployed models on server, PC, and edge devices. Co-author of a NATO paper.

Deep LearningInfrared ImagingHyperspectral ImagingEdge Deployment

2015 — 2019

Vision Scientist

Robotiq

Co-developed an industrial pick-and-place product for Universal Robots, from camera firmware to object detection. Co-inventor of the patent "Robotic arm camera system and method" (CA2977077A1).

Computer VisionRoboticsEmbeddedC++

2018

Visiting PhD Researcher

Rutgers University

Object localization and 6D pose estimation from simulation and weakly labeled real images. Work published at ICRA 2019.

Sim-to-Real6D PoseDeep Learning

2013 — 2021

PhD & MSc in Computer Science

Université Laval

PhD thesis "Deep learning for object detection in robotic grasping contexts"; MSc on 3D vision for semi-automatic robotic grasping. Pierre Marchand Excellence Award (2014).

Deep Learning3D VisionRoboticsResearch

Selected Publications

2022

Thermal and Visible Image Registration Using Deep Homography

FUSION · International Conference on Information Fusion

B. Debaque, H. Perreault, J.-P. Mercier, M.-A. Drouin, et al.

2021

Drone vs. Bird Detection: Deep Learning Algorithms and Results from a Grand Challenge

Sensors · MDPI

A. Coluccia et al. (incl. J.-P. Mercier)

2021

Deep Template-based Object Instance Detection

WACV · Winter Conference on Applications of Computer Vision

J.-P. Mercier, M. Garon, P. Giguère, J.-F. Lalonde

2019

Learning Object Localization and 6D Pose Estimation from Simulation and Weakly Labeled Real Images

ICRA · International Conference on Robotics and Automation

J.-P. Mercier, C. Mitash, P. Giguère, A. Boularias

2017

Deep Object Ranking for Template Matching

WACV · Winter Conference on Applications of Computer Vision

J.-P. Mercier, L. Trottier, P. Giguère, B. Chaib-draa

2017

Robotic Arm Camera System and Method

Canadian Patent · CA2977077A1

V. Paquin, M.-A. Lacasse, Y. Drolet-Mihelic, J.-P. Mercier

2015

Multisensor Placement in 3D Environments via Visibility Estimation and Derivative-Free Optimization

ICRA · International Conference on Robotics and Automation

F.-M. De Rainville, J.-P. Mercier, C. Gagné, P. Giguère, D. Laurendeau

All publications on Google Scholar

Projects

Showing 16 / 16

Template-based Object Instance Detection (DTOID)

2D detection of a specific object instance from a handful of reference views, with no retraining for each new object — nearly 30 mAP better than template matching on Occluded LINEMOD. Published at WACV 2021, public code (jpmerc/DTOID).

Few-ShotObject DetectionPyTorch

6D Pose Estimation with Minimal Real Annotations

Object localization and 6D pose estimation in cluttered, occluded scenes: trained in simulation, then domain-adapted with as few as ten weakly labeled real images per object. Published at ICRA 2019 (with Rutgers University).

Sim-to-Real6D PoseDomain Adaptation

Deep Template Ranking for Pick-and-Place

Deep networks that quickly rank the most relevant templates for a target object, speeding up template matching in robotic pick-and-place systems. Published at WACV 2017.

Template MatchingDeep LearningRobotics

Industrial Pick-and-Place for Collaborative Robots

Co-developed at Robotiq a vision product for industrial pick-and-place on Universal Robots: camera firmware, calibration, object detection, and robot integration. Patented technology (CA2977077A1) deployed on factory floors.

Computer VisionRoboticsC++Embedded

Jaco–Kinect Calibration & 3D Perception

Calibration of the transform between a Kinova Jaco arm and a Kinect camera using AR tags, with a documented ROS/RViz workflow (jpmerc/perception3d).

ROSCalibration3D Perception

Autonomous Multisensor Placement in 3D

Simultaneous optimization of the number and position of sensor robots in unknown, occluded 3D environments, using a visibility model and derivative-free optimization (CMA-ES, cooperative coevolution). Published at ICRA 2015.

OptimizationCMA-ESROS

Real-Time 3D Tracking with AR Tags

Extension of ar_track_alvar: robust camera-to-tag transform computed with RANSAC over 3D corners instead of solvePnP, with a public video demo (jpmerc/3D_Tracking).

RANSAC3D TrackingROS

Leddar Sensor Tooling

Qt configuration interface for a Leddar range finder — oversampling, accumulation, LED intensity, thresholds — and integration of the sensor into a ROS stack (jpmerc/leddarUI, jpmerc/leddartech).

C++QtROS

Semi-automatic Robotic Grasping with 3D Vision

Master's research on 3D vision for semi-automatic robotic grasping. Pierre Marchand Excellence Award (2014).

3D VisionRoboticsGrasping

Object Detection & Tracking for Defence

Nearly ten contracts for the Department of National Defence at Aerex Avionics: object detection, tracking, and classification in visible, infrared, and hyperspectral imagery, deployed on server, PC, and edge devices. Co-author of a NATO paper.

Deep LearningInfrared ImagingHyperspectral ImagingEdge Deployment

Drone vs. Bird — International Grand Challenge

Competed in the international "Drone-vs-Bird" grand challenge: detecting drones in video sequences in the presence of birds and other distractors, while avoiding false alarms. Results published in the journal Sensors.

Deep LearningObject DetectionVideo

Thermal-Visible Registration via Deep Homography

Alignment of thermal and visible images through deep-network homography estimation, for multi-sensor fusion. Published at FUSION 2022.

Image RegistrationInfrared ImagingDeep Learning

Automated Assessment of Mining Infrastructure

Machine learning system developed at NQB to assess the degradation severity of mine shaft structural components from inspection data. Paper submitted in 2026.

Computer VisionDeep LearningInspection

AI Agents in Production — Quotes, Email, and Voice

Designed and deployed production AI agents for NQB clients: automated quote generation, automated email replies for reservations, and a voice sales agent.

AI AgentsLLMNLP

Multi-site COVID-19 Wave Forecasting

Time series forecasting to anticipate COVID-19 waves across multiple sites, combining statistical methods and deep learning.

Time SeriesDeep LearningForecasting

Route & Schedule Optimization

Applied operations research at NQB: vehicle routing and schedule construction under real-world business constraints.

Operations ResearchOptimizationPython