Industrial Applications with Large-Scale 3D Point Cloud Processing

Tutorial @ ICRA 2015 : 2015 IEEE International Conference on Robotics and Automation (ICRA)

Half-Day Tutorial.
Please bring your own laptop computer and install our Virtual machine before.
Alternatively, work natively (much faster) by installing our software - instructions are given here.

Please click on this link to download software and data sets.

Recently, 3D point cloud processing became popular in the robotics community due to the appearance of the Microsoft Kinect camera. The emerging field of 3D laser scanning advances, too. The objective of the tutorial is to present state of the art 3D scanning technologies, recent developments for efficient processing of large scale 3D point clouds, and how it is applied in industrial applications. To achieve these objectives, the tutorial is structured along industrial applications examples. To gain hands-on experiences on processing large-scale 3D point clouds, we interleave presentations, software demonstrations and software trials. Participants are requested to bring their Linux, MacOS or Windows laptops.

The increasing need for rapid characterization and quantification of complex environments has created challenges for data analysis. This critical need comes from many important areas, including industrial automation, architecture, agriculture, and the construction or maintenance of tunnels and mines. 3D CAD models are necessary for factory design, facility management, urban and regional planning. While precise 3D data is nowadays reliably obtained by using professional 3D laser scanners, efficient, large-scale 3D point clouds processing software must be available for these applications. The tutorial will focus on 3D point cloud processing in the following areas:

  • 3D underground mine mapping
  • Factory digitalization
  • Tunnel inspection
  • Urban 3D modeling
  • Workpiece verification
  • ...

 

Organizers

Prof. Dr. Andreas Nüchter
University of Würzburg
Am Hubland, 97074 Würzburg, Germany
www7.informatik.uni-wuerzburg.de/mitarbeiter/nuechter/
andreas@nuechti.de

Dr. Thomas Wiemann
University of Osnabrück
Albrechtstr. 28, 49069 Osnabrück, Germany
www.informatik.uni-osnabrueck.de/twiemann/
twiemann@uni-osnabrueck.de

Content

Light Detection and Ranging (LiDAR) is a technology for three-dimensional measurement of object surfaces. Structured light laser range finders such as the sensors from the company PrimeSense project a patter into the scene, detect it with a camera and calculate depth information using triangulation. Time of flight sensors such as the new kinect or safety laser scanners from automation (SICK) emit either a laser pulse and measure returned light, or they modulate laser light and measure the phase shift. Using these means 3D point clouds with millions to billions 3D points can be measured.

3D underground mine mapping. 

A GraphSLAM (simultaneous localization and mapping using pose graphs) variant, namely 6D SLAM [1] for 3D point clouds, enables us to digitize large-scale underground environments using a survey grade 3D laser scanner. The SLAM variant uses efficient data structures and iterated point matching with relaxation to yield high accurate results for the mining industry.

Figure: 3D model of an underground mine in Chile (Codelco) (left and middle) and robot used for automatic
acquisition (right).

Factory digitalization.

A SLAM variant, namely semi-rigid 6D GraphSLAM for 3D point clouds, enables us to digitize large-scale factory environments without the need to stop production. The acquired 3D point clouds are precise and suitable to detect objects that collide with items moved along the production line [2]. This application involves the need for efficient collision detection. Automotive industry will serve as the example in this part of the tutorial.

Figure: 3D model of a Volkswagen factory with a simulated car (left). Results of the collision detection and
calculated depth of penetration (right).

Tunnel inspection.

By law, tunnels such as railway tunnels, have to inspected in a regular fashion, to ensure structural integrity and for example that no objects such a roots cause problems. To this end, mobile mapping is used (cf. Fig. below) for inspection. Besides introducing mobile mapping, challenges in data analysis are discussed.

Figure: Left: The Optech Lynx Mobile Mapper on the back of a train wagon. Middel: Terrestrial 3D scanning
with a survey grade VZ-400 scanner. Right: Aligned train wagon (yellow) inside the tunnel environment (gray)
and trajectory (red).

Urban 3D Modeling.

Airborne LiDAR, kinematic laser scanning and terrestrial laser scanning are methods that are commercial available. In addition SFM (Structure from Motion) is available to obtain 3D models, e.g., from UAVs. City planning and inventories are the applications in this field.

Figure: 3D point cloud acquired by terrestrial laser scanning with reflectances (grey values), thermal infor-
mation (middle) and color information (church on the right). The thermal information has been acquired with
a co-calibrated thermal camera, while the color information has been matched using a point cloud and SfM
(bundler).

Workpiece verification.

To check, whether a piece produced meets the specification an acquired point cloud of the item is fitted and compared to with a CAD model. Due to NDAs with our industrial partners, we present our verification procedure on an robotic application example, namely semantic mapping of different furniture pieces based on Kinect data: knowledge about the scanned entities is available for reasoning in some knowledge base with an associated reasoning engine. We explore, how background knowledge gives a boost to model-based object recognition in large-scale 3D laser data, cf. Figures below. Obviously, instead of furniture, pre-produced parts can be checked before assembly.

Figure: Matching of several CAD chair models against a noisy input cloud with increasing rotational and
displacement error from top to bottom row.

Figure: Matching of furniture models to cluttered and noisy point clouds. Pose estimations are gained via
comparison of these models with a polygonal reconstruction of the point cloud data.

 

References

[1] D. Borrmann, J. Elseberg, K. Lingemann, A. N üchter, and J. Hertzberg. Globally consistent 3d mapping with scan matching. Journal Robotics and Autonomous Systems (JRAS), 56(2):130–142, February 2008.

[2] J. Elseberg, D. Borrmann, J. Schauer, A. Nüchter, D. Koriath, and U. Rautenberg. A sensor skid for precise 3d modeling of production lines. In ISPRS Ann. Photogramm. Remote Sens. Spatial Inf. Sci., II-5, Proceedings of ISPRS Technical Commission V Symposium “Close-range imaging, ranging and applications”, pages 117–122, June 2014.