Rossi, A.P., Unnithan, V., Torrese, P., Borrmann, D., Nüchter, A., Lauterbach, H., Ortenzi, G., Jährig, T., Sohl, F.: Augmented field Geology and Geophysics for Planetary Analogues (Poster).Proceedings of the EGU General Assembly 2018, Pico session: Planetary geobiological analogs for Mars and beyond: Field, lab and simulations. , Vienna, Austria (2018).
Martell, A., Lauterbach, H.A., Schilling, K., Nüchter, A.: Benchmarking Structure from Motion Algorithms of Urban Environments with Applications to Reconnaissance in Search and Rescue Scenarios.Proceedings of the 16th IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '18). p. 1--7. , Philadelphia, PA, USA (2018).
Pfitzner, C., May, S., Nüchter, A.: Body Weight Estimation for Dose-Finding and Health Monitoring of Lying, Standing and Walking Patients Based on RGB-D Data.Sensors.18, (2018).
This paper describes the estimation of the body weight of a person in front of an RGB-D camera. A survey of different methods for body weight estimation based on depth sensors is given. First, an estimation of people standing in front of a camera is presented. Second, an approach based on a stream of depth images is used to obtain the body weight of a person walking towards a sensor. The algorithm first extracts features from a point cloud and forwards them to an artificial neural network (ANN) to obtain an estimation of body weight. Besides the algorithm for the estimation, this paper further presents an open-access dataset based on measurements from a trauma room in a hospital as well as data from visitors of a public event. In total, the dataset contains 439 measurements. The article illustrates the efficiency of the approach with experiments with persons lying down in a hospital, standing persons, and walking persons. Applicable scenarios for the presented algorithm are body weight-related dosing of emergency patients.
Nüchter, A., Bleier, M., Schauer, J., Janotta, P.: Continuous-Time SLAM --- Improving Google’s Cartographer 3D Mapping. In: Remondino, F., Georgopoulos, A., Gonzalez-Aguilera, D., and Agrafiotis, P. (eds.) Latest Developments in Reality-Based 3D Surveying and Modelling. p. 53--73. MDPI, Basel, Switzerland (2018).
This paper shows how to use the result of Google’s simultaneous localization and mapping (SLAM) solution, called Cartographer, to bootstrap a continuous-time SLAM algorithm that was developed by the authors and presented in previous publications. The presented approach optimizes the consistency of the global point cloud, and thus improves on Google’s results. Algorithms and data from Google are used as input for the continuous-time SLAM software. In preceding work, the continuous-time SLAM was successfully applied to a similar backpack system which delivers consistent 3D point clouds even in the absence of an IMU. Continuous-time SLAM means that the trajectory of a mobile mapping system is treated in a semi-rigid fashion, i.e., the trajectory is deformed to yield a consistent 3D point cloud of the measured environment.
van der Lucht, J., Bleier, M., Leutert, F., Schilling, K., Nüchter, A.: Korrektur der Brechung an der Wasseroberfläche beim triangulationsbasierten 3D-Laserscannen.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2018, Jade Hochschule. p. 87--102 (2018).
Die vorliegende Arbeit beschäftigt sich mit der Korrektur der Brechung an der Wasseroberfläche beim triangulationsbasierten 3D Laserscannen. Hierzu wurde eine Methode entwickelt, die es ermöglicht, mit einem Structured Light (SL) System 3D-Daten von teilweise mit Wasser bedeckten Strukturen anzufertigen und die dabei entstehende Brechung an der Wasseroberfläche zu korrigieren. Diese wurde anschließend in einem Versuchsaufbau evaluiert. Der Scanner wurde dabei an einem KUKA KR-16 Manipulatorarm befestigt, was die Möglichkeit bietet den Scanner gleichmäßig, definiert und wiederholbar zu bewegen. Die Bewegungen des Scanners wurden dabei durch ein externes Trackingsystem erfasst. Ebenfalls wurde bei diesen Versuchen der Einfluss verschiedener Einstrahlwinkel betrachtet. Zu diesem Zweck wurde der Scanner in verschiedenen Winkeln relativ zur Wasseroberfläche bewegt. Durch die entwickelte Methode konnten Fehler durch die Brechung an der Wasseroberfläche erfolgreich korrigiert werden. Außerdem konnte die Lage der Wasseroberfläche ohne externe Markierungen aus den 3D-Daten bestimmt werden.
Almeida, J., Martins, A., Almeida, C., Dias, A., Matias, B., Ferreira, A., Jorge, P., Martins, R., Bleier, M., Nüchter, A., Pidgeon, J., Kapusniak, S., Silva, E.: Positioning. Navigation and Awareness of the !VAMOS! Underwater Robotic Mining System.IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '18). p. 1527--1533 (2018).
This paper presents the positioning, navigation and awareness (PNA) system developed for the Underwater Robotic Mining System of the !VAMOS! project [1]. It describes the main components of the !VAMOS! system, the PNA sensors in each of those components, the global architecture of the PNA system, and its main subsystems: Position and Navigation, Realtime Mine Modeling, 3D Virtual reality HMI and Real-time grade system. General results and lessons learn during the first mining field trial in Lee Moor, Devon, UK during the months of September and October 2017 are presented.
Lauterbach, H.A., Nüchter, A.: Preliminary Results on Instantaneous UAV-Based 3D Mapping for Rescue Applications.Proceedings of the 16th IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '18). p. 1--2. , Philadelphia, PA, USA (2018).
This report presents a novel approach to generate a 3D map with an UAV while flying over a disaster scene with the aim to present the map instantaneously to the operator and the rescue workers. Our approach extends the well-known ICP algorithm.
Borrmann, D., Jörissen, S., Nüchter, A.: RADLER - A RADial LasER scanning device.Proceedings of the 16th International Symposium of Experimental Robotics (ISER '18). , Buenos Aires, Argentina (2018).
Mobile scanning, i.e., the practice of mounting laser scanners on moving platforms is an efficient way to acquire accurate and dense 3D point clouds of outdoor environments for urban and regional planning and architecture. The mobile scenario puts high requirements on the accuracy of the calibration of the measurement system, as small calibration inaccuracies lead to large errors in the resulting point cloud. We propose a novel algorithm for the calibration of a mobile scanning system that estimates the calibration parameters for all sensor components simultaneously without relying on additional hardware. We evaluate the calibration algorithm on several real world data sets where ground truth is available via an accurate geodetic model.
Torrese, P., Rossi, A.P., Unnithan, V., Borrmann, D., Lauterbach, H., Ortenzi, G., Jährig, T., Pozzobon, R., Sauro, F., Santagata, T., Nüchter, A., Sohl, F.: Reconstructing the subsurface of planetary volcanic analogues: ERT imaging of Lanzarote lava tubes complemented with drone stereogrammetry, surface and in-cave LiDAR and seismic investigations (Poster).Proceedings of the EGU General Assembly 2018, Pico session: Planetary geobiological analogs for Mars and beyond: Field, lab and simulations. , Vienna, Austria (2018).
Schauer, J., Nüchter, A.: Removing non-static objects from 3D laser scan data.ISPRS Journal of Photogrammetry and Remote Sensing (JPRS).143,15--38 (2018).
For the purpose of visualization and further post-processing of 3D point cloud data, it is often desirable to remove moving objects from a given data set. Common examples for these moving objects are pedestrians, bicycles and motor vehicles in outdoor scans or manufactured goods and employees in indoor scans of factories. We present a new change detection method which is able to partition the points of multiple registered 3D scans into two sets: points belonging to stationary (static) objects and points belonging to moving (dynamic) objects. Our approach does not require any object detection or tracking the movement of objects over time. Instead, we traverse a voxel grid to find differences in volumetric occupancy for “explicit” change detection. Our main contribution is the introduction of the concept of “point shadows” and how to efficiently compute them. Without them, using voxel grids for explicit change detection is known to suffer from a high number of false positives when applied to terrestrial scan data. Our solution achieves similar quantitative results in terms of F1-score as competing methods while at the same time being faster.
van der Lucht, J., Bleier, M., Leutert, F., Schilling, K., Nüchter, A.: Structured-light based 3D laser scanning of semi-submerged structures.Proceedings of the Technical Commision II Mid-term Symposium ``Towards Photogrammetry 2020''. p. 287--294. , Riva del Garda, Italy (2018).
In this work we look at 3D acquisition of semi-submerged structures with a triangulation based underwater laser scanning system. The motivation is that we want to simultaneously capture data above and below water to create a consistent model without any gaps. The employed structured light scanner consist of a machine vision camera and a green line laser. In order to reconstruct precise surface models of the object it is necessary to model and correct for the refraction of the laser line and camera rays at the water-air boundary. We derive a geometric model for the refraction at the air-water interface and propose a method for correcting the scans. Furthermore, we show how the water surface is directly estimated from sensor data. The approach is verified using scans captured with an industrial manipulator to achieve reproducable scanner trajectories with different incident angles. We show that the proposed method is effective for refractive correction and that it can be applied directly to the raw sensor data without requiring any external markers or targets.
Schauer, J., Nüchter, A.: The Peopleremover --- Removing Dynamic Objects From 3-D Point Cloud Data by Traversing a Voxel Occupancy Grid.IEEE Robotics and Automation Letters (RAL).3,1679--1686 (2018).
Even though it would be desirable for most postprocessing purposes to obtain a point cloud without moving objects in it, it is often impractical or downright impossible to free a scene from all nonstatic clutter. Outdoor environments contain pedestrians, bicycles, and motor vehicles which cannot easily be stopped from entering the sensor range and indoor environments like factory production lines cannot be evacuated due to production losses during the time of the scan. In this letter, we present a solution to this problem that we call the "peopleremover." Given a registered set of 3-D point clouds, we build a regular voxel occupancy grid and then traverse it along the lines of sight between the sensor and the measured points to find the differences in volumetric occupancy between the scans. Our approach works for scan slices from mobile mapping as well as for the more general scenario of terrestrial scan data. The result is a clean point cloud free of dynamic objects.
Lehtola, V., Kaartinen, H., Nüchter, A.: Autonomous 3D Modelling of Indoor Spaces.GIM International.31,20--23 (2017).
Mobile scanning can be an equally accurate yet more cost-effective solution than traditional terrestrial laser scanning done with tripods. To succeed, however, mobile scanners not only require a suitable combination of sensors, but also reliable and continuous knowledge about where the scanners are located and the direction in which they are pointing during scanning. There are multiple ways to achieve this, which has led to the development of various scientific and commercial solutions. This article compares several mobile scanning solutions for 3D modelling of indoor spaces and highlights their strengths and weaknesses.
Leung, K.Y.K., Lühr, D., Houshiar, H., Inostroza, F., Borrmann, D., Adams, M., Nüchter, A., del Solar, J.R.: Chilean underground mine dataset.International Journal of Robotics Research (IJRR).36,16--23 (2017).
This article presents a robotic dataset collected from the largest underground copper mine in the world. The sensor measurements from a 3D scanning lidar, a 2D radar, and stereo cameras were recorded from an approximately two kilometer traverse of a production-active tunnel. The equipment used and the data collection process is discussed in detail, along with the format of the data. This dataset is suitable for research in robotic navigation, as well as simultaneous localization and mapping. The download instructions are available at the following website http://dataset.amtc.cl.
Lehtola, V., Kaartinen, H., Nüchter, A., Kaijaluoto, R., Kukko, A., Litkey, P., Honkavaara, E., Rosnell, T., Vaaja, M., Virtanen, J.-P., et al.,: Comparison of the Selected State-Of-The-Art 3D Indoor Scanning and Point Cloud Generation Methods.Remote Sensing.796 (2017).
Accurate three-dimensional (3D) data from indoor spaces are of high importance for various applications in construction, indoor navigation and real estate management. Mobile scanning techniques are offering an efficient way to produce point clouds, but with a lower accuracy than the traditional terrestrial laser scanning (TLS). In this paper, we first tackle the problem of how the quality of a point cloud should be rigorously evaluated. Previous evaluations typically operate on some point cloud subset, using a manually-given length scale, which would perhaps describe the ranging precision or the properties of the environment. Instead, the metrics that we propose perform the quality evaluation to the full point cloud and over all of the length scales, revealing the method precision along with some possible problems related to the point clouds, such as outliers, over-completeness and misregistration. The proposed methods are used to evaluate the end product point clouds of some of the latest methods. In detail, point clouds are obtained from five commercial indoor mapping systems, Matterport, NavVis, Zebedee, Stencil and Leica Pegasus: Backpack, and three research prototypes, Aalto VILMA, FGI Slammer and the Würzburg backpack. These are compared against survey-grade TLS point clouds captured from three distinct test sites that each have different properties. Based on the presented experimental findings, we discuss the properties of the proposed metrics and the strengths and weaknesses of the above mapping systems and then suggest directions for future research.
Koch, R., May, S., Nüchter, A.: Detection and Purging of Specular Reflective and Transparent Object Influences in 3D Range Measurements.Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 377--384. , Nafplio, Greece (2017).
3D laser scanners are favoured sensors for mapping in mobile service robotics at indoor and outdoor applications, since they deliver precise measurements at a wide scanning range. The resulting maps are detailed since they have a high resolution. Based on these maps robots navigate through rough terrain, fulfil advanced manipulation, and inspection tasks. In case of specular reflective and transparent objects, e.g., mirrors, windows, shiny metals, the laser measurements get corrupted. Based on the type of object and the incident angle of the incoming laser beam there are three results possible: a measurement point on the object plane, a measurement behind the object plane, and a measurement of a reflected object. It is important to detect such situations to be able to handle these corrupted points. This paper describes why it is difficult to distinguish between specular reflective and transparent surfaces. It presents a 3D- Reflection-Pre-Filter Approach to identify specular reflective and transparent objects in point clouds of a multi-echo laser scanner. Furthermore, it filters point clouds from influences of such objects and extract the object properties for further investigations. Based on an Iterative-Closest-Point-algorithm reflective objects are identified. Object surfaces and points behind surfaces are masked according to their location. Finally, the processed point cloud is forwarded to a mapping module. Furthermore, the object surface corners and the type of the surface is broadcasted. Four experiments demonstrate the usability of the 3D-Reflection-Pre-Filter. The first experiment was made in a empty room containing a mirror, the second experiment was made in a stairway containing a glass door, the third experiment was made in a empty room containing two mirrors, the fourth experiment was made in an office room containing a mirror. This paper demonstrate that for single scans the detection of specular reflective and transparent objects in 3D is possible. It is more reliable in 3D as in 2D. Nevertheless, collect the data of multiple scans and post-filter them as soon as the object was bypassed should pursued. This is why future work concentrates on implementing a post-filter module. Besides, it is the aim to improve the discrimination between specular reflective and transparent objects.
Schauer, J., Nüchter, A.: Digitizing automotive production lines without interrupting assembly operations through an automatic voxel-based removal of moving objects.Proceedings of the 13th IEEE International Conference on Control and Automation (ICCA '17). p. 701--706. , Ohrid, Macedonia (2017).
We present an efficient method to partition a point cloud gathered through kinematic laser scanning into static and dynamic points. The presented algorithm utilizes a voxel grid data structure and uses a ray intersection test to mark voxels as dynamic. The algorithm does not require any ego-motion estimations, computationally expensive object recognition or tracking of moving objects over time. It is easy to implement and can be executed on many cores in parallel. We show the viability of this approach by applying our algorithm to a dataset that we gathered by mounting a FARO Focus3D Laser scanner onto a skid which was then sent along a production line for consumer car chassis in a factory of the Volkswagen corporation. Since factory operators are interested in acquiring digital models of their production lines without suspending factory operations, the resulting point cloud will contain many dynamic objects like humans or other car bodies. We show how our algorithm is able to successfully remove these dynamic objects from the resulting point cloud with minimal errors. Our implementation is published under a free license as part of 3DTK.
Koch, R., May, S., Nüchter, A.: Effective Distinction Of Transparent And Specular Reflective Objects In Point Clouds Of A Multi-Echo Laser Scanner.Proceedings of the 18th IEEE International Conference on Advanced Robotics (ICAR '17). p. 566--571. , Hong Kong, China (2017).
A favoured sensor for mapping is a 3D laser scanner since it allows a wide scanning range, precise measurements, and is usable indoor and outdoor. Hence, a mapping module delivers detailed and high resolution maps which makes it possible to navigate safely. Difficulties result from transparent and specular reflective objects which cause erroneous and dubious measurements. At such objects, based on the incident angle, measurements result from the object surface, an object behind the transparent surface, or an object mirrored with respect to the reflective surface. This paper describes an enhanced Pre-Filter-Module to distinguish between these cases. Two experiments demonstrate the usability and show that for single scans the identification of mentioned objects in 3D is possible. The first experiment was made in an empty room with a mirror. The second experiment was made in a stairway which contains a glass door. Further, results show that a discrimination between a specular reflective and a transparent object is possible. Especially for transparent objects the detected size is restricted to the incident angle. That is why future work concentrates on implementing a post-filter module. Gained experience shows that collecting the data of multiple scans and postprocess them as soon as the object was bypassed will improve the map.
Koch, B., Leblebici, R., Martell, A., Jörissen, S., Schilling, K., Nüchter, A.: Evaluating continuous-time SLAM using a predefined trajectory provided by a robotic arm.Proceedings of the ISPRS Geospatial Week 2017, Laserscanning 2017. p. 17--23. , Wuhan, China (2017).
Recently published approaches to SLAM algorithms process laser sensor measurements and output a map as a point cloud of the environment. Often the actual precision of the map remains unclear, since SLAMalgorithms apply local improvements to the resulting map. Unfortunately, it is not trivial to compare the performance of SLAMalgorithms objectively, especially without an accurate ground truth. This paper presents a novel benchmarking technique that allows to compare a precise map generated with an accurate ground truth trajectory to a map with a manipulated trajectory which was distorted by different forms of noise. The accurate ground truth is acquired by mounting a laser scanner on an industrial robotic arm. The robotic arm is moved on a predefined path while the position and orientation of the end-effector tool are monitored. During this process the 2D profile measurements of the laser scanner are recorded in six degrees of freedom and afterwards used to generate a precise point cloud of the test environment. For benchmarking, an offline continuous-time SLAM algorithm is subsequently applied to remove the inserted distortions. Finally, it is shown that the manipulated point cloud is reversible to its previous state and is slightly improved compared to the original version, since small errors that came into account by imprecise assumptions, sensor noise and calibration errors are removed as well.
Pfitzner, C., May, S., Nüchter, A.: Evaluation of Features from RGB-D Data for Human Body Weight Estimation.Proceedings of the 20th World Congress of the International Federation of Automatic Control (IFAC WC '17). , Toulouse, France (2017).
Body weight is a crucial parameter when it comes to drug or radiation dosing. In case of emergency treatment time is short so that physicians estimate the body weight by the visual appearance of a patient. Further, visual body weight estimation might be a feature for person identification. This paper presents the anthropometric feature extraction from RGB-D sensor data, recorded from frontal view. The features are forwarded to an artificial neural network for weight estimation. Experiments with 233 people demonstrate the capability of different features for body weight estimation. To prove robustness against sensor modalities, a structured light sensor is used, as well as a time-of-flight sensor. An additional experiment including temperature features from a thermal camera improves the body weight estimation beyond.
Koch, R., May, S., Murmann, P., Nüchter, A.: Identification of Transparent and Specular Reflective Material in Laser Scans to Discriminate Affected Measurements for Faultless Robotic SLAM.Journal of Robotics and Autonomous Systems (JRAS).87,296--312 (2017).
Mapping with laser scanners is the state-of-the-art method applied in service, industrial, medical, and rescue robotics. Although a lot of research has been done, maps still suffer from interferences caused by transparent and specular reflective objects. Glass, mirrors, shiny or translucent surfaces cause erroneous measurements depending on the incident angle of the laser beam. In past experiments the Mirror Detector Approach was implemented to determine such measurements with a multi-echo laser scanner. Recognition values are based on their differences in recorded measurements in regard to the distance of the echoes. This paper describes the research to distinguish between reflective and transparent objects. The implemented Mirror Detector was specifically modified for recognition of said objects for which four experiments were conducted; one experiment to show the map of the original Mirror Detector; two experiments to investigate intensity characteristics based on angle, distance, and material; and one experiment to show an applied discrimination with the extended version of the Mirror Detector, the Reflection Classifier Approach. To verify the results, a comparison with existing models was performed. This study showed that shiny metals, like aluminium, etc., provide significant characteristics, while mirrors are to be characterized by a mixed model of glass and shiny metal. Transparent objects turned out to be challenging because their appearance in the sensor data strongly depends on the background. Nevertheless, these experiments show that discrimination of transparent and reflective materials based on the reflected intensity is possible and feasible.
Nüchter, A., Bleier, M., Schauer, J., Janotta, P.: Improving Google's Cartographer 3D Mapping by Continuous-Time SLAM.Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 543--549. , Nafplio, Greece (2017).
This paper shows how to use the result of Google's SLAM solution, called Cartographer, to bootstrap our continuous-time SLAM algorithm. The presented approach optimizes the consistency of the global point cloud, and thus improves on Google’s results. We use the algorithms and data from Google as input for our continuous-time SLAM software. We also successfully applied our software to a similar backpack system which delivers consistent 3D point clouds even in absence of an IMU.
Bleier, M., Nüchter, A.: Low-cost 3D Laser Scanning in Air or Water Using Self-calibrating Structured Light.Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 105--112. , Nafplio, Greece (2017).
In-situ calibration of structured light scanners in underwater environments is time-consuming and complicated. This paper presents a self-calibrating line laser scanning system, which enables the creation of dense 3D models with a single fixed camera and a freely moving hand-held cross line laser projector. The proposed approach exploits geometric constraints, such as coplanarities, to recover the depth information and is applicable without any prior knowledge of the position and orientation of the laser projector. By employing an off-the-shelf underwater camera and a waterproof housing with high power line lasers an affordable 3D scanning solution can be built. In experiments the performance of the proposed technique is studied and compared with 3D reconstruction using explicit calibration. We demonstrate that the scanning system can be applied to above-the-water as well as underwater scenes.
Chen, L., Fan, L., Xie, G., Huang, K., Nüchter, A.: Moving-Object Detection From Consecutive Stereo Pairs Using Slanted Plane Smoothing.IEEE Transactions on Intelligent Transportation Systems.18,1--10 (2017).
Detecting moving objects is of great importance for autonomous unmanned vehicle systems, and a challenging task especially in complex dynamic environments. This paper proposes a novel approach for the detection of moving objects and the estimation of their motion states using consecutive stereo image pairs on mobile platforms. First, we use a variant of the semi-global matching algorithm to compute initial disparity maps. Second, assisted by the initial disparities, boundaries in the image segmentation produced by simple linear iterative clustering are classified into coplanar, hinge, and occlusion. Moving points are obtained during ego-motion estimation by a modified random sample consensus) algorithm without resorting to time-consuming dense optical flow. Finally, the moving objects are extracted by merging superpixels according to the boundary types and their movements. The proposed method is accelerated on the GPU at 20 frames per second. The data which we use for testing and benchmarking is released, thus completing similar data sets. It includes 812 image pairs and 924 moving objects with ground truth for better algorithms evaluation. Experimental results demonstrate that the proposed method achieves competitive results in terms of moving-object detection and their motion state estimation in challenging urban scenarios.
Koch, R., Böttcher, L., Jahrsdörfer, M., Maier, J., Trommer, M., May, S., Nüchter, A.: Out of lab calibration of a rotating 2D scanner for 3D mapping.Proceedings of the SPIE optical metrology, Videometrics, Range Imaging, and Applications. pp. 10332 - 10332 - 8. , Munich, Germany (2017).
Mapping is an essential task in mobile robotics. To fulfil advanced navigation and manipulation tasks a 3D representation of the environment is required. Applying stereo cameras or Time-of-flight cameras (TOF cameras) are one way to archive this requirement. Unfortunately, they suffer from drawbacks which makes it difficult to map properly. Therefore, costly 3D laser scanners are applied. An inexpensive way to build a 3D representation is to use a 2D laser scanner and rotate the scan plane around an additional axis. A 3D point cloud acquired with such a custom device consists of multiple 2D line scans. Therefore the scanner pose of each line scan need to be determined as well as parameters resulting from a calibration to generate a 3D point cloud. Using external sensor systems are a common method to determine these calibration parameters. This is costly and difficult when the robot needs to be calibrated outside the lab. Thus, this work presents a calibration method applied on a rotating 2D laser scanner. It uses a hardware setup to identify the required parameters for calibration. This hardware setup is light, small, and easy to transport. Hence, an out of lab calibration is possible. Additional a theoretical model was created to test the algorithm and analyse impact of the scanner accuracy. The hardware components of the 3D scanner system are an HOKUYO UTM-30LX-EW 2D laser scanner, a Dynamixel servo-motor, and a control unit. The calibration system consists of an hemisphere. In the inner of the hemisphere a circular plate is mounted. The algorithm needs to be provided with a dataset of a single rotation from the laser scanner. To achieve a proper calibration result the scanner needs to be located in the middle of the hemisphere. By means of geometric formulas the algorithms determine the individual deviations of the placed laser scanner. In order to minimize errors, the algorithm solves the formulas in an iterative process. First, the calibration algorithm was tested with an ideal hemisphere model created in Matlab. Second, laser scanner was mounted differently, the scanner position and the rotation axis was modified. In doing so, every deviation, was compared with the algorithm results. Several measurement settings were tested repeatedly with the 3D scanner system and the calibration system. The results show that the length accuracy of the laser scanner is most critical. It influences the required size of the hemisphere and the calibration accuracy.
Bleier, M., Dias, A., Ferreira, A., Pidgeon, J., Almeida, J., Silva, E., Schilling, K., Nüchter, A.: Real-time 3D Mine Modelling in the !VAMOS! Project. In: Buxton, M. and Benndorf, J. (eds.) Real Time Mining. p. 91--102. Technische Universität Bergakademie Freiberg, Institut für Markscheidewesen und Geodäsie, Amsterdam, The Netherlands (2017).
The project Viable Alternative Mine Operating System (\textexclamdown VAMOS!) develops a new safe, clean and low visibility mining technique for excavating raw materials from submerged inland mines. During operations, the perception data of the mining vehicle can only be communicated to the operator via a computer interface. In order to assist remote control and facilitate assessing risks a detailed view of the mining process below the water surface is necessary. This paper presents approaches to real-time 3D reconstruction of the mining environment for immersive data visualisation in a virtual reality environment to provide advanced spatial awareness. From the raw survey data a more consistent 3D model is created using post-processing techniques based on a continuous-time simultaneous localization and mapping (SLAM) solution. Signed distance function (SDF) based mapping is employed to fuse the measurements from multiple views into a single representation and reduce sensor noise. Results of the proposed techniques are demonstrated on a dataset captured in an sub- merged inland mine.
Bleier, M., Dias, A., Ferreira, A., Pidgeon, J., Almeida, J.M., Silva, E., Schilling, K., Nüchter, A.: Signed Distance Function Based Surface Reconstruction of a Submerged Inland Mine Using Continuous-Time SLAM.Proceedings of the 20th World Congress of the International Federation of Automatic Control (IFAC WC '17). , Toulouse, France (2017).
The planning of mining operations in water filled open-pit mines requires detailed bathymetry to create a mine plan and assess the involved risks. This paper presents post- processing techniques for creating an improved 3D model from a survey carried out using an autonomous surface vehicle with a multibeam sonar and a GPS/INS navigation system. Inconsistencies of the created point cloud as a result of calibration errors or GPS signal loss are corrected using a continuous-time simultaneous localization and mapping (SLAM) solution. Signed distance function based mapping is employed to fuse the measurements from multiple runs into a consistent representation and reduce sensor noise. From the signed distance function model we reconstruct a 3D surface mesh. We use this terrain model to establish a virtual reality scene for immersive data visualization of the mining operations for testing and planing during development. Results of the proposed approach are demonstrated on a dataset captured in an abandoned submerged inland mine.
Struckmeier, O., Borrmann, D., Nüchter, A.: Teach-In für die 3D-Scan Akquise mit einem Roboter.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2017, Jade Hochschule. p. 108--119 (2017).
Die 3D-Erfassung einer kompletten Umgebung mittels 3D-Laserscanner stellt abhängig von der zu scannenden Umgebung und dem Scanner einen hohen Zeitaufwand für einen menschlichen Operator dar. Der Scanner muss an die einzelnen Scanpositionen bewegt und dort verortet werden. Mit steigender Qualität der Messung nimmt zudem die Dauer der einzelnen Scanvorgänge zu. In diesem Beitrag wird ein Teach-In Ansatz vorgestellt und evaluiert, der den manuelle n Vorgang verkürzt. Dabei führt ein Roboter die vom Vermesser geplanten und eingespeicherten zeitintensiven Schritte automatisch durch und entlastet somit den Bediener.
Nüchter, A., Elseberg, J., Janotta, P.: Towards Mobile Mapping of Underground Mines. In: Buxton, M. and Benndorf, J. (eds.) Real Time Mining. p. 27--38. Technische Universität Bergakademie Freiberg, Institut für Markscheidewesen und Geodäsie, Amsterdam, The Netherlands (2017).
Mobile laser scanning systems automate the acquisition of 3D point clouds of environments. The mapping systems are commonly mounted on cars or ships. This paper presents a flexible mapping solution mounted on an underground vehicle, that is able to map underground mines in 3D in walking speeds. A clever choice of hard- and software enables the system to generate 3D maps without using GPS (global positioning system) information and without relying on highly expensive IMU (inertial measurement unit) systems.
Lauterbach, H.A., Borrmann, D., Nüchter, A.: Towards Radiometrical Alignment of 3D Point Clouds.Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 419--424. , Nafplio, Greece (2017).
3D laser scanners are typically not able to collect color information. Therefore coloring is often done by projecting photos of an additional camera to the 3D scans. The capturing process is time consuming and therefore prone to changes in the environment. The appearance of the colored point cloud is mainly effected by changes of lighting conditions and corresponding camera settings. In case of panorama images these exposure variations are typically corrected by radiometrical aligning the input images to each other. In this paper we adopt existing methods for panorama optimization in order to correct the coloring of point clouds. Therefore corresponding pixels from overlapping images are selected by using geometrically closest points of the registered 3D scans and their neighboring pixels in the images. The dynamic range of images in raw format allows for correction of large exposure differences. Two experiments demonstrate the abilities of the approach.
Bleier, M., Nüchter, A.: Towards robust self-calibration for handheld 3D line laser scanning.Proceedings of LowCost 3D 2017. p. 31--36. , Hamburg, Germany (2017).
This paper studies self-calibration of a structured light system, which reconstructs 3D information using video from a static consumer camera and a handheld cross line laser projector. Intersections between the individual laser curves and geometric constraints on the relative position of the laser planes are exploited to achieve dense 3D reconstruction. This is possible without any prior knowledge of the movement of the projector. However, inaccurrately extracted laser lines introduce noise in the detected intersection positions and therefore distort the reconstruction result. Furthermore, when scanning objects with specular reflections, such as glossy painted or metalic surfaces, the reflections are often extracted from the camera image as erroneous laser curves. In this paper we investiagte how robust estimates of the parameters of the laser planes can be obtained despite of noisy detections.
Borrmann, D., Leutert, F., Maurovic, I., Seder, M., Nüchter, A.: Automatische Grundrisserstellung mittels Laserscandaten.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2016, Jade Hochschule. p. 108--119 (2016).
In den letzten Jahren haben sich Laserscanner zum Stand der Technik bei der Erstellung von Gebäudemodellen entwickelt. Im Gegensatz zur vergleichsweise kurzen Aufnahmezeit bei der eigentlichen Erfassung der Umgebung mit dem Scanner nimmt die Nachbearbeitung der Daten einen deutlich höheren Zeitanteil ein. Eine manuelle Datenanalyse ist zeitaufwändig und fehleranfällig. Dies erhöht den Bedarf an automatischen Verfahren zur quantitativen Erfassung und Charakterisierung von Umgebungen. In diesem Beitrag präsentieren wir eine automatische Grundrisserstellung basierend auf dem 3D Toolkit (3DTK - http://www.threedtk.de). Nach der autonomen Akquise mit einem mobilen Roboter werden die Daten mittels des iterativen Verfahrens der nächsten Punkte (engl. Iterative Closest Point (ICP)) registriert. Anschließend erfolgt eine automatische Vektorisierung eines 2D-Schnitts der Umgebung basierend auf Verfahren aus der Bildverarbeitung. Der so erzeugte Grundriss dient als Grundlage für eine semantische Karte.
Nüchter, A.: Effiziente Speicherung großer Punktwolken -- Datenstrukturen für Algorithmen für mobile und terrestrische Laserscansysteme.Terrestrisches Laserscanning (TLS 2016) Beiträge zum 154. DVW-Seminar am 28. und 29. November in Fulda. p. 105--120. , Fulda, Germany (2016).
Lehtola, V.V., Virtanen, J.-P., Rönnholm, P., Nüchter, A.: LOCALIZATION CORRECTIONS FOR MOBILE LASER SCANNER USING LOCAL SUPPORT-BASED OUTLIER FILTERING.Proceedings of the ISPRS Congress 2016. p. 81--88. , Prague, Czech Republic (2016).
Following the pioneering work introduced in [Lehtola et al., ISPRS J. Photogramm. Remote Sens. 99, 2015, pp. 25–29], we extend the state-of-the-art intrinsic localization solution for a single two-dimensional (2D) laser scanner from one into (quasi) three dimensions (3D). By intrinsic localization, we mean that no external sensors are used to localize the scanner, such as inertial measurement devices (IMU) or global navigation satellite systems (GNSS). Specifically, the proposed method builds on a novel concept of local support-based filtering of outliers, which enables the use of six degrees-of-freedom (DoF) simultaneous localization and mapping (SLAM) for the purpose of enacting appropriate trajectory corrections into the previous one-dimensional solution. Moreover, the local support-based filtering concept is platform independent, and is therefore likely to be widely generalizable. The here presented overall method is yet limited into quasi-3D by its inability to recover trajectories with steep curvature, but in the future, it may be further extended into full 3D.
Lehtola, V.V., Virtanen, J.-P., Vaaja, M.T., Hyyppä, H., Nüchter, A.: Localization of a Mobile Laser Scanner via Dimensional Reduction.ISPRS Journal of Photogrammetry and Remote Sensing (JPRS).121,48--59 (2016).
We extend the concept of intrinsic localization from a theoretical one-dimensional (1D) solution onto a 2D manifold that is embedded in a 3D space, and then recover the full six degrees of freedom for a mobile laser scanner with a simultaneous localization and mapping algorithm (SLAM). By intrinsic localization, we mean that no reference coordinate system, such as global navigation satellite system (GNSS), nor inertial measurement unit (IMU) are used. Experiments are conducted with a 2D laser scanner mounted on a rolling prototype platform, VILMA. The concept offers potential in being extendable to other wheeled platforms.
Koch, P., May, S., Schmidpeter, M., Kühn, M., Pfitzner, C., Merkl, C., Koch, R., Fees, M., Martin, J., Nüchter, A.: Multi-Robot Localization and Mapping based on Signed Distance Functions.Journal of Intelligent and Robotic Systems.83,409--428 (2016).
This publication describes a 2D Simultaneous Localization and Mapping approach applicable to multiple mobile robots. The presented strategy uses data of 2D LIDAR sensors to build a dynamic representation based on Signed Distance Functions. Novelties of the approach are a joint map built in parallel instead of occasional merging of smaller maps and the limited drift localization which requires no loop closure detection. A multi-threaded software architecture performs registration and data integration in parallel allowing for drift-reduced pose estimation of multiple robots. Experiments are provided demonstrating the application with single and multiple robot mapping using simulated data, public accessible recorded data, two actual robots operating in a comparably large area as well as a deployment of these units at the Robocup rescue league.
Pfitzner, C., May, S., Nüchter, A.: Neural Network-based Visual Body Weight Estimation for Drug Dosage Finding.Proceedings of the SPIE 9784, Medical Imaging 2016: Image Processing. , San Diego, CA, USA (2016).
Body weight adapted drug dosages are important for emergency treatments. This paper describes an improved body weight estimation approach for emergency patients in a trauma room, based on images from a RGBD sensor and a thermal camera. The improvements are archived by several extensions: The sensor fusion of RGBD and thermal camera eases filtering and segmentation of the patient's body from the background. Robustness and accuracy is gained by an artificial neural network (ANN), which considers features from the sensors as input to calculate the patient's body weight, e.g. the patient's volume, surface and shape parameters. The ANN is trained offline with 30 percent of the patients data. Preliminary experiments with 69 real patients show an accuracy close to 90 percent for a threshold of ten percent relative error in real body estimation. Results are compared to the patient's self estimation, a physician's guess and an anthropometric method: If the patient is knowledgeable it is the best possibility for body weight adapted drug dosages with 97 percent accuracy. The treating physicians and the anthropometric estimation achieve an accuracy of approximately 70 percent. The here presented approach gets an accuracy of nearly 90 percent and would be the best solution if a patient can not provide his own body weight and can not be weighted on a scale. These preliminary results demonstrate a sufficient approach for an upcoming clinical trial with 1,000 patients for body weight estimation.
Schauer, J., Bedkowski, J., Majek, K., Nüchter, A.: Performance comparison between state-of-the-art point-cloud based collision detection approaches on the CPU and GPU.Proceedings of the 4th IFAC Symposium on Telematics Applications (TA '13). p. 54--59. , Porto Alegre, Brazil (2016).
We present two fundamentally different approaches to detect collisions between two point clouds and compare their performance on multiple datasets. A collision between points happens if they are closer to each other than a given threshold radius. One approach utilizes the main CPU with a k-d tree datastructure to efficiently carry out fixed range searches around points in 3D while the other mainly executes on a GPU using a regular grid decomposition technique implemented in the CUDA framework. We will show how massively parallel 3D range searches on a grid based datastructure on a GPU performs similarly well as a tree based approach on the CPU with orders of magnitude less parallelization. We also show how each method scales with varying input sizes and how they perform differently well depending on the spatial structure of the input data.
Konolige, K., Nüchter, A.: Range Sensors. In: Siciliano, B. and Khatib, O. (eds.) Handbook of Robotics. p. 783--810. Springer (2016).
Leutert, F., Borrmann, D., Schilling, K., Nüchter, A.: Spatial Projection of Thermal Data for Visual Inspection.Proceedings of the 14th IEEE International Conference on Control, Automation, Robotics and Vision (ICARCV '16). p. 1--6. , Phuket, Thailand (2016).
Since the advent of thermal imaging, devices with a high optical resolution that use detector arrays to capture the emitted radiance in the thermal infrared range of an entire scene simultaneously have developed as a standard in monitoring energy related. They have had a huge impact on the building industry and in manifacturing, where they are commonly used to monitor proecces that require stable temperature conditions. As beneficial as contactless measurements are, the subsequent localization of points of interest in the environment is often difficult. To overcome this problem we propose a portable system that combines thermal imaging with Augmented Reality (AR). The idea of the approach is to project the gathered temperature information back into the scene to facilitate visual inspection.
Houshiar, H., Nüchter, A.: 3D Point Cloud Compression using Conventional Image Compression for Efficient Data Transmission.Proceedings of the XXV International Symposium on Information, Communication and Automation Technologies (ICAT '15). p. 1--8. IEEE Xplore, Sarajevo, Bosnia (2015).
Modern 3D laser scanners make it easy to collect large 3D point clouds. In this paper we present the use of conventional image based compression methods for 3D point clouds. We map the point cloud onto panorama images to encode the range, reflectance and color value for each point. An encoding method is presented to map the floating point measured ranges on to a three channel image. The image compression methods are used to compress the generated panorama images. We present the results of several lossless compression methods and the lossy JPEG on point cloud compression. Lossless compression methods are designed to retain the original data. On the other hand lossy compression methods sacrifice the details for higher compression ratio. This produces artefacts in the recovered point cloud data. We study the effects of these artefacts on encoded range data. A filtration process is presented for determination of range outliers from uncompressed point clouds.
Nüchter, A., Borrmann, D., Elseberg, J., Redondo, D.: A Backpack-Mounted 3D Mobile Scanning System.Allgemeine Vermessungs-Nachrichten (AVN), Special Issue MoLAS 2014.122,301--307 (2015).
Mobile laser scanning systems automate the acquisition of 3D point clouds of environments. The mapping systems are commonly mounted on cars or ships. This paper presents a mapping solution mounted on a backpack. A clever choice of hard- and software enables the system to generate 3D maps without using GPS (global positioning system) information and without relying on expensive IMU (inertial measurement unit) systems. Therefore, it enables flexible indoor mapping.
Nüchter, A., Borrmann, D., Koch, P., Kühn, M., May, S.: A Man-Portable, IMU-free Mobile Mapping.Proceedings of the ISPRS Geospatial Week 2015, Laserscanning 2015. p. 17--23. , La Grande Motte, France (2015).
Mobile mapping systems are commonly mounted on cars, ships and robots. The data is directly geo-referenced using GPS data and expensive IMU (inertial measurement systems). Driven by the need for flexible, indoor mapping systems we present an inexpensive mobile mapping solution that can be mounted on a backpack. It combines a horizontally mounted 2D profiler with a constantly spinning 3D laser scanner. The initial system featuring a low-cost MEMS IMU was revealed and demonstrated at MoLaS: Technology Workshop Mobile Laser Scanning at Fraunhofer IPM in Freiburg in November 2014. In this paper, we present an IMU-free solution.
Houshiar, H., Elseberg, J., Borrmann, D., Nüchter, A.: A Study of Projections for Key Point Based Registration of Panoramic Terrestrial 3D Laser Scans.Journal of Geo-spatial Information Science.18,11--31 (2015).
This paper surveys state-of-the-art image features and descriptors for the task of 3D scan registration based on panoramic reflectance images. As modern terrestrial laser scanners digitize their environment in a spherical way, the sphere has to be projected to a two-dimensional image. To this end, we evaluate the equirectangular, the cylindrical, the Mercator, the rectilinear, the Pannini, the stereographic, and the z-axis projection. We show that the Mercator and the Pannini projection outperform the other projection methods.
Houshiar, H., Borrmann, D., Elseberg, J., Nüchter, A., Winkler, S., Näth, F.: CASTLE3D -- A Computer Aided System for Labelling Archaeological Excavations in 3D.Proceedings of the XXV International CIPA Symposium. p. 111--118. , Taipei, Taiwan (2015).
Documentation of archaeological excavation sites with conventional methods and tools such as hand drawings, measuring tape and archaeological notes is time consuming. This process is prone to human errors and the quality of the documentation depends on the qualification of the archaeologist on site. Use of modern technology and methods in 3D surveying and 3D robotics facilitate and improve this process. Computer-aided systems and databases improve the documentation quality and increase the speed of data acquisition. 3D laser scanning is the state of the art in modelling archaeological excavation sites, historical sites and even entire cities or landscapes. Modern laser scanners are capable of data acquisition of up to 1 million points per second. This provides a very detailed 3D point cloud of the environment. 3D point clouds and 3D models of an excavation site provide a better representation of the environment for the archaeologist and for documentation. The point cloud can be used both for further studies on the excavation and for the presentation of results. This paper introduces a Computer aided system for labelling archaeological excavations in 3D (CASTLE3D). Consisting of a set of tools for recording and georeferencing the 3D data from an excavation site, CASTLE3D is a novel documentation approach in industrial archaeology. It provides a 2D and 3D visualisation of the data and an easy-to-use interface that enables the archaeologist to select regions of interest and to interact with the data in both representations. The 2D visualisation and a 3D orthogonal view of the data provide cuts of the environment that resemble the traditional hand drawings. The 3D perspective view gives a realistic view of the environment. CASTLE3D is designed as an easy-to-use on-site semantic mapping tool for archaeologists. Each project contains a predefined set of semantic information that can be used to label findings in the data. Multiple regions of interest can be joined under one label. Further information such as color, orientation and archaeological notes are added to the label to improve the documentation. The available 3D information allows for easy measurements in the data. The full 3D information of a region of interest can be segmented from the entire data. By joining this data from different georeferenced views the full 3D shape of findings is stored. All the generated documentation in CASTLE3D is exported to an XML format and serves as input for other systems and databases. Apart from presenting the functionalities of CASTLE3D we evaluate its documentation process in a sample project. For this purpose we export the data to the Adiuvabit database (http://adiuvabit.de) where more information is added for further analysis. The documentation process is compared to traditional documentation methods and it is shown how the automated system helps in accelerating the documentation process and decreases errors to a minimum.
Schauer, J., Nüchter, A.: Collision detection between point clouds using an efficient \($k$\)-d tree implementation.Journal Advanced Engineering Informatics (JAdvEI).29,440--458 (2015).
Context: An important task in civil engineering is the detection of collisions of a 3D model with an environment representation. Existing methods using the structure gauge provide an insufficient measure because the model either rotates or because the trajectory makes tight turns through narrow passages. This is the case in either automotive assembly lines or in narrow train tunnels. Objective: Given two point clouds, one of the environment and one of a model and a trajectory with six degrees of freedom along which the model moves through the environment, find all colliding points of the environment with the model within a certain clearance radius. Method: This paper presents two collision detection (CD) methods called kd-CD and kd-CD-simple and two penetration depth (PD) calculation methods called kd-PD and kd-PD-fast. All four methods are based on searches in a k-d tree representation of the environment. The creation of the k-d tree, its search methods and other features will be explained in the scope of their use to detect collisions and calculate depths of penetration. Results: The algorithms are benchmarked by moving the point cloud of a train wagon with 2.5 million points along the point cloud of a 1144m long train track through a narrow tunnel with overall 18.92 million points. Points where the wagon collides with the tunnel wall are visually highlighted with their penetration depth. With a safety margin of 5cm kd-PD-simple finds all colliding points on its trajectory which is sampled into 19,392 positions in 77s on a standard desktop machine of 1.6GHz. Conclusion: The presented methods for collision detection and penetration depth calculation are shown to solve problems for which the structure gauge is an insufficient measure. The underlying k-d tree is shown to be an effective data structure for the required look-up operations.
Borrmann, D., Houshiar, H., Elseberg, J., Nüchter, A., Näth, F., Winkler, S.: Das Castle3D Framework zur fortlaufenden semantischen 3D-Kartierung von archäologischen Ausgrabungsstätten.Allgemeine Vermessungs-Nachrichten (AVN).122,233--246 (2015).
Das 3D Laserscanning ist Stand der Technik bei der Modellierung archäologischer Ausgrabungsstätten, historischer Anlagen und sogar ganzer Städte oder Landschaften. Die Dokumentation der Befunde auf einer Ausgrabungsstätte ist eine wesentliche archäologische Aufgabe. Ein automatisiertes System würde diesen Prozess beschleunigen und die Anzahl der Fehler auf ein Minimum reduzieren. Dieser Beitrag stellt einen neuen Ansatz in der Dokumentation industrieller Archäologie vor durch die Entwicklung einer standardisierten Herangehensweise an die computerunterstützte Dokumentation einer archäologischen Ausgrabungsstätten. Ausserdem wird eine Reihe von Tools zur Erfassung und Registrierung von 3D-Daten auf Ausgrabungsstätten vorgestellt, die den Hauptbestandteil der Arbeitskette abdecken. Wir stellen ein effizientes Werkzeug zur Verfügung für die Visualisierung der erworbenen 3D-Punktwolken im 3D- und 2D-Modus. Der Hauptzweck dieser Software ist Archäologen ein einfach zu bedienendes Tool für die semantische Kartierung vor Ort zu bieten. Es enthält Funktionen für die Auswahl und Kennzeichnung von Funden. Jedes Label kann mit weiteren Informationen versehen werden. Diese Daten werden im XML-Format exportiert und dienen als Eingabe für andere Systeme und Datenbanken.
Koch, R., May, S., Koch, P., Kühn, M., Nüchter, A.: Detection of Specular Reflections in Range Measurements for Faultless Robotic SLAM.Proceedings of ROBOT'2015: Second Iberian Robotics Conference, Advances in Robotics, Volume 1. p. 133--145. Springer, Lisbon, Portugal (2015).
Laser scanners are state-of-the-art devices used for mapping in service, industry, medical and rescue robotics. Although a lot of work has been done in laser-based SLAM, maps still suffer from interferences caused by objects like glass, mirrors and shiny or translucent surfaces. Depending on the surface's reflectivity, a laser beam is deflected such that returned measurements provide wrong distance data. At certain positions phantom-like objects appear. This paper describes a specular reflectance detection approach applicable to the emerging technology of multi-echo laser scanners in order to identify and filter reflective objects. Two filter stages are implemented. The first filter reduces errors in current scans on the fly. A second filter evaluates a set of laser scans, triggered as soon as a reflective surface has been passed. This makes the reflective surface detection more robust and is used to refine the registered map. Experiments demonstrate the detection and elimination of reflection errors. They show improved localization and mapping in environments containing mirrors and large glass fronts is improved.
Al khawaldah, M., Nüchter, A.: Enhanced frontier-based exploration for indoor environment with multiple robots.Advanced Robotics.28,657--669 (2015).
n this paper, the exploration and map-building of unknown environment by a team of mobile robots is intensively investigated. A new exploration technique is proposed to increase the exploration efficiency. In particular, the new technique has two main objectives: firstly, it aims at reducing the exploration time and the traveled distance by reducing the overlap which takes place when a certain area in the environment is explored by more than one robot. To achieve this, a new procedure to assign the next target location for each individual robot is proposed. And secondly, it aims at reducing computations complexity required by target selection and path planning tasks. More importantly, the proposed technique obviates the need for environment segmentation complex procedures which is adopted in some previous important research works. The new technique is intensively tested with different environments. The results showed the effectiveness of the proposed technique.
Lauterbach, H.A., Borrmann, D., Hess, R., Eck, D., Schilling, K., Nüchter, A.: Evaluation of a Backpack-Mounted 3D Mobile Scanning System.Remote Sensing.7,13753--13781 (2015).
Recently, several backpack-mounted systems, also known as personal laser scanning systems, have been developed. They consist of laser scanners or cameras that are carried by a human operator to acquire measurements of the environment while walking. These systems were first designed to overcome the challenges of mapping indoor environments with doors and stairs. While the human operator inherently has the ability to open doors and to climb stairs, the flexible movements introduce irregularities of the trajectory to the system. To compete with other mapping systems, the accuracy of these systems has to be evaluated. In this paper, we present an extensive evaluation of our backpack mobile mapping system in indoor environments. It is shown that the system can deal with the normal human walking motion, but has problems with irregular jittering. Moreover, we demonstrate the applicability of the backpack in a suitable urban scenario.
Borrmann, D., Hess, R., Eck, D., Houshiar, H., Nüchter, A., Schilling, K.: Evaluation of Methods for Robotic Mapping of Cultural Heritage Sites.Proceedings of the 2th IFAC conference on Embedded Systems, Computer Intelligence and Telematics (CESCIT '15). p. 105--110. , Maribor, Slovenia (2015).
In archaeological studies the use of new technologies has moved into focus in the past years creating new challenges such as the processing of the massive amounts of data. In this paper we present steps and processes for smart 3D modelling of environments by use of the mobile robot Irma3D. A robot that is equipped with multiple sensors, most importantly a photo camera and a laser scanner, enables the automation of most of the processes, including data acquisition and registration. The robot was tested in the Würzburg Residence. Methods for automatic 3D color reconstructions of cultural heritage sites are evaluated in this paper.
Bedkowski, J., Majek, K., Majek, P., Musialik, P., Pelka, M., Nüchter, A.: Intelligent Mobile System for Improving Spatial Design Support and Security Inside Buildings.Mobile Networks and Applications.20,1--14 (2015).
This paper concerns an intelligent mobile application for spatial design support and security domain. Mobility has two aspects in our research: The first one is the usage of mobile robots for 3D mapping of urban areas and for performing some specific tasks. The second mobility aspect is related with a novel Software as a Ser- vice system that allows access to robotic functionalities and data over the Ethernet, thus we demonstrate the use of the novel NVIDIA GRID technology allowing to virtualize the graphic processing unit. We introduce Complex Shape Histogram, a core component of our artificial intelligence engine, used for classifying 3D point clouds with a Support Vector Machine. We use Complex Shape Histograms also for loop closing detection in the simultaneous localization and mapping algorithm. Our intelligent mobile system is built on top of the Qualitative Spatio-Temporal Representa- tion and Reasoning framework. This framework defines an ontology and a semantic model, which are used for building the intelligent mobile user interfaces. We show experiments demonstrating advantages of our approach. In addition, we test our prototypes in the field after the end-user case studies demonstrating a relevant contribution for future intelligent mobile systems that merge mobile robots with novel data centers.
Demisse, G., Borrmann, D., Nüchter, A.: Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency.Journal of Intelligent and Robotic Systems.77,55--72 (2015).
In recent years 3D models of buildings are used in maintenance and inspection, preservation, and other building related applications. However, the usage of these models is limited because most models are pure representations with no or little associated semantics. In this paper we present a pipeline of techniques used for interior interpretation, object detection, and adding energy related semantics to windows of a 3D thermal model. A sequence of algorithms is presented for building the fundamental semantics of a 3D model. Among other things, these algorithms enable the system to differentiate between objects in a room and objects that are part of the room, e.g. floor, windows. Subsequently, the thermal information is used to construct a stochastic mathematical model– namely Markov Random Field– of the temperature distribution of the detected windows. As a result, the MAP(Maximum a posteriori) framework is used to further label the windows as either open, closed or damaged based upon their temperature distribution. The experimental results showed the robustness of the techniques. Furthermore, a strategy to optimize the free parameters is described, in cases where there is a sample training dataset.
Pfitzner, C., May, S., Merkl, C., Breuer, L., Köhrmann, M., Braun, J., Dirauf, F., Nüchter, A.: Libra3D: Body Weight Estimation for Emergency Patients in Clinical Environment with a 3D Structured Light Sensor.Proceedings of the IEEE International Conference Robotics and Automation (ICRA '15). , Seattle, WA, USA (2015).
his paper describes the application of a weight estimation method for emergency patients in clinical environments. The approach applies established algorithms for point cloud processing and filtering to data from a low-cost, structured light sensor. A patient's volume is estimated on the basis of their visible front surface. The approach is currently being tested in the workflow of the emergency room at the Universitätsklinikum Erlangen, Germany. Preliminary results show the accuracy of the approach in relation to other conservative means of weight measurements, for example, by physicians and anthropometric measurements.
Käshammer, P., Nüchter, A.: Mirror Identification and Correction of 3D Point Clouds.Proceedings of the 6th ISPRS International Workshop 3D-ARCH 2015: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 109--114. , Avila, Spain (2015).
In terrestrial laser scanning (TLS), the surface geometry of objects is scanned by laser beams and recorded digitally. This produces a discrete set of scan points, commonly referred to as a point cloud. The coordinates of the scan points are determined by measuring the angles and the time-of-flight relative to the origin (scanner position). However, if it comes to mirror surfaces laser beams are fully reflected, due to the high reflectivity. Mirrors do not appear in the point cloud at all. Instead, for every reflected beam, a incorrect scan point is created behind the actual mirror plane. Consequently, problems arise in multiple derived application fields such as 3D virtual reconstruction of complex architectures. The paper presents a new approach to automatically detect framed rectangular mirrors with known dimensions and to correct the 3D point cloud, using the calculated mirror plane.
Koch, P., May, S., Schmidpeter, M., Kühn, M., Pfitzner, C., Merkl, C., Koch, R., Fees, M., Martin, J., Nüchter, A.: Multi-Robot Localization and Mapping based on Signed Distance Functions.Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC '15). p. 77--82. , Vila Real, Portugal (2015).
This publication describes a 2D Simultaneous Localization and Mapping approach applicable to multiple mobile robots. The presented strategy uses data of 2D LIDAR sensors to build a dynamic representation based on Signed Distance Functions. A multi-threaded software architecture performs registration and data integration in parallel allowing for drift-reduced pose estimation of multiple robots. Experiments are provided demonstrating the application with single and multiple robot mapping using simulated data, public accessible recorded data as well as two actual robots operating in a comparably large area.
Borrmann, D., Hess, R., Eck, D., Nüchter, A., Schilling, K.: Robotic Mapping of Cultural Heritage Sites.Proceedings of the 6th ISPRS International Workshop 3D-ARCH 2015: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 9--16. , Avila, Spain (2015).
In archaeological studies the use of new technologies has moved into focus in the past years creating new challenges such as the processing of the massive amounts of data. In this paper we present steps and processes for smart 3D modelling of environments by use of the mobile robot Irma3D. A robot that is equipped with multiple sensors, most importantly a photo camera and a laser scanner, enables the automation of most of the processes, including data acquisition and registration. The robot was tested in two scenarios, Ostia Antica and the Würzburg Residence. The paper describes the steps for creating 3D color reconstructions of these renown cultural heritage sites.
Gailis, J., Nüchter, A.: Towards Globally Consistent Scan Matching With Ground Truth Integration.Proceedings of the ISPRS International Conference on Photogrammetric Image Analysis (PIA '15). p. 59--64. , Munich, Germany (2015).
The scan matching based simultaneous localization and mapping method with six dimensional poses is capable of creating a three dimensional point cloud map of the environment, as well as estimating the six dimensional path that the vehicle has travelled. The essence of it is the registering and matching of sequentially acquired 3D laser scans, while moving along a path, in a common coordinate frame in order to provide 6D pose estimations at the respective positions, as well as create a three dimensional map of the environment. An approach that could drastically improve the reliability of acquired data is to integrate available ground truth information. This paper is about implementing such functionality as a contribution to 6D SLAM (simultaneous localization and mapping with 6 DoF) in the 3DTK – The 3D Toolkit software (Nüchter and Lingemann, 2011), as well as test the functionality of the implementation using real world datasets.
May, S., Koch, P., Koch, R., Merkl, C., Pfitzer, C., A.Nüchter,: A Gereralized 2D and 3D Multi-Sensor Data Integration Approach based on Signed Distance Functions for Multi-Modal Robotic Mapping.Proceedings of th 19th International Workshop on Vision, Modeling and Visualization (VMV '14). , Darmstadt, Germany (2014).
This paper describes a data integration approach for arbitrary 2D/3D depth sensing units exploiting assets of the signed distance function. The underlying framework generalizes the KinectFusion approach with an objectoriented model respecting different sensor modalities. For instance, measurements of 2D/3D laser range finders and RGB-D cameras can be integrated into the same representation. Exemplary, an environment is reconstructed with a 3D laser range finder, while adding fine details from objects of interest by closer inspection with an RGB-D sensor. A typical application of this approach is the exploration in rescue environments, where large-scale mapping is performed on the basis of long-range laser range finders while hollows are inspected with lightweight sensors attached to a manipulator arm.
Borrmann, D., Nüchter, A., DJakulovi'c, M., Maurovi'c, I., Petrovi'c, I., Osmankovi'c, D., Velagi'c, J.: A mobile robot based system for fully automated thermal 3D mapping.Journal Advanced Engineering Informatics (JAdvEI).28,425--440 (2014).
It is hard to imagine living in a building without electricity and a heating or cooling system these days. Factories and data centers are equally dependent on a continuous functioning of these systems. As beneficial as this development is for our daily life, the consequences of a failure are critical. Malfunctioning power supplies or temperature regulation systems can cause the close-down of an entire factory or data center. Heat and air conditioning losses in buildings lead to a large waste of the limited energy resources and pollute the environment unnecessarily. To detect these flaws as quickly as possible and to prevent the negative consequences constant monitoring of power lines and heat sources is necessary. To this end, we propose a fully automatic system that creates 3D thermal models of indoor environments. The proposed system consists of a mobile platform that is equipped with a 3D laser scanner, an RGB camera and a thermal camera. A novel 3D exploration algorithm ensures efficient data collection that covers the entire scene. The data from all sensors collected at different positions is joined into one common reference frame using calibration and scan matching. In the post-processing step a model is built and points of interest are automatically detected. A viewer is presented that aids experts in analyzing the heat flow and localizing and identifying heat leaks. Results are shown that demonstrate the functionality of the system.
Elseberg, J., Borrmann, D., Schauer, J., Nüchter, A., Koriath, D., Rautenberg, U.: A sensor skid for precise 3D modeling of production lines.Proceedings of the Commision V Symposium Close-range imaging, ranging and applications. p. 117--122. , Riva del Garda, Italy (2014).
Motivated by the increasing need of rapid characterization of environments in 3D, we designed and built a sensor skid that automates the work of an operator of terrestrial laser scanners. The system combines terrestrial laser scanning with kinematic laser scanning and uses a novel semi-rigid SLAMmethod. It enables us to digitize factory environments without the need to stop production. The acquired 3D point clouds are precise and suitable to detect objects that collide with items moved along the production line.
Li, Q., Chen, L., Li, M., Shaw, S.-L., Nüchter, A.: A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios.IEEE Transactions on Vehicular Technology.63,540--555 (2014).
Autonomous vehicle navigation is challenging since various types of road scenarios in real urban environments have to be considered, particularly when only perception sensors are used, without position information. This paper presents a novel real-time optimal-drivable-region and lane detection system for autonomous driving based on the fusion of light detection and ranging (LIDAR) and vision data. Our system uses a multisensory scheme to cover the most drivable areas in front of a vehicle. We propose a feature-level fusion method for the LIDAR and vision data and an optimal selection strategy for detecting the best drivable region. Then, a conditional lane detection algorithm is selectively executed depending on the automatic classification of the optimal drivable region. Our system successfully handles both structured and unstructured roads. The results of several experiments are provided to demonstrate the reliability, effectiveness, and robustness of the system.
Nüchter, A., Rusu, R.B., Holz, D., Munoz, D.: Editorial: Semantic Perception, Mapping and Exploration.Journal of Robotics and Autonomous Systems (JRAS), Special Issue on Semantic Perception, Mapping and Exploration.62,1--2 (2014).
Schauer, J., Nüchter, A.: Efficient Point Cloud Collision Detection and Analysis in a Tunnel Environment using Kinematic Laser Scanning and k-d Tree Search.Proceedings of the Photogrammetric Computer Vision (PCV '14). p. 289--295. , Zürich, Switzerland (2014).
Measuring the structure gauge of tunnels and other narrow passages has so far been the only way to evaluate whether large vehicles can pass through them. But especially for very long vehicles like train wagons and their cargo, the structure gauge is an insufficient measure because the center part of the vehicle between two bogies will inevitably leave the swept volume of its cross section when moving along any other trajectory than a straight line perpendicular to its cross section. In addition, the vehicle as well as the cargo must keep a minimum safety margin from the environment at all points of its trajectory. This paper explores an automated method to check for possible collisions of a model represented by a 3D point cloud moving through the 3D point cloud of an environment. We were given environment data of a train track through a narrow tunnel where simply relying on the structure gauge would indicate that a given wagon would pass through without any collision even though in reality, the train wagon would collide with the inner tunnel wall inside a sharp turn of the tracks. The k-d tree based collision detection method presented in this paper is able to correctly highlight these collisions and indicate the penetration depth of each colliding point of the environment into the model of the train wagon. It can be generalized for any setup where two static point clouds have to be tested for intersection along a trajectory.
Houshiar, H., Borrmann, D., Nüchter, A.: Fortlaufende semantische 3D-Kartierung von archäologischen Ausgrabungsstätten.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2014, Jade Hochschule. p. 268--277 (2014).
Das 3D Laserscanning ist Stand der Technik bei der Modellierung archäologischer Ausgrabungsstätten, historischer Anlagen und sogar ganzer Städte oder Landschaften. Die Dokumentation der Befunde auf einer Ausgrabungsstätte ist eine wesentliche archäologische Aufgabe. Ein automatisiertes System würde diesen Prozess beschleunigen und die Anzahl der Fehler auf ein Minimum reduzieren. Dieser Beitrag stellt einen neuen Ansatz in der Dokumentation industrieller Archäologie vor. Er besteht aus einer Reihe von Tools zur Erfassung und Registrierung von 3D-Daten auf Ausgrabungsstätten. Wir stellen ein effizientes Werkzeug zur Verfügung für die Visualisierung der erworbenen 3D-Punktwolken im 3D- und 2D-Modus. Der Hauptzweck dieser Software ist Archäologen ein einfach zu bedienendes Tool für die semantische Kartierung vor Ort zu bieten. Es enthält Funktionen für die Auswahl und Kennzeichnung von Funden. Jedes Label kann mit weiteren Informationen versehen werden. Diese Daten werden im XML-Format exportiert und dienen als Eingabe für andere Systeme und Datenbanken.
Bruder, G., Steinicke, F., Nüchter, A.: Immersive Point Cloud Virtual Environments (Poster).Proceedings of IEEE Symposium on 3D User Interfaces 3DUI Proceedings of IEEE Symposium on 3D User Interfaces (3DUI '14). p. 161--162 (2014).
Today's three-dimensional (3D) virtual environments (VEs) are usually based on textured polygonal 3D models, which represent the appearance and geometry of the virtual world. However, some application domains require other graphical paradigms, which are currently not adequately addressed by 3D user interfaces. We introduce a novel approach for a technical human-robot telepresence setup that allows a human observer to explore a VE, which is a 3D reconstruction of the real world based on point clouds. Such point cloud virtual environments (PCVEs) represent the external environment, and are usually acquired by 3D scanners. We present an application scenario, in which a mobile robot captures 3D scans of a terrestrial environment, which are automatically registered to a coherent PCVE. This virtual 3D reconstruction is displayed in an immersive virtual environment (IVE) in which a user can explore the PCVE. We explain and describe the technical setup, which opens up new vistas of presenting a VE as points rather than a polygonal representation.
Nüchter, A., Rusu, R.B., Holz, D., Munoz, D. eds: Journal of Robotics and Autonomous Systems (JRAS). (2014).
Al khawaldah, M., Nüchter, A.: Multi-Robot Cooperation for Efficient Exploration.AUTOMATIKA -- Journal for Control, Measurement, Electronics, Computing and Communications.55,276--286 (2014).
This paper addresses the problem of exploration of an unknown environment by developing effective exploration strategies for a team of mobile robots equipped with continuously rotating 3D scanners. The main aim of the new strategies is to reduce the exploration time of unknown environment. Unlike most of other published works, to save time, the laser scanners rotate and scan the environment while robots are in motion. Furthermore, the new strategies are able to explore large outdoor environments as a considerable reduction of the required computations, especially those required for path planning, have been achieved. Moreover, another new exploration strategy has been developed so that robots continuously replan the order to visit the remaining unexplored areas according to the new data (i.e. updated map) collected by the robot in question or by the other team members. This new extension led to further enhancements over the above mentioned ones, but with slightly higher computational costs. Finally, to assess our new exploration strategies with different levels of environment complexity, new set of experiments were conducted in environments where obstacles are distributed according to the Hilbert curve. The results of these experiments show the effectiveness of the proposed technique to effectively distribute the robots over the environment. More importantly, we show how the optimal number of robots is related to the environment complexity.
Bedkowski, J., Majek, K., Nüchter, A.: Nowy algorytm 6DSLAM wykorzystujacy semantyczne rozpoznanie otoczenia.13 Krajowka Konferencja Robotyki Kudowa Zdroj Prace Naukowe, Elektronika z. 194 Post. Robotyki Tom II. p. 513--520 (2014).
W artykule przedstawiono modyfikację algorytmu 6DSLAM wykorzystującą semantyczne rozpoznawanie otoczenia. Zastosowanie semantycznego podejścia nie tylko poprawia w porównaniu do klasycznej metody dokładność tworzonej mapy metrycznej przez robota mobilnego, ale także umożliwia tworzenie takiej mapy w trudnych warunkach terenowych. Przedstawiono eksperymenty tworzenia mapy metrycznej/semantycznej budynków uwzględniając jazdę poziomą oraz kierunku pionowym po schodach. Porównano wynik z poprzednią implementacją algorytmu 6DSLAM. Nowe podejście poprawia spójność oraz dokładność mapy metrycznej. Zastosowanie semantycznego rozpoznawania otoczenia pozwala na rozszerzenie mapy metrycznej o nowe informacje o charakterze jakościowym, w tym przypadku rozróżniane są ściany, podłoga, sufit oraz punkty charakteryzujące się otoczeniem nieuporządkowanym.
Elseberg, J., Borrmann, D., Nüchter., A.: A Study of Scan Patterns for Mobile Mapping.Proceedings of the ISPRS Conference on "Serving Society with Geoinformatics" (ISPRS-SSG '13). p. 75--80. , Antalya, Turkey (2013).
Mobile terrestrial scanning systems automate terrestrial laser scanning. Continous scanning mobile terrestrial systems constantly spin the terrestrial laser scanner and thus combine terrestrial scanning with kinematic laser scanning. This paper presents a scan pattern analysis for these systems. We aim at finding the most advantageous combination of terrestrial and kinematic systems. The resulting 3D point cloud depends on the scan pattern and the trajectory and velocity of the mobile system.
Elseberg, J., Borrmann, D., Nüchter, A.: Algorithmic solutions for computing accurate maximum likelihood 3D point clouds from mobile laser scanning platforms.Remote Sensing.5,5871--5906 (2013).
Mobile laser scanning puts high requirements on the accuracy of the positioning systems and the calibration of the measurement system. We present a novel algorithmic approach for calibration with the goal of improving the measurement accuracy of mobile laser scanners. We describe a general framework for calibrating mobile sensor platforms that estimates all configuration parameters for any arrangement of positioning sensors, including odometry. In addition, we present a novel semi-rigid Simultaneous Localization and Mapping (SLAM) algorithm that corrects the vehicle position at every point in time along its trajectory, while simultaneously improving the quality and precision of the entire acquired point cloud. Using this algorithm, the temporary failure of accurate external positioning systems or the lack thereof can be compensated for. We demonstrate the capabilities of the two newly proposed algorithms on a wide variety of datasets.
Nüchter, A., Elseberg, J., Borrmann, D.: Automation in 3D Laser Scanning -- From an Automated Tripod towards Optimal 3D Point Clouds from Mobile Laser Scanning.GIM International.27, (2013).
Gurau, C., Nüchter, A.: Challenges in Using Semantic Knowledge for 3D Object Classification.Proceedings of the KI 2013 Workshop on Visual and Spatial Cognition, KIK - KI & Kognition Workshop Series. , Koblenz, Germany (2013).
To cope with a wide variety of tasks, robotic systems need to perceive and understand their environments. In particular, they need a representation of individual objects, as well as contextual relations between them. Visual information is the primary data source used to make predictions and inferences about the world. There exists, however, a growing tendency to introduce high-level semantic knowledge to enable robots to reason about objects. We use the Semantic Web framework to represent knowledge and make inferences about sensor data, in order to detect and classify objects in the environment. The contribution of this work is the identification of several challenges that co-occur when combining sensor data processing with such a reasoning method.
Bkedkowski, J., Majek, K., Nüchter, A.: General Purpose Computing on Graphics Processing Units for Robotic Applications.Journal of Software Engineering for Robotics (JOSER).4,23--33 (2013).
This paper deals with research related with the improvements of state of the art algorithms used in robotic applications based on parallel computation. The main goal is to decrease the computational complexity of 3D cloud of points processing in such applications as: data filtering, normal vector estimation, data registration and calculation of point feature histogram. The presented results efficiently improve the existing implementations with minimal lost of accuracy. The main contribution is a regular grid decomposition originally implemented for nearest neighborhood search. This data structure is used almost for all presented methods, it provides an efficient method for decreasing the time of computation. The results are compared with well-known robotic frameworks such as PCL and 3DTK.
Dumitru, R.-C., Borrmann, D., Nüchter, A.: Interior Reconstruction using the 3D Hough Transform.Proceedings of the 5th ISPRS International Workshop 3D-ARCH 2013: "3D Virtual Reconstruction and Visualization of Complex Architectures". , Trento, Italy (2013).
Laser scanners are often used to create accurate 3D models of buildings for civil engineering purposes, but the process of manually vectorizing a 3D point cloud is time consuming and error-prone (Adan and Huber, 2011). Therefore, the need to characterize and quantify complex environments in an automatic fashion arises, posing challenges for data analysis. This paper presents a system for 3D modeling by detecting planes in 3D point clouds, based on which the scene is reconstructed at a high architectural level through removing automatically clutter and foreground data. The implemented software detects openings, such as windows and doors and completes the 3D model by inpainting.
Demisse, G., Borrmann, D., Nüchter, A.: Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency.Proceedings of the 16th IEEE International Conference on Advanced Robotics (ICAR '13). p. 1--8. , Montevideo, Uruguay (2013).
In recent years, 3D models of buildings are used in maintenance and inspection, preservation, and other building related applications. However, the usage of these models is limited, because most models are pure representations with no or little associated semantics. In this paper, we present a pipeline of techniques used for interior interpretation, object detection, and adding energy related semantics to windows of a 3D thermal model. A sequence of algorithms is presented for building the fundamental semantics of a 3D model. Furthermore, a Markov Random Field is used to model the temperature distribution of detected windows to further label the windows as either open, closed or damaged.
Nüchter, A., Elseberg, J., Borrmann, D.: Irma3D -- An Intelligent Robot for Mapping Applications.Proceedings of the 3rd IFAC Symposium on Telematics Applications (TA '13). p. 119--124. , Seoul, Korea (2013).
Motivated by the increasing need of rapid characterization of environments in 3D, we designed a robot system that automates the work of an operator of terrestrial laser scanners. The built system enables to work without using special targets or markers and thus enables the surveyors to save more than 75% of the time spent in the field. Another impulse for developing the platform is the demand for a remote multi-sensor inspection tool. The robot is capable of surveying remote sites or danger areas, such as plants, underground mines, tunnels, caves, or channels. The results are precise, multi-modal digital 3D maps. This paper presents the recently developed robot Irma3D, its hardware, the developed interconnected software modules, the associated sensor calibration methods and a few applications.
Zhang, L., Li, Q., Li, M., Mao, Q.Z., Nüchter, A.: Multiple Vehicle-like Target Tracking Based on Velodyne Lidar.Proceedings of the 6th IFAC Symposium on Intelligent Autonomous Vehicles (IAV '13). , Gold Coast, Australia (2013).
This paper proposes a novel multiple vehicle-like target tracking method based on a Velodyne HDL64E light detection and ranging (LiDAR) system. The proposed method combines multiple hypothesis tracking (MHT) algorithm with dynamic point cloud registration (DPCR), which is able to solve the multiple vehicle-like target tracking in highly dynamic urban environments without any auxiliary information from GPS or IMU. Specifically, to track targets consistently, the DPCR is developed to calculate accurately the pose of the ego-vehicle for the transformation of raw measurements taken in the moving coordinate systems into a static absolute coordinate system; while in turn, MHT helps to improve the performance of DPCR by discriminating and removing the dynamic points from the scene. Furthermore, the proposed MHT method is also able to solve the occlusion problem existing in the point cloud. Experiments on sets of urban environments prove that the presented method is effective and robust, even in highly dynamic environments.
Houshiar, H., Borrmann, D., Elseberg, J., Nüchter, A., Winkler, S., Näth, F.: On-Site Semantic mapping of Archeological Excavation Areas.Proceedings of the XXIV International CIPA Symposium. p. 163--168. , Strasbourg, France (2013).
3D laser scanning is the state of the art in modeling archaeological excavation sites, historical sites and even entire cities or landscapes. The documentation of findings on an excavation site is an essential archaeological task. Automated systems accelerate this process and decrease the amount of error to a minimum. This paper presents a new documentation approach in industrial archaeology. It consists of a set of tools for recording and registering 3D data from excavation sites. We provide an efficient tool for visualization of acquired 3D point clouds in 3D and 2D modes. The main purpose of this software is to provide an easy to use, on-site semantic mapping tool for archaeologists. It includes functions for selecting and labeling findings. Additional information can be provided for each label. This data is exported to an XML format and serves as input for other systems and databases.
Elseberg, J., Borrmann, D., Nüchter, A.: One Billion Points in the Cloud --- An Octree for Efficient Processing of 3D Laser Scans.ISPRS Journal of Photogrammetry & Remote Sensing (JPRS), Special issue on terrestrial 3D modelling.76,76--88 (2013).
Automated 3-dimensional modeling pipelines include 3D scanning, registration, data abstraction, and visualization. All steps in such a pipeline require the processing of a massive amount of 3D data, due to the ability of current 3D scanners to sample environments with a high density. The increasing sampling rates make it easy to acquire Billions of spatial data points. This paper presents algorithms and data structures for handling these data. We propose an efficient octree to store and compress 3D data without loss of precision. We demonstrate its usage for an exchange file format, fast point cloud visualization, sped-up 3D scan matching, and shape detection algorithms. We evaluate our approach using typical terrestrial laser scans.
Nüchter, A., Elseberg, J., Borrmann, D.: Optimale 3D-Punktwolken aus mobilen Laserscandaten.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2013, Jade Hochschule. p. 186--193 (2013).
Mobiles Laserscanning stellt hohe Anforderungen an die Exaktheit der Positionierung und Kalibrierung des Messsystems, da sich bereits kleinste Ungenauigkeiten in den akquirierten 3D-Daten drastisch niederschlagen können. Dieser Beitrag präsentiert eine neuen Ansatz zur Kalibrierung und zur Lösung des SLAM-Problems mit dem Ziel, die Messgenauigkeit des mobilen Scansystems zu optimieren. Bei dem Kalibrierungsverfahren handelt es sich um eine allgemeingültige Methode, die die Positionierung und Orientierung sämtlicher Sensoren berechnet, einschliesslich der Odometrie. Der vorgestellte SLAM-Ansatz (Simultaneous Localization and Mapping) ist semi-rigide. Dabei wird die Trajektorie so deformiert und jede Pose dahingehend optimiert, dass die Gesamtqualität der 3D-Punktwolke optimal wird. Das SLAM-Verfahren kann kurzfristige Sensorausfälle oder Ungenauigkeiten ausgleichen und ermöglicht vielfältige neue Anwendungen.
Houshiar, H., Elseberg, J., Borrmann, D., Nüchter, A.: Panorama Based Point Cloud Reduction and Registration.Proceedings of the 16th IEEE International Conference on Advanced Robotics (ICAR '13). p. 1--8. , Montevideo, Uruguay (2013).
To reconstruct environments 3D point clouds acquired by laser scanners are registered. This is an important but also time consuming part of any mapping system for mobile robots. The time needed for mapping is drastically reduced when the size of the input data is reduced. This paper examines different ways of reducing the size of point clouds without losing vital information for the matching process. We present novel point cloud reduction methods on the basis of panorama images. It is shown that the reduced point clouds are ideally suited for feature based registration on panorama images. We evaluate the presented reduction methods based on their effect on the performance of the registration algorithm.
Borrmann, D., de Rezende, P.J., de Souza, C.C., Fekete, S.P., Friedrich, S., Kröller, A., Nüchter, A., Schmidt, C., Tozoni, D.C.: Point Guards and Point Clouds: Solving general Art Gallery Problems.Proccedings of the 20th ACM Annual Symposium on Computational Geometry (SoCG '13). p. 347--348. , Rio de Janeiro, Brazil (2013).
In this video, we illustrate how one of the classical areas of computational geometry has gained in practical relevance, which in turn gives rise to new, fascinating geometric problems. In particular, we demonstrate how the robot platform IRMA3D can produce high-resolution, virtual 3D environments, based on a limited number of laser scans. Computing an optimal set of scans amounts to solving an instance of the Art Gallery Problem (AGP): Place a minimum number of stationary guards in a polygonal region P, such that all points in P are guarded.
Okal, B., Nüchter, A.: Sliced Curvature Scale Space for Representing and Recognizing 3D Object.Proceedings of the 16th IEEE International Conference on Advanced Robotics (ICAR '13). p. 1--7. , Montevideo, Uruguay (2013).
Perception plays a key role in the development of intelligent autonomous systems. In particular object recognition and registration tasks are crucial to any intelligent autonomous system such as autonomous cars or personal robots. The representation of 3D object sensor measurements largely affects the choice of higher level processing possible on the sensor data. We explore the use of scale space theory via the curvature scale space and extend it to represent 3D objects in our new SCSS (Sliced Curvature Scale Space) framework. We further develop techniques of further processing the SCSS representation including feature extraction and dimensionality reduction for use in learning frameworks. We perform an array of experiments to validate the effectiveness of our method and demonstrate recognition performance using support vector machines. The results indicate that our new representation retains the nice qualities of the original curvature scale space method while being robust and compact for 3D object representation and recognition.
Borrmann, D., Houshiar, H., Elseberg, J., Nüchter, A.: Vom Kombinieren von 3D-Modellen mit Farb- und Temperaturinformationen.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2013, Jade Hochschule. p. 246--253 (2013).
In den letzten Jahren wurden grosse Fortschritte bei der automatischen Registrierung unterschiedlicher Daten gemacht, zum Beispiel bei Laserscandaten, Fotos und Thermalbilder. So wurden zuverlässige Methoden entwickelt, um Daten aus einem einzelnen Aufnahmeprozess zu registrieren. Probleme bereiten jedoch Daten, die zu unterschiedlichen Zeiten aufgenommen wurden. Dies ist aber essentiell um wertvolle Informationen verschiedener Sensoren zu kombinieren und den Anforderungen an die einzelnen Sensoren gerecht zu werden. Verwertbare Informationen von Thermalbildern erhält man ausschliesslich ohne störende Einflüsse des Sonnenlichts. Farbfotos hingegen werden optimal bei guten Farbverhältnissen aufgenommen. Laserscanner sind relativ robust gegenüber wechselnden Lichtverhältnissen. Für alle Sensoren ist ein Aufnahmezeitpunkt wünschenswert, zu dem möglichst wenige dynamische Objekte die Szene beeinflussen. Üblicherweise ist eine genaue Georeferenzierung der Daten notwendig um Konsistenz zwischen den einzelnen Modellen zu erreichen. Dies erhöht den Aufwand für die Datenakquise erheblich. In diesem Beitrag präsentieren wir robuste Verfahren zur vereinfachten Kombination unterschiedlicher Datensätze.
Chen, L., Li, Q., Zhu, Q., Li, M., Nüchter, A.: 3D LIDAR Point Cloud based Intersection Recognition for Autonomous Driving.Proceedings of the 2012 IEEE Intelligent Vehicles Symposium (IV '12). p. 456--461. , Alcala de Henares, Madrid, Spain (2012).
Finding road intersections in advance is crucial for navigation and path planning of moving autonomous vehicles, especially when there is no position or geographic auxiliary information available. In this paper, we investigate the use of a 3D point cloud based solution for intersection and road segment classification in front of an autonomous vehicle. It is based on the analysis of the features from the designed beam model. First, we build a grid map of the point cloud and clear the cells which belong to other vehicles. Then, the proposed beam model is applied with a specified distance in front of autonomous vehicle. A feature set based on the length distribution of the beam is extracted from the current frame and combined with a trained classifier to solve the road-type classification problem, i.e., segment and intersection. In addition, we also make the distinction between +-shaped and T-shaped intersections. The results are reported over a series of real-world data. A performance of above 80% correct classification is reported at a real-time classification rate of 5 Hz.
Elseberg, J., Borrmann, D., Nüchter, A.: 6DOF Semi-Rigid SLAM for Mobile Scanning.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '12). p. 1865--1870. , Vilamoura, Algarve, Portugal (2012).
The terrestrial acquisition of 3D point clouds by laser range finders has recently moved to mobile platforms. Measuring the environment while simultaneously moving the vehicle demands a high level of accuracy from positioning systems such as the IMU, GPS and odometry. We present a novel semi-rigid SLAM algorithm that corrects the global position of the vehicle at every point in time, while simultaneously improving the quality and accuracy of the entire acquired map. Using the algorithm the temporary failure of positioning systems or the lack thereof can be compensated for. We demonstrate the capabilities of our approach on a wide variety of systems and data sets.
Wiemann, T., Lingemann, K., Nüchter, A., Hertzberg, J.: A Toolkit for Automatic Generation of Polygonal Maps -- Las Vegas Reconstruction.Proceedings of the 7th German Conference Robotik 2012. p. 446--451. , Munich, Germany (2012).
In this paper we present a new open source software package for automatic generation of polygonal 3D maps from point cloud data for robotic purposes called "Las Vegas Reconstruction Toolkit" [11]. The implemented algorithms focus on minimizing the computational costs and optimization of the number of polygons in the generated maps. Furthermore, we present two application examples: 6D self localization and scene interpretation.
Sima, M.-C., Nüchter, A.: An extension of the Felzenszwalb-Huttenlocher segmentation to 3D point clouds.Proceedings of the 5th International Conference on Machine Vision (ICMV '12). , Wuhan, China (2012).
This paper investigates the segmentation algorithm proposed by Felzenszwalb and Huttenlocher1 and its compatibility to 3D point clouds acquired with state-of-the-art 3D laser scanners. To use the algorithm, we adapt the range and intensity data to the smoothed graph structure used by the algorithm. We investigate the influence of the algorithm’s parameters to its performance and result that are meaningful to both the machines and the humans.
J. Elseberg, D.B.: Automatic and Full Calibration of Mobile Laser Scanning Systems.Proceedings of the 13th International Symposium of Experimental Robotics (ISER '12). p. 907--917. , Quebec City, Canada (2012).
Mobile scanning, i.e., the practice of mounting laser scanners on moving platforms is an efficient way to acquire accurate and dense 3D point clouds of outdoor environments for urban and regional planning and architecture. The mobile scenario puts high requirements on the accuracy of the calibration of the measurement system, as small calibration inaccuracies lead to large errors in the resulting point cloud. We propose a novel algorithm for the calibration of a mobile scanning system that estimates the calibration parameters for all sensor components simultaneously without relying on additional hardware. We evaluate the calibration algorithm on several real world data sets where ground truth is available via an accurate geodetic model.
Elseberg, J., Magnenat, S., Siegwart, R., Nüchter, A.: Comparison on nearest-neigbour-search strategies and implementations for efficient shape registration.Journal of Software Engineering for Robotics (JOSER).3,2--12 (2012).
The iterative closest point (icp) algorithm is one of the most popular approaches to shape registration currently in use. At the core of icp is the computationally-intensive determination of nearest neighbors (NN). As of now there has been no comprehensive analysis of competing search strategies for NN. This paper compares several libraries for nearest-neighbor search (NNS) on both simulated and real data with a focus on shape registration. In addition, we present a novel efficient implementation of NNS via k-d trees as well as a novel algorithm for NNS in octrees.
Li, M., Li, W., Wang, J., Li, Q., Nüchter, A.: Dynamic VeloSLAM -- Preliminary Report on 3D Mapping of Dynamic Environments.Proceedings of the 2012 IEEE Intelligent Vehicles Symposium (IV '12), Workshop on Navigation, Perception, Accurate Positioning and Mapping for Intelligent Vehicles. , Alcala de Henares, Madrid, Spain (2012).
3D mapping using point cloud registration is a basic inevitable problem for many applications, especially for modeling of large scale complicated environments. This paper presents a novel approach for mapping highly dynamic environments, i.e., we present a system capable for mapping road traffic scenarios. Given 3D laser scans acquired at a high frame rate and no other sensor input, a 3D map is built by removing dynamic parts of the scene and estimating the ego-motion of the vehicle precisely at the same time. We extend the well-known ICP algorithm for HDL-64 laser scan data and build a system for solving the simultaneous localization and mapping problem in urban road scenarios. This paper presents initial results on two data sets.
Borrmann, D., Elseberg, J., K.C., P.N., Nüchter, A.: Ein Punkt pro Kubikmeter -- präzise Registrierung von terrestrischen Laserscans mit Scanmatching.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2012, Jade Hochschule. p. 4--11 (2012).
Die präzise Registrierung von terrestrischen Laserscans erfordert geschultes Personal, professionelle Scanner Hardware, meist proprietäre Software sowie genügend Zeit für die Anbringung von Zielmarken, Durchführung der Messungen und die anschlie&szilg;ende Verarbeitung der akquirierten Daten. Wir zeigen, wie man auf einige dieser Punkte verzichten und dennoch zu hochqualitativen Ergebnissen kommen kann. Im Zuge einer Bachelorarbeit wurde ein der Vermessungstechnik fremder Informatikstudent damit beauftragt, den Campus der Jacobs University auf einer Teilfläche von ca. 9 ha zu vermessen. Zur Verfügung standen ein professioneller Scanner in Form des Riegl VZ-400. Insgesamt wurden 131 Aufnahmen mit insgesamt über 2 Milliarden Punkten gemacht. Nach manueller Vorregistrierung der Punktwolken erfolgte die präzise Anordnung dann vollautomatisch mit der frei erhältlichen Software 3DTK, die dazu in der Lage war die gesamte Datenmenge in kürzester Zeit mit höchster Genauigkeit zu registrieren.
Elseberg, J., Borrmann, D., Nüchter, A.: Eine Milliarde 3D-Punkte mit Standardhardware verarbeiten -- Processing One Billion 3D Points on a Standard Computer.Allgemeine Vermessungs-Nachrichten (AVN).119,11--23 (2012).
Dieser Beitrag stellt eine neue Implementation der Octree-Datenstruktur vor. Sie ermöglicht es, eine Milliarde 3D-Punkte in 8 GB Hauptspeicher exakt zu repräsentieren und effiziente Algorithmen zu implementieren. Der Octree gibt eine Hierarchie vor, die dazu verwendet werden kann, grosse Punktwolken zu inspizieren und flüssig in ihnen zu navigieren. Des Weiteren schlagen wir in diesem Artikel ein effizentes binäres Dateiformat für den Austausch von 3D-Scans vor.
Demisse, G., Mihalyi, R., Okal, B., Poudel, D., Schauer, J., Nüchter, A.: Mixed Palletizing and Task Completion for Virtual Warehouses.Virtual Manufacturing Automation (VMAC '12) Workshop at IEEE International Conference Robotics and Automation, ICRA. , St. Paul, MN, USA (2012).
Palletizing, or packing rectangular boxes of various sizes onto pallets, is a frequently encountered task in many commercial scenarios, e.g., shipment of goods. An extension of this problem is the automated placement of boxes on pallets, henceforth task completion, by means of industrial robot arms. The palletizing and task completion problems are treated in the context of the IEEE ICRA 2012 Virtual Manufacturing Automation Competition. We approach the palletizing challenge by a winner-takes-all strategy, where multiple heuristics are evaluated against the given datasets. Our results show a performance comparable to that of the commercial software benchmark from the previous competitions. We solve task completion in USARSim by picking boxes and placing them on a pallet. Our inverse kinematics algorithm consists of a geometric and numeric component.
Hertzberg, J., Lingemann, K., Nüchter, A.: Mobile Roboter: Eine Einführung aus Sicht der Informatik.Springer, Heidelberg, Germany (2012).
Al khawaldah, M., Nüchter, A.: Multi-Robot Exploration and Mapping with a rotating 3D Scanner.Proceedings of the 10th International IFAC Symposium on Robot Control (SYROCO '12). , Dubrovnik, Croatia (2012).
This paper investigates the field of exploration and map-building with multiple cooperating mobile robots. New and efficient exploration and mapping technique is proposed by employing laser scanners. The paper also aims to extend existing exploration and mapping techniques of single robot to multi-robot to increase the exploration efficiency (i.e. to reduce the environment exploration time required). The goal of the proposed method is to have multiple mobile robots exploring a given unknown environment as fast as possible, while coordinating their actions and sharing their local maps in certain time instances. In the suggested technique, each robot is equipped with a laser scanner that is continuously rotating to scan the environment, and is employing a frontier-based exploration algorithm which is important to guide the robots during the exploration. A new factor is introduced to enhance the performance of the frontier-based exploration. This factor aims at spreading robots in the environment to reduce overlap.
Borrmann, D., Afzal, H., Elseberg, J., Nüchter, A.: Mutual Calibration for 3D Thermal Mapping.Proceedings of the 10th International IFAC Symposium on Robot Control (SYROCO '12). , Dubrovnik, Croatia (2012).
Three-dimensional digital heat distribution maps are needed to assess the energy efficiency of real estates. The availability of such maps are of great importance for reducing the ecological footprint of houses, buildings, and factories. Designing estates has reached the point, where so-called Passivhaus buildings make extensive use of the intrinsic heat from internal sources such as waste heat from lighting, white goods, and other electrical devices, but without using dedicated heaters. In our approach for creating high-precise heat distribution maps a robot is equipped with a 3D laser scanner, a thermal camera, and a color camera. Data from all the sensors are combined to model the environment precisely. This paper describes the setup of the sensors and the processing of the acquired data, including the automatic co-calibration needed to fulfill this task.
Nüchter, A., Houshair, H., Borrmann, D., Elseberg, J.: Projektionen für die Scanregistrierung mit Hilfe von Bildmerkmalen.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2012, Jade Hochschule. p. 12--21 (2012).
Die Aufnahme und das Registrieren von terrestrischen 3D-Laserscans ist eine grundlegende Fragestellung in vielen Anwendungsbereichen. Um den Prozess des Registrierens von zwei 3D-Scans zu automatisieren, werden Merkmale aus den Scandaten extrahiert, diese einander zugeordnet (assoziiert) und die Registrierung berechnet. Als Merkmale kommen neben Strukturmerkmalen wie 3D-Flächen oft Bildmerkmale zum Einsatz. Dazu wird ein flächiges Bild der Reflektionswerte eines 3D-Scans erzeugt. In der Regel wird hierbei auf der x-Achse der Rotationswinkel um die Stehachse des Scanners und auf der y-Achse der Rotationswinkel des Spiegels abgetragen. In diesem Beitrag untersuchen wir wie sich die Registrierungsergebnisse ändern, wenn andere Projektionen verwendet werden. Dazu analysieren wir mit Hilfe von SIFT-Merkmalen die gleichmäßige Rektangularprojektion, die zylindrische Projektion, die Hochachsenrojektion, die Mercatorprojektion, die Rektangularprojektion, die Pannini-Projektion, sowie die stereographische Projektion.
May, S., Koch, R., Scherlipp, R., Nüchter, A.: Robust Registration of Narrow-Field-of-View Range Images.Proceedings of the 10th International IFAC Symposium on Robot Control (SYROCO '12). , Dubrovnik, Croatia (2012).
This paper focuses on range image registration for robot localization and environment mapping. It extends the well-known Iterative Closest Point (ICP) algorithm in order to deal with erroneous measurements. The dealing with measurement errors originating from external lighting, occlusions or limitations in the measurement range is only rudimentary in literature. In this context we present a non-parametric extension to the ICP algorithm that is derived directly from measurement modalities of sensors in projective space. We show how aspects from reverse calibration can be embedded in search-tree-based approaches. Experiments demonstrate the applicability to range sensors like the Kinect device, Time-of-Flight cameras and 3D laser range finders. As a result the image registration becomes faster and more robust.
Borrmann, D., Nüchter, A., Dakulovic, M., Maurovic, I., Petrovic, I., Osmankovic, D., Velagic, J.: The Project ThermalMapper -- Thermal 3D Mapping of Indoor Environments for Saving Energy.Proceedings of the 10th International IFAC Symposium on Robot Control (SYROCO '12). , Dubrovnik, Croatia (2012).
Heat and air conditioning losses in buildings and factories lead to a large amount of wasted energy. The Action Plan for Energy Efficiency of the Commission of the European Communities (2008) estimates that the largest cost-effective energy savings potential lies in residential (\($\sim$\)27\%) and commercial (\($\sim$\)30\%) buildings. Imagine a technology that creates a precise digital 3D model of heat distribution and heat flow enabling one to detect all sources of wasted energy and to modify buildings to reach these savings. This paper presents our overall approach to map indoor environments with thermal data in 3D.
Borrmann, D., Elseberg, J., A, N.: Thermal 3D Mapping of Building Facades.Proceedings of the 8th Conference on Intelligent Autonomous Systems (IAS '12). p. 173--182. Springer, Jeju Island, Korea (2012).
Never before in history were humans as dependant on energy as we are today. But the natural ressources are limited and a waste of energy has drastic influences on the environment. In their Action Plan for Energy Efficiency [6] the European Commission estimates that the largest and cost-effictive energy savings potential lies in residential (\($\sim$\)27\%) and commercial (\($\sim$\)30\%) buildings. To eliminate heat and air conditioning losses in buildings and factories heat and air leaks need to be localized and identified. Imagine the availability of a complete 3D model of every building that architects can use to analyze the heat insulation of buildings and to identify necessary modifications. In these 3D models temperature peaks are not only detectable but also their extent is visible. A robot equiped with a 3D laser scanner, a thermal camera, and a color camera constitutes the basis for our approach. The data from all three sensors and from different locations are joined into one high-precise 3D model that shows the heat distribution. This paper describes the setup of the hardware and the methods applied to create the 3D model, including the automatic co-calibration of the sensors. Challenges unique to the task of thermal mapping of outdoor environments are discussed.
Borrmann, D., Afzal, H., Elseberg, J., Nüchter, A.: Thermal 3D Modeling of Indoor Environments for Saving Energy.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '12). p. 4538--4539. , Vilamoura, Algarve, Portugal (2012).
Heat and air conditioning losses in buildings and factories lead to a large amount of wasted energy. The Action Plan for Energy Efficiency [4] of the European Commission estimates that the largest cost-effective energy savings potential lies in residential (\($\sim$\)27\%) and commercial (\($\sim$\)30\%) buildings. Imagine a technology that creates a precise digital 3D model of heat distribution and heat flow enabling one to detect all sources of wasted energy and to modify buildings to reach these savings. This video presents our approach to this task. Methods for creating a consistent laser scan model enhanced with information from thermal and optical cameras are presented.
Li, M., Li, W., Wang, J., Li, Q., Nüchter, A.: Towards Reliable Object Anchoring in Highly Dynamic Traffic Scenes.Proceedings of the ICRA 2012 WORKSHOP Semantic Perception and Mapping for Knowledge-enabled Service Robotics (with interactive session and demonstrations). , St. Paul, MN, USA (2012).
Sprickerhof, J., Nüchter, A., Lingemann, K., Hertzberg, J.: A Heuristic Loop Closing Technique for Large-Scale 6D SLAM.AUTOMATIKA -- Journal for Control, Measurement, Electronics, Computing and Communications, Special Issue with selected papers from the 4th European Conference on Mobile Robots.52, (2011).
This paper presents a novel heuristic for correcting scan pose estimations after loop closing in SLAM using 3D laser scans. Contrary to state of the art approaches, the built SLAM graph is sparse, and optimization is done without any iteration between the SLAM front and back end, yielding a highly efficient loop closing method. Several experiments were carried out in an urban environment and evaluated against ground truth. The results are compared to other state of the art algorithms, proving the high quality, yet achieved faster by an order of magnitude.
Elseberg, J., Borrmann, D., Nüchter, A.: Efficient Processing of Large 3D Point Clouds.Proceedings of the XXIII International Symposium on Information, Communication and Automation Technologies (ICAT '11). IEEE Xplore, Sarajevo, Bosnia (2011).
Autonomous robots equipped with laser scanners acquire data at an increasingly high rate. Registration, data abstraction and visualization of this data requires the processing of a massive amount of 3D data. The increasing sampling rates make it easy to acquire Billions of spatial data points. This paper presents algorithms and data structures for handling this data. We propose an efficient octree to store and compress 3D data without loss of precision. We demonstrate its usage for fast 3D scan matching and shape detection algorithms. We evaluate our approach using typical data acquired by mobile scanning platforms.
Elseberg, J., Borrmann, D., Nüchter, A.: Eine effiziente Octree-Datenstruktur für das Verarbeiten von grossen 3D-Punktwolken.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2011, Fachhochschule Oldenburg/Ostfr./Whv. p. 72--79 (2011).
Dieser Beitrag stellt eine neue Implementation der Octree-Datenstruktur vor, die es ermöglicht, eine Milliarde 3D-Punkte in 8 GB Hauptspeicher exakt zu repräsentieren und effiziente Algorithmen zu implementieren. Der Octree gibt eine Hierarchie vor, die dazu verwendet werden kann, große Punktewolken zu inspizieren und flüssig in ihnen zu navigieren.
Elseberg, J., Borrmann, D., Nüchter, A.: Full Wave Analysis in 3D Laser Scans for Vegetation Detection in Urban Environments.Proceedings of the XXIII International Symposium on Information, Communication and Automation Technologies (ICAT '11). IEEE Xplore, Sarajevo, Bosnia (2011).
This paper presents a novel technique for detecting vegetation of virtually all forms in terrestrial laser scanning data of urban environments. We make use of a modern laser range finder capability to measure multiple echoes per laser pulse via Full Wave Analysis. The algorithm is able to efficiently, i.e., less than acquisition time, identify vegetation to a high degree of accuracy (more than 99 percent). We present and evaluate three alternatives to classify candidate regions as either vegetation or non-vegetation.
Grosan, F., Tandrau, A., and A. Nüchter,: Localizing Google SketchUp Models in Outdoor 3D Scans.Proceedings of the XXIII International Symposium on Information, Communication and Automation Technologies (ICAT '11). IEEE Xplore, Sarajevo, Bosnia (2011).
This work introduces a novel solution for localizing objects based on search strings and freely available Google SketchUp models. To this end we automatically download and preprocess a collection of 3D models to obtain equivalent point clouds. The outdoor scan is segmented into individual objects, which are sequentially matched with the models by a variant of iterative closest points algorithm using seven degrees of freedom and resulting in a highly precise pose estimation of the object. An error function evaluates the similarity level. The approach is verified using various segmented cars and their corresponding 3D models.
Nüchter, A., Gutev, S., Borrmann, D., Elseberg, J.: Skyline-Based Registration of 3D Laser Scans.Proceedings of the Joint ISPRS workshop on 3D city modelling & applications and the 6th 3D GeoInfo (3DCMA '11), The Chinese Academic Journal (CD ROM version) CN 11-9251/G. , Wuhan, China (2011).
Acquisition and registration of terrestrial 3D laser scans is a fundamental task in mapping and modeling of cities in three dimensions. To automate this task marker-free registration methods are required. Based on the existence of skyline features this paper proposes a novel method. The skyline features are extracted from panoramic 3D scans and encoded as strings enabling the use of string matching for merging the scans. Initial results of the proposed method in the old city center of Bremen are presented.
Nüchter, A., Gutev, S., Borrmann, D., Elseberg, J.: Skyline-Based Registration of 3D Laser Scans.Journal Geo-spatial Information Science (GSIS), Special Issue with selected papers from the 3D City Modeling and Applications Workshop.14,85--90 (2011).
Acquisition and registration of terrestrial 3D laser scans is a fundamental task in mapping and modeling of cities in three dimensions. To automate this task marker-free registration methods are required. Based on the existence of skyline features this paper proposes a novel method. The skyline features are extracted from panoramic 3D scans and encoded as strings enabling the use of string matching for merging the scans. Initial results of the proposed method in the old city center of Bremen are presented.
Nüchter, A., Feyzabadi, S., Qiu, D., May, S.: SLAM à la carte - GPGPU for Globally Consistent Scan Matching.Proceedings of the 4th European Conference on Mobile Robots (ECMR '11). , Örebro, Sweden (2011).
Borrmann, D., Elseberg, J., Nüchter, A., Lingemann, K.: The 3D Hough Transform for Plane Detection in Point Clouds -- A Review and A new Accumulator Design.Journal of 3D Research.2,1--13 (2011).
The Hough Transform is a well-known method for detecting parameterized objects. It is the de facto standard for detecting lines and circles in 2-dimensional data sets. For 3D it has attained little attention so far. Even for the 2D case high computational costs have lead to the development of numerous variations for the Hough Transform. In this article we evaluate different variants of the Hough Transform with respect to their applicability to detect planes in 3D point clouds reliably. Apart from computational costs, the main problem is the representation of the accumulator. Usual implementations favor geometrical objects with certain parameters due to uneven sampling of the parameter space. We present a novel approach to design the accumulator focusing on achieving the same size for each cell and compare it to existing designs.
Hertzberg, J., Lingemann, K., Lörken, C., Nüchter, A., Stiene, S., Wiemann, T.: 3D-Roboterkartenbau in Osnabrück.KI -- Künstliche Intelligenz: Themenschwerpunk Simultaneous Localization and Mapping (SLAM).24,245--248 (2010).
Seit Herbst 2004 existiert die Arbeitsgruppe „Wissensbasierte Systeme“ am Institut für Informatik der Universität Osnabrück. Ein Langfristziel der Arbeitsgruppe besteht darin, Schlussfolgerungs- und Planungsverfahren der KI für den Einsatz online und onboard auf mobilen Robotern einsetzbar zu machen. Ein daraus abgeleitetes Arbeitsthema ist der Bau von semantischen Roboterkarten basierend auf 3D-Laserscans bei 6-dimensionalen Scanposen. Wir geben einen Überblick über die wichtigsten Ergebnisse dazu und über unsere Perspektive dieses Themas für die Zukunft.
Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A.: A Data Structure for the 3D Hough Transform for Plane Detection.Proceedings of the 5th IFAC Symposium on Intelligent Autonomous Vehicles (IAV '10). , Lecce, Italy (2010).
The Hough Transform is a well-known method for detecting parametrized objects. It is the de facto standard for the detection of lines and circles in 2-dimensional data sets. For 3D it has attained little attention so far. Apart from computational costs, the main problem is the representation of the accumulator: Usual implementations favor geometrical objects with certain parameters due to uneven sampling of the parameter space. In this paper we present a novel approach to design the accumulator focusing on achieving the same size for each cell. The proposed accumulator is compared to previously known designs.
Wiemann, T., Nüchter, A., Lingemann, K., Stiene, S., Hertzberg, J.: Automatic Construction of Polygonal Maps From Point Cloud Data.Proceedings of the IEEE International Workshop on Safety, Security and Rescue Robotics (SSRR '10). , Bremen, Germany (2010).
This paper presents a novel approach to create polygonal maps from 3D point cloud data. The gained map is augmented with an interpretation of the scene. Our procedure produces accurate maps of indoor environments fast and reliably. These maps are successfully used by different robots with varying sensor configurations for reliable self localization.
Pathak, K., Borrmann, D., Elseberg, J., Vaskevicius, N., Birk, A., Nüchter, A.: Evaluation of the Robustness of Planar-Patches based 3D-Registration using Marker-based Ground-Truth in an Outdoor Urban Scenario.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '10). p. 5725--5730. , Taipei, Taiwan (2010).
The recently introduced Minimum Uncertainty Maximum Consensus (MUMC) algorithm for 3D scene registration using planar-patches is tested in a large outdoor urban setting without any prior motion estimate whatsoever. With the aid of a new overlap metric based on unmatched patches, the algorithm is shown to work successfully in most cases. The absolute accuracy of its computed result is corroborated for the first time by ground-truth obtained using reflective markers. There were a couple of unsuccessful scan-pairs. These are analyzed for the reason of failure by formulating two kinds of overlap metrics: one based on the actual overlapping surface-area and another based on the extent of agreement of range-image pixels. We conclude that neither metric in isolation is able to predict all failures, but that both taken together are able to predict the difficulty level of a scan-pair vis-à-vis registration by MUMC.
Digor, E., Birk, A., Nüchter, A.: Exploration Strategies for a Robot with a Continously Rotating 3D Scanner.Proceedings of the Second International Conference on Simulation, Modeling and Programming for Autonomous Robots (SIMPAR '10). p. 374--386. , Darmstadt, Germany (2010).
To benchmark the efficiency of exploration strategies one has to use robot simulators. In an exploration task, the robot faces an unknown environment. Of course one could test the algorithm in different real-world scenarios, but a competitive strategy must have good performance in any environment that can be systematically constructed inside a simulator. This paper presents an evaluation of exploration strategies we developed for a specific sensor. A continously rotating 3D laser scanner that scans only into one direction at a time moves through the environment sampling the surrounding. Our evaluation framework features an efficient scanning and robot simulator for kinematic feasible trajectories. We will show that shorter trajectories do not necessarily imply quicker exploration. A simple simulator framework is sufficient for evaluating these properties of path planning algorithms.
Borrmann, D., Elseberg, J., Raunyar, S.S., Nüchter, A.: Lifelong 3D Mapping -- Monitoring with a 3D Scanner.Proceedings of the IROS Workshop on Robotics for Environmental Monitoring (WREM '10). , Taipei, Taiwan (2010).
Geodesy and surveying are the sciences for monitoring the earth. In recent years traditional surveying equipment has been pushed aside by the emerging technology of laser scanners, that automate the precise measurement of points in the environment. A further step of automation is achieved by operating the surveying equipment automatically and the usage of robotic mapping. Thus, robotic mapping will become a key component in monitoring and surveillance tasks. This paper evaluates a 3D laser scanner from surveying for its usage in robotic monitoring tasks. We examine how seasonal changes and weather conditions impact the data of the 3D scanner and how to deal with these changes. For this analysis 3D scans of various predetermined locations on the campus of the Jacobs University Bremen were taken on a weekly basis over a period of 13 weeks using the RIEGL VZ-400 3D laser range finder. The scans have been registered by means of conventional surveying markers, SIFT features and using our point cloud based 6D SLAM framework. An analysis of the changes in the environment over the course of the scanning period and an evaluation of the matching results complete the analysis.
Nüchter, A., Elseberg, J., Schneider, P., Paulus, D.: Linearization of Rotations for Globally Consistent \($n$\)-Scan Matching.Proceedings of the IEEE International Conference Robotics and Automation (ICRA '10). p. 1373--1379. , Anchorage, Alaska, USA (2010).
The ICP (Iterative Closest Point) algorithm is the de facto standard for geometric alignment of three-dimensional models when an initial relative pose estimate is available. The basis of the algorithm is the minimization of an error function that takes point correspondences into account. While four closed-form solution methods are known for minimizing this function, linearization seems necessary for solving the global scan registration problem. This paper presents such linear solutions for registering n-scans in a global and simultaneous fashion. It studies parameterizations for the rigid body transformations of the n-scan registration problem.
Elseberg, J., Borrmann, D., Nüchter, A., Lingemann, K.: Non-Rigid Registration and Rectification of 3D Laser Scans.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '10). p. 1546--1552. , Taipei, Taiwan (2010).
Three dimensional point clouds acquired by range scanners often do not represent the environment precisely due to noise and errors in the acquisition process. These latter systematical errors manifest as deformations of different kinds in the 3D range image. This paper presents a novel approach to correct deformations by an analysis of the structures present in the environment and correcting them by non-rigid transformations. The resulting algorithms are used for creating high-accuracy 3D indoor maps.
Nüchter, A., Elseberg, J., Schneider, P., Paulus, D.: Study of Parameterizations for the Rigid Body Transformations of The Scan Registration Problem.Journal Computer Vision and Image Understanding (CVIU).114,963--980 (2010).
The iterative closest point (ICP) algorithm is the de facto standard for geometric alignment of three-dimensional models when an initial relative pose estimate is available. The basis of the algorithm is the minimization of an error function that takes point correspondences into account. Four closed-form solution methods are known for minimizing this function. This paper presents novel linear solutions to the scan registration problem, i.e., to the problem of putting and aligning 3D scans in a common coordinate system. We extend the methods for registering n-scans in a global and simultaneous fashion, such that the registration of the nth scan influences all previous registrations in one step.
Wülfing, J., Hertzberg, J., Lingemann, K., Nüchter, A., Wiemann, T., Stiene, S.: Towards Real Time Robot 6D Localization in a Polygonal Indoor Map Based on 3D ToF Camera Data.Proceedings of the 5th IFAC Symposium on Intelligent Autonomous Vehicles (IAV '10). , Lecce, Italy (2010).
This paper reports a method and results for solving the following problem: Given a 3D polygonal indoor map and a mobile robot equipped with a 3D time of flight (ToF) camera, localize at frame rate the 6D robot pose with respect to the map. To solve the problem, the polygonal map is represented for efficient usage as a solid-leaf BSP tree; at each control cycle, the 6D pose change is estimated a priori from odometry or IMU, the expected ToF camera view at the prior pose sampled from the BSP tree, and the pose change estimation corrected a posteriori by fast ICP matching of the expected and the measured ToF image. Our experiments indicate that, first, the method is in fact real-time capable; second, the 6D pose is tracked reliably in a correct map under regular sensor conditions; and third, the tracking can recover from some faults induced by local map inaccuracies and transient or local sensing errors.
Elseberg, J., Nüchter, A., Borrmann, D., Lingemann, K.: Verbesserte Kartenqualität durch Thin Plate Splines und Hough-Transformation.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2010, Fachhochschule Oldenburg/Ostfr./Whv. p. 134--141 (2010).
Laserscanner sind präzise Messgeräte zur Ermittlung von Distanzwerten. Dennoch weist eine jede mit einem Laserscanner akquirierte Punktwolke Fehler auf, die auch durch Kalibrierung nicht vollständig verhindert werden können. Neben Sensorrauschen kommt es auch zu systematischen Fehlern. Die meisten künstlich geschaffenen Umgebungen bestehen aus einer großen Anzahl ebener Flächen, die helfen können, die Qualität von Laserscan-Karten zu verbessern. Dieser Aufsatz stellt Methoden vor, die unter Zuhilfenahme von ebenen Umgebungsstrukturen die Fehler in den Laserscans durch nicht-rigide Verformungen verringern
May, S., Dröschel, D., Holz, D., Fuchs, S., Malis, E., Nüchter, A., Hertzberg, J.: 3D Mapping with Time-of-Flight Cameras.Journal of Field Robotics (JFR), Special Issue on Three-Dimensional Mapping.26,892--914 (2009).
This article investigates the use of time-of-flight (ToF) cameras in mapping tasks for autonomous mobile robots, in particular in simultaneous localization and mapping (SLAM) tasks. Although ToF cameras are in principle an attractive type of sensor for three-dimensional (3D) mapping owing to their high rate of frames of 3D data, two features make them difficult as mapping sensors, namely, their restricted field of view and influences on the quality of range measurements by high dynamics in object reflectivity; in addition, currently available models suffer from poor data quality in a number of aspects. The paper first summarizes calibration and filtering approaches for improving the accuracy, precision, and robustness of ToF cameras independent of their intended usage. Then, several ego motion estimation approaches are applied or adapted, respectively, in order to provide a performance benchmark for registering ToF camera data. As a part of this, an extension to the iterative closest point algorithm has been developed that increases the robustness under restricted field of view and under larger displacements. Using an indoor environment, the paper provides results from SLAM experiments using these approaches in comparison. It turns out that the application of ToF cameras is feasible to SLAM tasks, although this type of sensor has a complex error characteristic.
Nüchter, A.: 3D Robotic Mapping: The Simultaneous Localization and Mapping Problem with Six Degrees of Freedom.Springer (2009).
Nüchter, A.: 6D SLAM mit Global Konsistentem Scanmatching.Terrestrisches Laserscanning (TLS 2009) Beiträge zum DVW-Seminar am 18. und 19. November in Fulda. p. 69--92. , Fulda, Germany (2009).
Terrestrisches Laserscanning (TLS) hat die in den letzten Jahren die Vermessungstechnik revolutioniert. Durch die Entwicklung des kinematischen terrestrischen Laserscannings (k-TLS) oder Mobile Mapping, das die Aufnahme von geometrische Umgebungsinformation von einer bewegten Plattform aus erlaubt, wurde ein wesentlicher Schritt zur weiteren Automatisierung in der Vermessungstechnik getan. Leider ist k-TLS nicht überall einsetzbar, da neben den Daten des Laserscanners hochgenaue Informationen über die Pose (Position und Orientierung) der mobilen Plattform vorliegen müssen. Technischer Fortschritt erlaubt den Bau von autonomen Robotersystemen, die mit 3D-Laserscannern ausgestattet sind und es besteht Potential für weitere Automatisierung. Dazu muss das Problem der gleichzeitigen Lokalisation und Kartierung gelöst werden. Dieses klassische Robotikproblem ist ein Henne-und-Ei-Problem: Mit genauster Kenntnis der Position des mobilen Roboters lassen sich korrekte Karten erzeugen. Mit Hilfe von Karten können sich Roboter sehr genau lokalisieren. Beides gleichzeitig durchzuführen stellt neue hohe Anforderungen an die Algorithmen, die Scannerdaten verarbeiten.
Sprickerhof, J., Nüchter, A., Lingemann, K., Hertzberg, J.: An Explicit Loop Closing Technique for 6D SLAM.Proceedings of the 4th European Conference on Mobile Robots (ECMR '09). , Mlini/Dubrovnic, Croatia (2009).
Simultaneous Localization and Mapping (SLAM) is the problem of building a map of an unknown environment by a mobile robot while at the same time navigating the environment, using the unfinished map. For SLAM, two tasks have to be solved: First reliable feature extraction and data association, second the optimal estimation of poses and features. These two parts are often referred to as SLAM frontend and backend. Algorithms that solve SLAM by using laser scans commonly rely on matching closest points in the frontend part. Then the SLAM front- and backend have to be iterated to ensure that the map converges. This paper presents a novel approach for solving SLAM using 3D laser range scans. We aim at avoiding the iteration between the SLAM front- and backend and propose a novel explicit loop closing heuristic (ELCH). It dissociates the last scan of a sequence of acquired scans, reassociates it to the map, built so far by scan registration, and distributes the difference in the pose error over the SLAM graph. We describe ELCH in the context of SLAM with 3D scans considering 6 DoF. The performance is evaluated using ground truth data of an urban environment.
Magnusson, M., Andreasson, H., Nüchter, A., Lilienthal, A.J.: Appearance-Based Place Recognition from 3D Laser Data Using the Normal Distributions Transform.Proceedings of the IEEE International Conference Robotics and Automation (ICRA '09). p. 23--28. , Kobe, Japan (2009).
To advance robotic science it is important to perform experiments that can be replicated by other researchers to compare different methods. However, these comparisons tend to be biased, since re-implementations of reference methods often lack thoroughness and do not include the hands-on experiences obtained during the original development process. This paper presents the results of a field experiment, carried out by two research groups that are leading in the field of 3D robotic mapping. The iterative closest points algorithm (ICP) is compared to the normal distributions transform (NDT). We also present an improved version of NDT with a substantially larger valley of convergence than previously published versions.
Frintrop, S., Nüchter, A., Pervölz, K., Surmann, H., Mitri, S., Hertzberg, J.: Attentive Classification.International Journal of Applied Artificial Intelligence in Engineering Systems.1, (2009).
In this paper, we present a two-step approach for object recognition based on principles of human perception: Attentive Classification. First, regions of interest are detected by a biologically motivated attention system. Second, these regions are analyzed by a fast classifier based on the Adaboost learning technique. Thus, the classification effort is restricted to a small data subset. The approach has two advantages over normal classification: First, the system becomes considerably faster, which is an important factor for real-time systems. Second, since the attention system is able to make use of top-down target-information, the combination of the systems yields a significant reduction of false detections for objects which are usually difficult to discriminate from the surrounding. We show the performance of the system in several experiments in robotic scenarios. The presented attentive classification system represents an important step towards effective general object recognition which is fast, robust and flexibly adaptable to a current task.
Magnusson, M., Andreasson, H., Nüchter, A., Lilienthal, A.J.: Automatic Appearance-Based Loop Detection from 3D Laser Data Using the Normal Distributions Transform.Journal of Field Robotics (JFR), Special Issue on Three-Dimensional Mapping.26,934--965 (2009).
We propose a new approach to appearance-based loop detection for mobile robots, using three-dimensional (3D) laser scans. Loop detection is an important problem in the simultaneous localization and mapping (SLAM) domain, and, because it can be seen as the problem of recognizing previously visited places, it is an example of the data association problem. Without a flat-floor assumption, two-dimensional laser-based approaches are bound to fail in many cases. Two of the problems with 3D approaches that we address in this paper are how to handle the greatly increased amount of data and how to efficiently obtain invariance to 3D rotations. We present a compact representation of 3D point clouds that is still discriminative enough to detect loop closures without false positives (i.e., detecting loop closure where there is none). A low false-positive rate is very important because wrong data association could have disastrous consequences in a SLAM algorithm. Our approach uses only the appearance of 3D point clouds to detect loops and requires no pose information. We exploit the normal distributions transform surface representation to create feature histograms based on surface orientation and smoothness. The surface shape histograms compress the input data by two to three orders of magnitude. Because of the high compression rate, the histograms can be matched efficiently to compare the appearance of two scans. Rotation invariance is achieved by aligning scans with respect to dominant surface orientations. We also propose to use expectation maximization to fit a gamma mixture model to the output similarity measures in order to automatically determine the threshold that separates scans at loop closures from nonoverlapping ones. We discuss the problem of determining ground truth in the context of loop detection and the difficulties in comparing the results of the few available methods based on range information. Furthermore, we present quantitative performance evaluations using three real-world data sets, one of which is highly self-similar, showing that the proposed method achieves high recall rates (percentage of correctly identified loop closures) at low false-positive rates in environments with different characteristics.
Magnusson, M., A. Nüchter, C.L., Lilienthal, A.J., Hertzberg, J.: Evaluation of 3D Registration Reliability and Speed -- A Comparison of ICP and NDT.Proceedings of the IEEE International Conference Robotics and Automation (ICRA '09). p. 3907--3912. , Kobe, Japan (2009).
To advance robotic science it is important to perform experiments that can be replicated by other researchers to compare different methods. However, these comparisons tend to be biased, since re-implementations of reference methods often lack thoroughness and do not include the hands-on experience obtained during the original development process. This paper presents a thorough comparison of 3D scan registration algorithms based on a 3D mapping field experiment, carried out by two research groups that are leading in the field of 3D robotic mapping. The iterative closest points algorithm (ICP) is compared to the normal distributions transform (NDT). We also present an improved version of NDT with a substantially larger valley of convergence than previously published versions.
Qiu, D., May, S., Nüchter, A.: GPU-accelerated Nearest Neighbor Search for 3D Registration.Proceedings of the 7th International Conference on Computer Vision Systems (ICVS '09). p. 194--203. , Liège, Belgium (2009).
Nearest Neighbor Search (NNS) is employed by many computer vision algorithms. The computational complexity is large and constitutes a challenge for real-time capability. The basic problem is in rapidly processing a huge amount of data, which is often addressed by means of highly sophisticated search methods and parallelism. We show that NNS based vision algorithms like the Iterative Closest Points algorithm (ICP) can achieve real-time capability while preserving compact size and moderate energy consumption as it is needed in robotics and many other domains. The approach exploits the concept of general purpose computation on graphics processing units (GPGPU) and is compared to parallel processing on CPU. We apply this approach to the 3D scan registration problem, for which a speed-up factor of 88 compared to a sequential CPU implementation is reported.
Nüchter, A., Elseberg, J.: Linearisierte Lösung der ICP-Fehlerfunktion f"ur global konsistentes Scanmatching.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2009, Fachhochschule Oldenburg/Ostfr./Whv. p. 74--81 (2009).
Dieser Artikel beschreibt eine Linearisierung in geschlossener Form für die Minimierung der Fehlerfunktion, die beim ICP-Algorithmus auftaucht. Die Linearisierung approximiert die tatsächliche Lösung und nutzt die Annahme aus, dass die auftretenden Winkel klein sind. Weiterhin zeigt der Artikel die Möglichkeit auf, die Linearisierung für global konsistentes Scanmatching zu verwenden. Global konsistentes Scanmatching minimiert den Gesamtfehler, wenn mehr als zwei 3D-Punktwolken vorliegen.
Nüchter, A.: Parallel and Cached Scan Matching for Robotic 3D Mapping.Journal of Computing and Information Technology (eCIT).17,51--65 (2009).
Intelligent autonomous acting of mobile robots in unstructured environments requires 3D maps. Since manual mapping is a tedious job, automatization of this job is necessary. Automatic, consistent volumetric modeling of environments requires a solution to the simultaneous localization and map building problem (SLAM problem). In 3D this task is computationally expensive, since the environments are sampled with many data points with state of the art sensing technology. In addition, the solution space grows exponentially with the additional degrees of freedom needed to represent the robot pose. Mapping environments in 3D must regard six degrees of freedom to characterize the robot pose. This paper summarizes our 6D SLAM algorithm and presents novel algorithmic and technical means to reduce computation time, i.e., the data structure cached k-d tree and parallelization. The availability of multi-core processors as well as efficient programming schemes as OpenMP permit the parallel execution of robotics tasks.
May, S., Dröschel, D., Holz, D., Fuchs, S., Nüchter, A.: Robust 3D-Mapping with Time-of-Flight Cameras.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '09). p. 1673--1678. , St. Louis, MO, USA (2009).
Time-of-flight cameras constitute a smart and fast technology for 3D perception but lack in measurement precision and robustness. The authors present a comprehensive approach for 3D environment mapping based on this technology. Imprecision of depth measurements are properly handled by calibration and application of several filters. Robust registration is performed by a novel extension to the Iterative Closest Point algorithm. Remaining registration errors are reduced by global relaxation after loop-closure and surface smoothing. A laboratory ground truth evaluation is provided as well as 3D mapping experiments in a larger indoor environment.
Magnusson, M., Nüchter, A., Lörken, C., Lilienthal, A.J., Hertzberg, J.: 3D Mapping the Kvarntorp Mine: A Field Experiment for Evaluation of 3D Scan Matching Algorithms.Proceedings of the Workshop on 3D-Mapping at the IEEE International Conference on Intelligent Robots and Systems (IROS '08). , Nice, France (2008).
Wulf, O., Nüchter, A., Hertzberg, J., Wagner, B.: Benchmarking Urban 6D SLAM.Journal of Field Robotics (JFR).25,148--163 (2008).
Quite a number of approaches for solving the simultaneous localization and mapping (SLAM) problem exist by now. Some of them have recently been extended to mapping environments with six degrees of freedom (DoF) poses, yielding 6D SLAM approaches. To demonstrate the capabilities of the respective algorithms, it is common practice to present generated maps and successful loop closings in large outdoor environments. Unfortunately, it is non-trivial to compare different 6D SLAM approaches objectively, because ground truth data about the outdoor environments used for demonstration is typically unavailable. We present a novel benchmarking method for generating this ground truth data based on reference maps. The method is then demonstrated by comparing the absolute performance of some previously existing 6D SLAM algorithms which build a large urban outdoor map.
Hertzberg, J., Lingemann, K., Lörken, C., Nüchter, A., Stiene, S.: Does it help a robot navigate to call navigability an affordance?Towards Affordance-Based Robot Control. Proceedings of Dagstuhl Seminar 06231, Dagstuhl Castle. p. 16--26. Springer LNAI (2008).
Gibson’s notion of affordance seems to attract roboticists’ attention. On a phenomenological level, it allows functions, which have “somehow” been implemented, to be described using a new terminology. However, that does not mean that the affordance notion is of help for building robots and their controllers. This paper explores viewing an affordance as an abstraction from a robot-environment relation that is of inter-individual use, but requires an individual implementation. Therefore, the notion of affordance helps share environment representations and theories among robots. Examples are given for navigability, as afforded by environments of different types to robots of different undercarriages and sensor configurations. Work in parts done in the projects (1) LISA, which is funded by the German Federal Ministry of Education and Research (BMBF) within the Framework Concept ”Research for Tomorrow’s Production” (fund number 02PB2170-02PB2177) and managed by the Project Management Agency Forschungszentrum Karlsruhe, Production and Manufacturing Technologies Division (PTKA-PFT); and (2) MACS, which is funded by the European Commission’s 6th Framework Programme IST Project MACS under contract/grant number FP6-004381. The Commission’s support is gratefully acknowledged.
Nüchter, A., Lingemann, K., Hertzberg, J.: Evaluating a 3D Camera for RoboCup Rescue.Proceedings of the SICE Annual Conference 2008: International Conference on Instrumentation, Control and Information Technology (SICE '08). p. 2070--2075. , Tokyo, Japan (2008).
The following paper evaluates a time-of-flight 3D camera with regards to its usability for RoboCup rescue. This includes an evaluation of the influence of outer conditions to the camera data, as well as its usage for automatic 3D mapping by scan registration. A color camera is calibrated with respect to the 3D camera in order to gain colored texture information for the acquired measurements.
Nüchter, A., Lingemann, K., Bormann, D., Elseberg, J., Böhm, J.: Global Konsistente 3D-Kartierung mit Scanmatching.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2008, Fachhochschule Oldenburg/Ostfr./Whv (2008).
Das Einpassen bzw. Registrieren von Punktmengen unter starren Transformationen ist eines der Grundprobleme in der Bildverarbeitung. Hierbei können die 3D-Daten aus Laserscannern, Stereokameras u.ä. stammen. Für zwei 3D-Punktmengen bildet der ICP-Algorithmus (Iterative Closest Points) einen de facto Standard für das Registrieren. Weitet man diesen Algorithmus jedoch auf viele 3D-Scans aus, akkumulieren sich Registrierungsfehler. Der vorliegende Beitrag skizziert einen neuen Algorithmus für das global konsistente 3D-Scanmatching (Borrmann, Elseberg, Lingemann, Nüchter und Hertzberg 2008) und zeigt eine Anwendung für das Erfassen von Gebäuden. Die Genauigkeit des Algorithmus wird bezüglich geodätischer Methoden evaluiert.
Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A., Hertzberg, J.: Globally consistent 3D mapping with scan matching.Journal Robotics and Autonomous Systems (JRAS).56,130--142 (2008).
A globally consistent solution to the simultaneous localization and mapping (SLAM) problem in 2D with three degrees of freedom (DoF) poses was presented by Lu and Milios [F. Lu, E. Milios, Globally consistent range scan alignment for environment mapping, Autonomous Robots 4 (April) (1997) 333-349]. To create maps suitable for natural environments it is however necessary to consider the 6DoF pose case, namely the three Cartesian coordinates and the roll, pitch and yaw angles. This article describes the extension of the proposed algorithm to deal with these additional DoFs and the resulting non-linearities. Simplifications using Taylor expansion and Cholesky decomposition yield a fast application that handles the massive amount of 3D data and the computational requirements due to the 6DoF. Our experiments demonstrate the functionality of estimating the exact poses and their covariances in all 6DoF, leading to a globally consistent map. The correspondences between scans are found automatically by use of a simple distance heuristic.
Wiemann, T., Nüchter, A., Lingemann, K., Stiene, S., Hertzberg, J.: Surface Reconstruction for 3D Robotic Mapping.Proceedings of the Workshop on 3D-Mapping at the IEEE International Conference on Intelligent Robots and Systems (IROS '08). , Nice, France (2008).
Borrmann, D., Elseberg, J., Lingemann, K., Nüchter, A., Hertzberg, J.: The Efficient Extension of Globally Consistent Scan Matching to 6 DoF.Proceedings of the 4th International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT '08). p. 29--36. , Atlanta, USA (2008).
Over ten years ago, Lu and Milios presented a probabilistic scan matching algorithm for solving the simultaneous localization and mapping (SLAM) problem with 2D laser range scans, a standard in robotics. This paper presents an extension to this GraphSLAM method. Our iterative algorithm uses a sparse network to represent the relations between several overlapping 3D scans, computes in every step the 6 degrees of freedom (DoF) transformation in closed form and exploits efficient data association with cached k-d trees. Our approach leads to globally consistent 3D maps, precise 6D pose and covariance estimates, as demonstrated by various experimental results.
Nüchter, A., Hertzberg, J.: Towards Semantic 3D Maps.Journal Robotics and Autonomous Systems (JRAS), Special Issue on Semantic Knowledge in Robotics.56,915--926 (2008).
Intelligent autonomous action in ordinary environments calls for 3D maps. 3D geometry is necessary for avoiding collision with complex obstacles and to self localize in six degrees of freedom (6 DoF) (x, y, z positions, roll, yaw, and pitch angles). Meaning, in addition to geometry, becomes inevitable if the robot is supposed to interact with its environment in a goal-directed way. A semantic stance enables the robot to reason about objects; it helps disambiguate or round off sensor data; and the robot knowledge becomes reviewable and communicable. The paper describes an approach and a completed robot system for semantic mapping. The prime sensor is a 3D laser scanner. Individual scans are registered into a coherent 3D geometry map by 6D SLAM. Coarse scene features (e.g., walls, floors in a building) are determined by semantic labeling. More delicate objects are then detected by a trained classifier and localized. In the end, the semantic maps can be visualized for human inspection. We sketch the overall architecture of the approach, explain the respective steps and their underlying algorithms, give examples based on a working robot implementation, and discuss the findings.
Nüchter, A., Lingemann, K., Hertzberg, J., Surmann, H.: 6D SLAM -- 3D Mapping Outdoor Environments.Journal of Field Robotics (JFR), Special Issue on Quantitative Performance Evaluation of Robotic and Intelligent Systems.24,699--722 (2007).
6D SLAM (Simultaneous Localization and Mapping) or 6D Concurrent Localization and Mapping of mobile robots considers six dimensions for the robot pose, namely, the x, y and z coordinates and the roll, yaw and pitch angles. Robot motion and localization on natural surfaces, e.g., driving outdoor with a mobile robot, must regard these degrees of freedom. This paper presents a robotic mapping method based on locally consistent 3D laser range scans. Iterative Closest Point (ICP) scan matching, combined with a heuristic for closed loop detection and a global relaxation method, results in a highly precise mapping system. A new strategy for fast data association, cached kd tree search, leads to feasible computing times. With no ground-truth data available for outdoor environments, point relations in maps are compared to numerical relations in uncalibrated aerial images in order to assess the metric validity of the resulting 3D maps.
Nüchter, A., Lingemann, K., Hertzberg, J.: 6D SLAM with Cached k-d tree Search.Proceedings of the 13th IASTED International Conference on Robotics and Applications (RA '07). p. 181--186. , Würzburg, Germany (2007).
6D SLAM (Simultaneous Localization and Mapping) or 6D Concurrent Localization and Mapping of mobile robots considers six degrees of freedom for the robot pose, namely, the x, y and z coordinates and the roll, yaw and pitch angles. In previous work we presented our scan matching based 6D SLAM approach [cite deleted], where scan matching is based on the well known iter ative closest point (ICP) algorithm [3]. Effcient implementations of this algorithm are a result of a fast computation of closest points. The usual approach, i.e., using k-d trees is extended in this paper. We describe a novel search strategy, that lead s to significant speed-ups. Our mapping system is real-time capable, i.e., 3D maps are computed using the resources of the used robotic hardware.
Nüchter, A., Lingemann, K., Hertzberg, J.: 6D SLAM with Kurt3D.Robotics Today, Society of Manufacturing Engineers, First Quater.20, (2007).
Nüchter, A.: Algorithmen zur Erstellung virtueller 3D-Welten mit mobilen Robotern.Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2007, Fachhochschule Oldenburg/Ostfr./Whv (2007).
3D-Laserscanner sind eine junge Technologie, die die Erfassung räumlicher Daten revolutioniert und Robotern das dreid imensionale Abtasten von Objekten möglich macht. Roboter, die ihre Umgebung dreidimensional kartieren können, eignen sich zum automatischen Erstellen virtuelle 3D-Welten. Eine 3D-Welt, oder 3D-Umgebungskarte muss mit der wirklichen Umgebung übereinstimmen, also korrekt und konsistent sein. Ist die Position des Roboters genau bekannt, können die loka len Karten auf der Grundlage dieser Position zusammengefügt werden. Leider ist die Selbstlokalisation eines Roboters stets fehlerbehaftet. Daher darf der Kartenbau nicht nur auf der Roboterposition basieren, sondern muss auch auf der Grundlage der Sensorwerte geschehen. In diesem Zusammenhang spricht man vom simultanen Lokalisations- und Kartierungs problem (SLAM, simultaneous localization and mapping problem). Korrekte, global konsistente Modelle entstehen durch e inen 6D-SLAM Algorithmus. Hierbei werden 6 Freiheitsgrade in der Roboterpose berücksichtigt, geschlossene Kreise erka nnt und der globale Fehler minimiert. Die Basis des 6D-SLAM ist ein sehr schneller ICP-Algorithmus.
Stiene, S., Nüchter, A., Lingemann, K., Hertzberg, J.: An Experiment in Semantic Correction of Sensor Data.Proceedings Workshop on Semantic Information in Robotics at the IEEE International Conference Robotics and Automation (ICRA '07). , Rome, Italy (2007).
Steger, J., Märtin, R., Lingemann, K., Nüchter, A., Hertzberg, J., König, P.: Assessing stereo matching algorithms using ground-truth disparity maps of natural scenes.Proceedings of the 7th Meeting of the German Neuroscience Society / 31th Göttingen Neurobiology Conference, Neuroforum 2007. , Göttingen, Germany (2007).
Nüchter, A., Lingemann, K., Hertzberg, J.: Cached \($k$\)-d tree search for ICP Algorithms.Proceedings of the 6th IEEE International Conference on Recent Advances in 3D Digital Imaging and Modeling (3DIM '07). p. 419--426. , Montreal, QC, Canada (2007).
Nüchter, A.: Effiziente Speicherung großer Punktwolken -- Datenstrukturen für Algorithmen für mobile und terrestrische Laserscansysteme.Terrestrisches Laserscanning (TLS 2016) Beiträge zum 154. DVW-Seminar am 28. und 29. November in Fulda. p. 105--120. , Fulda, Germany (2016).
Nüchter, A., Borrmann, D., Elseberg, J., Redondo, D.: A Backpack-Mounted 3D Mobile Scanning System.Allgemeine Vermessungs-Nachrichten (AVN), Special Issue MoLAS 2014.122,301--307 (2015).
Hertzberg, J., Lingemann, K., Nüchter, A.: Mobile Roboter: Eine Einführung aus Sicht der Informatik.Springer, Heidelberg, Germany (2012).