-
Bleier, M., Almeida, C., Ferreira, A., Pereira, R., Matias, B., Almeida, J., Pidgeon, J., van der Lucht, J., Schilling, K., Martins, A., Silva, E., Nüchter, A.: 3D UNDERWATER MINE MODELLING IN THE VAMOS PROJECT. Proceedings of the 2nd ISPRS International Workshop Underwater 3D Recording & Modelling: A tool for modern applications and CH recording. p. 105--112. , Limassol, Cyprus (2019).
The project Viable Alternative Mine Operating System (VAMOS) develops a novel underwater mining technique for extracting inland mineral deposits in flooded open-cut mines. From a floating launch and recovery vessel a remotely-operated underwater mining vehicle with a roadheader cutting machine is deployed. The cut material is transported to the surface via a flexible riser hose. Since there is no direct intervisibility between the operator and the mining machine, the data of the sensor systems can only be perceived via a computer interface. Therefore, part of the efforts in the project focus on enhancing the situational awareness of the operator by providing a 3D model of the mine combined with representations of the mining equipment and sensor data. We present a method how a positioning and navigation system, perception system and mapping system can be used to create a replica of the physical system and mine environment in Virtual Reality (VR) in order to assist remote control. This approach is beneficial because it allows visualizing different sensor information and data in a consistent interface, and enables showing the complete context of the mining site even if only part of the mine is currently observed by surveying equipment. We demonstrate how the system is used during tele-operation and show results achieved during the field trials of the complete system in Silvermines, Ireland.
-
Torrese, P., Rossi, A.P., Unnithan, V., Borrmann, D., Lauterbach, H., Luzzi, E., Pozzobon, R., Sauro, F., Bessone, L., Nüchter, A.: Imaging the subsurface of planetary volcanic analogues using ambient seismic noise data at the Tinguatón Volcano (Lanzarote, Canary Islands). Proceedings of the EGU General Assembly 2019, Poster: New mission concepts, enabling technologies and terrestrial analogue studies for planetary explorationn. , Vienna, Austria (2019).
-
van der Lucht, J., Bleier, M., Leutert, F., Schilling, K., Nüchter, A.: Korrektur der Brechung an der Wasseroberfläche beim triangulationsbasierten 3D-Laserscannen. Allgemeine Vermessungs-Nachrichten (AVN). 126, 43--52 (2019).
Die vorliegende Arbeit beschäftigt sich mit der Korrektur der Brechung an der Wasseroberfläche beim triangulationsbasierten 3D Laserscannen. Hierzu wurde eine Methode entwickelt, um mit einem Structured Light (SL) System 3D-Daten von teilweise mit Wasser bedeckten Strukturen anzufertigen und die dabei entstehende Brechung an der Wasseroberfläche zu korrigieren. Diese wurde anschließend in einem Versuchsaufbau evaluiert. Der Scanner wurde dabei an einem KUKA KR-16 Manipulatorarm befestigt, um den Scanner gleichmäßig, definiert und wiederholbar zu bewegen. Die Bewegungen des Scanners wurden dabei durch ein externes Trackingsystem erfasst. Ebenfalls wurde bei diesen Versuchen der Einfluss verschiedener Einstrahlwinkel betrachtet. Zu diesem Zweck wurde der Scanner in verschiedenen Winkeln relativ zur Wasseroberfläche positioniert. Durch die entwickelte Methode konnten Fehler durch die Brechung an der Wasseroberfläche erfolgreich korrigiert werden. Außerdem konnte die Lage der Wasseroberfläche ohne externe Markierungen aus den 3D-Daten bestimmt werden.
-
Edlinger, R., Nüchter, A.: MARC -- Modular Autonomous Adaptable Robot Concept. IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '19). p. 1--7 (2019).
The paper introduces a novel modular and adaptable payload concept for plugging in sensor and actuator platforms such as 3D LIDAR and visual sensor systems and robot manipulator and gripper systems. Integration, programming and operation of heterogeneous robot systems (such as mobile manipulators or robots in a machine network) are very complex tasks for plant operators. Heterogeneous system components have to be orchestrated (via proprietary interfaces) by higher-level control systems. Robot programs are created offline, in proprietary tools, and used through macros. System components are generally not compatible and interchangeable across manufacturers. These have to be programmed separately in manufacturer-specific tools. Cross-component debugging is difficult. The operation of complex systems is difficult, requires intensive training, and is currently limited to simple graphical user interfaces (GUI). As part of process optimization, it is usually necessary to optimize process points during the start-up phase. This requires highly skilled personnel capable of robot programming at the plant operators site. The combination of heterogeneous robot modules results in many new hazardous situations. It is therefore necessary to include an appropriate safety concept. The goal is to design a robot payload concept with a plug-and-play approach to be used as a modular and flexible unit. This shall reduce the effort for system integration and sensor calibrating significantly and provide a customized perception of the environment during certain work processes. The modular, autonomous and adaptable robot concept with several sensors and hardware components was implemented as a prototype on a rescue robot. The modules have already been integrated on other autonomous vehicles for exploration and dexterity tasks.
-
Lauterbach, H.A., Borrmann, D., Nüchter, A., Rossi, A.P., Unnithan, V., Torrese, P., Pozzobon, R.: Mobile mapping of the La Corona lavatube on Lanzarote. Proceedings of the ISPRS Geospatial Week 2019, Laserscanning 2019. p. 381--387. , Enschede, Netherlands (2019).
Planetary surfaces consist of rough terrain and cave-like environments. Future planetary exploration demands for accurate mapping. However, recent backpack mobile mapping systems are mostly tested in structured, indoor environments. This paper evaluates the use of a backpack mobile mapping system in a cave-like environment. The experiments demonstrate the abilities of an continuous-time optimization approach by mapping part of a lavatube of the La Corona volcano system on Lanzarote. We compare two strategies for trajectory estimation relying either on 2D or 3D laser scanners and show that a 3D laser scanner substantially improved the final results.
-
Nüchter, A.: Position and orientation of sensors for unmanned vehicle systems. In: Patias, P. and Armenakis, C. (eds.) Unmanned Vehicle Systems for Geomatics -- Towards Robotic Mapping. Whittles Publishing (2019).
-
Dumic, E., Bjelopera, A., Nüchter, A.: Projection based dynamic point cloud compression using 3DTK toolkit and H.265/HEVC. Proceedings of the 2nd International Colloquium on Smart Grid Metrology (SMAGRIMET''19). p. 1--4. IEEE Xplore, Split, Croatia (2019).
This paper presents novel compression method for dynamic point clouds based on projections and the H.265/HEVC video coder. We used 3DTK - The 3D Toolkit to create equirect-angular projection images and x265 as H.265/HEVC coder, to compress and extract created projection images. Compression was introduced in 3DTK generated projection image size (differ-ent pixel size, but compatible with later video compression), as well as lossless/lossy video compression. Visual inspection shows better results for compression only using different projection resolution, with lossless video compression. Lossy video compression adds noise and creates additional points, resulting in lower visual quality.
-
Borrmann, D., Jörissen, S., Nüchter, A.: RADLER -- Ein Einrad als RADialer LasER Scanner. Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2019, Jade Hochschule. p. 12--21 (2019).
Die Weiterentwicklung in der 3D-Messtechnik hat in den letzten Jahren eine Vielzahl an unterschiedlichen Messgeräten zu 3D-Umgebungserfassung hervorgebracht, jedes davon mit seinen eigenen Vorzügen und Schwächen. Diese Arbeit stellt ein modifiziertes Einrad vor, bei dem ein an der Radachse befestigter 2D Laserscanner ein radiales 3D Scanpattern erzeugt. Dieses neuartige, kostengünstige Messgerät verbindet die Vorteile von radgetriebenen Scangeräten mit denen von Handscannern. Nach der Vorstellung des Hardwareaufbaus und der Sensorintegration werden die Ergebnisse anhand von drei Testszenarien ausgewertet und mit den Daten eines terrestrischen 3D-Laserscanners verglichen.
-
Jörissen, S., Bleier, M., Nüchter, A.: Self-calibrated Surface Acquisition for Integrated Positioning Verification in Medical Applications. Proceedings of the IS&T International Symposium on Electronic Imaging Science and Technology. pp. 353-1--353-8. Society for Imaging Science and Technology, San Francisco, CA, USA (2019).
This paper presents a novel approach for a position verification system in medical applications. By replacing the already existing cross line laser projectors with galvo- or MEMS-based projectors and utilizing the surveillance cameras, a self-calibration of the system is performed and surface acquisition for positioning verification is demonstrated. The functionality is shown by analyzing the radii of calibration spheres and determining the quality of the captured surface with respect to a reference model. The paper focuses on the demonstration with one pair of camera and projector but can also be extended to a multi-camera-projector system, as present in treatment rooms. Compared to other systems, this approach does not need external hardware and is thus space and cost efficient.
-
Koch, P., May, S., Engelhardt, H., Ziegler, J., Nüchter, A.: Signed Distance Based Reconstruction for Exploration and Change Detection in Underground Mining Disaster Prevention. IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '19). p. 1--2 (2019).
This publication describes an application of a Truncated Signed Distance Mapping approach for disaster intervention in underground mine shafts through geometrical change detection of the shaft walls. The paper describes two main problems of such an approach (aligning two potentially huge point clouds and automatic change detection by comparing the reconstructed volumes) and explains in detail the proposed solution.
-
Lauterbach, H.A., Koch, C.B., Hess, R., Eck, D., Schilling, K., Nüchter, A.: The Eins3D Project -- Instantaneous UAV-Based 3D Mapping for Search and Rescue Applications. IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '19). p. 1--6 (2019).
The overview of a situation in a search and rescue disaster is the key aspect of an effective assistance. In the recent years the utilization of multicopter with various photogrammetry systems is an upcoming trend and an open field of research. This paper discusses the technical aspects of an automated integral system that will support rescuers during the strategic mission planning and will give situational awareness by instantaneous 3D mapping. The approach combines sensors including a 3D Laserscanner, a thermal camera and an attitude system as a payload unit on an Multicopter. The continuous data fusion and the down link are providing an instant 3D environment map that is continuously revised and updated.
-
Heydn, K.A.M., Dietrich, M.P., Barkowsky, M., Winterfeldt, G., von Mammen, S., Nüchter, A.: The Golden Bullet: A Comparative Study for Target Acquisition, Pointing and Shooting. 11th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games '19). p. 1--8 (2019).
In this study, we evaluate an interaction sequence performed by six modalities consisting of desktop-based (DB) and virtual reality (VR) environments using different input devices. For the given study, we implemented a vertical prototype of a first person shooter (FPS) game scenario, focusing on the genre-defining point-and-shoot mechanic. We introduce measures to evaluate the success of the according interaction sequence (times for target acquisition, pointing, shooting, overall net time, and number of shots) and conduct experiments to record and compare the users' performances. We show that interacting using head-tracking for landscape-rotation is performing similarly to the input of a screen-centered mouse and also yielded shortest times in target acquisition and pointing. Although using head-tracking for target acquisition and pointing was most efficient, subjects rated the modality using head-tracking for target acquisition and a 3DOF Controller for pointing best. Eye-tracking (ET) yields promising results, but calibration issues need to be resolved to enhance reliability and overall user experience.
-
Bleier, M., van der Lucht, J., Nüchter, A.: Towards an Underwater 3D Laser Scanning System for Mobile Mapping. Proceedings of the IEEE ICRA Workshop on Underwater Robotic Perception (ICRAURP '19). , Montreal, Canada (2019).
Digitizing archaeological or industrial sites on land using standard methods of photogrammetry like 3D modeling from photos or laser scanning is well understood. In contrast, precise underwater surveying with high resolution is still a complex and difficult task. In this paper, we present the development and construction of a structured light underwater laser scanning system and show first results on applying the system for mobile scanning in the water. The laser scanner employs two line lasers to project a cross on the scene. This enables mobile scanning in multiple directions and provides an overlapping scan pattern, which is exploited for trajectory optimization. We describe the image processing, calibration and 3D reconstruction methods used for creating point clouds using the system. In experiments conducted in a towing tank we demonstrate 3D scans captured by rotating the scanner on a robotic joint and first results of mobile scans acquired by moving the scanner through the water along a linear trajectory.
-
Rossi, A.P., Unnithan, V., Torrese, P., Borrmann, D., Nüchter, A., Lauterbach, H., Ortenzi, G., Jährig, T., Sohl, F.: Augmented field Geology and Geophysics for Planetary Analogues (Poster). Proceedings of the EGU General Assembly 2018, Pico session: Planetary geobiological analogs for Mars and beyond: Field, lab and simulations. , Vienna, Austria (2018).
-
Martell, A., Lauterbach, H.A., Schilling, K., Nüchter, A.: Benchmarking Structure from Motion Algorithms of Urban Environments with Applications to Reconnaissance in Search and Rescue Scenarios. Proceedings of the 16th IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '18). p. 1--7. , Philadelphia, PA, USA (2018).
-
Pfitzner, C., May, S., Nüchter, A.: Body Weight Estimation for Dose-Finding and Health Monitoring of Lying, Standing and Walking Patients Based on RGB-D Data. Sensors. 18, (2018).
This paper describes the estimation of the body weight of a person in front of an RGB-D camera. A survey of different methods for body weight estimation based on depth sensors is given. First, an estimation of people standing in front of a camera is presented. Second, an approach based on a stream of depth images is used to obtain the body weight of a person walking towards a sensor. The algorithm first extracts features from a point cloud and forwards them to an artificial neural network (ANN) to obtain an estimation of body weight. Besides the algorithm for the estimation, this paper further presents an open-access dataset based on measurements from a trauma room in a hospital as well as data from visitors of a public event. In total, the dataset contains 439 measurements. The article illustrates the efficiency of the approach with experiments with persons lying down in a hospital, standing persons, and walking persons. Applicable scenarios for the presented algorithm are body weight-related dosing of emergency patients.
-
Nüchter, A., Bleier, M., Schauer, J., Janotta, P.: Continuous-Time SLAM --- Improving Google’s Cartographer 3D Mapping. In: Remondino, F., Georgopoulos, A., Gonzalez-Aguilera, D., and Agrafiotis, P. (eds.) Latest Developments in Reality-Based 3D Surveying and Modelling. p. 53--73. MDPI, Basel, Switzerland (2018).
This paper shows how to use the result of Google’s simultaneous localization and mapping (SLAM) solution, called Cartographer, to bootstrap a continuous-time SLAM algorithm that was developed by the authors and presented in previous publications. The presented approach optimizes the consistency of the global point cloud, and thus improves on Google’s results. Algorithms and data from Google are used as input for the continuous-time SLAM software. In preceding work, the continuous-time SLAM was successfully applied to a similar backpack system which delivers consistent 3D point clouds even in the absence of an IMU. Continuous-time SLAM means that the trajectory of a mobile mapping system is treated in a semi-rigid fashion, i.e., the trajectory is deformed to yield a consistent 3D point cloud of the measured environment.
-
van der Lucht, J., Bleier, M., Leutert, F., Schilling, K., Nüchter, A.: Korrektur der Brechung an der Wasseroberfläche beim triangulationsbasierten 3D-Laserscannen. Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2018, Jade Hochschule. p. 87--102 (2018).
Die vorliegende Arbeit beschäftigt sich mit der Korrektur der Brechung an der Wasseroberfläche beim triangulationsbasierten 3D Laserscannen. Hierzu wurde eine Methode entwickelt, die es ermöglicht, mit einem Structured Light (SL) System 3D-Daten von teilweise mit Wasser bedeckten Strukturen anzufertigen und die dabei entstehende Brechung an der Wasseroberfläche zu korrigieren. Diese wurde anschließend in einem Versuchsaufbau evaluiert. Der Scanner wurde dabei an einem KUKA KR-16 Manipulatorarm befestigt, was die Möglichkeit bietet den Scanner gleichmäßig, definiert und wiederholbar zu bewegen. Die Bewegungen des Scanners wurden dabei durch ein externes Trackingsystem erfasst. Ebenfalls wurde bei diesen Versuchen der Einfluss verschiedener Einstrahlwinkel betrachtet. Zu diesem Zweck wurde der Scanner in verschiedenen Winkeln relativ zur Wasseroberfläche bewegt. Durch die entwickelte Methode konnten Fehler durch die Brechung an der Wasseroberfläche erfolgreich korrigiert werden. Außerdem konnte die Lage der Wasseroberfläche ohne externe Markierungen aus den 3D-Daten bestimmt werden.
-
Almeida, J., Martins, A., Almeida, C., Dias, A., Matias, B., Ferreira, A., Jorge, P., Martins, R., Bleier, M., Nüchter, A., Pidgeon, J., Kapusniak, S., Silva, E.: Positioning. Navigation and Awareness of the !VAMOS! Underwater Robotic Mining System. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '18). p. 1527--1533 (2018).
This paper presents the positioning, navigation and awareness (PNA) system developed for the Underwater Robotic Mining System of the !VAMOS! project [1]. It describes the main components of the !VAMOS! system, the PNA sensors in each of those components, the global architecture of the PNA system, and its main subsystems: Position and Navigation, Realtime Mine Modeling, 3D Virtual reality HMI and Real-time grade system. General results and lessons learn during the first mining field trial in Lee Moor, Devon, UK during the months of September and October 2017 are presented.
-
Lauterbach, H.A., Nüchter, A.: Preliminary Results on Instantaneous UAV-Based 3D Mapping for Rescue Applications. Proceedings of the 16th IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR '18). p. 1--2. , Philadelphia, PA, USA (2018).
This report presents a novel approach to generate a 3D map with an UAV while flying over a disaster scene with the aim to present the map instantaneously to the operator and the rescue workers. Our approach extends the well-known ICP algorithm.
-
Borrmann, D., Jörissen, S., Nüchter, A.: RADLER - A RADial LasER scanning device. Proceedings of the 16th International Symposium of Experimental Robotics (ISER '18). , Buenos Aires, Argentina (2018).
Mobile scanning, i.e., the practice of mounting laser scanners on moving platforms is an efficient way to acquire accurate and dense 3D point clouds of outdoor environments for urban and regional planning and architecture. The mobile scenario puts high requirements on the accuracy of the calibration of the measurement system, as small calibration inaccuracies lead to large errors in the resulting point cloud. We propose a novel algorithm for the calibration of a mobile scanning system that estimates the calibration parameters for all sensor components simultaneously without relying on additional hardware. We evaluate the calibration algorithm on several real world data sets where ground truth is available via an accurate geodetic model.
-
Torrese, P., Rossi, A.P., Unnithan, V., Borrmann, D., Lauterbach, H., Ortenzi, G., Jährig, T., Pozzobon, R., Sauro, F., Santagata, T., Nüchter, A., Sohl, F.: Reconstructing the subsurface of planetary volcanic analogues: ERT imaging of Lanzarote lava tubes complemented with drone stereogrammetry, surface and in-cave LiDAR and seismic investigations (Poster). Proceedings of the EGU General Assembly 2018, Pico session: Planetary geobiological analogs for Mars and beyond: Field, lab and simulations. , Vienna, Austria (2018).
-
Schauer, J., Nüchter, A.: Removing non-static objects from 3D laser scan data. ISPRS Journal of Photogrammetry and Remote Sensing (JPRS). 143, 15--38 (2018).
For the purpose of visualization and further post-processing of 3D point cloud data, it is often desirable to remove moving objects from a given data set. Common examples for these moving objects are pedestrians, bicycles and motor vehicles in outdoor scans or manufactured goods and employees in indoor scans of factories. We present a new change detection method which is able to partition the points of multiple registered 3D scans into two sets: points belonging to stationary (static) objects and points belonging to moving (dynamic) objects. Our approach does not require any object detection or tracking the movement of objects over time. Instead, we traverse a voxel grid to find differences in volumetric occupancy for “explicit” change detection. Our main contribution is the introduction of the concept of “point shadows” and how to efficiently compute them. Without them, using voxel grids for explicit change detection is known to suffer from a high number of false positives when applied to terrestrial scan data. Our solution achieves similar quantitative results in terms of F1-score as competing methods while at the same time being faster.
-
van der Lucht, J., Bleier, M., Leutert, F., Schilling, K., Nüchter, A.: Structured-light based 3D laser scanning of semi-submerged structures. Proceedings of the Technical Commision II Mid-term Symposium ``Towards Photogrammetry 2020''. p. 287--294. , Riva del Garda, Italy (2018).
In this work we look at 3D acquisition of semi-submerged structures with a triangulation based underwater laser scanning system. The motivation is that we want to simultaneously capture data above and below water to create a consistent model without any gaps. The employed structured light scanner consist of a machine vision camera and a green line laser. In order to reconstruct precise surface models of the object it is necessary to model and correct for the refraction of the laser line and camera rays at the water-air boundary. We derive a geometric model for the refraction at the air-water interface and propose a method for correcting the scans. Furthermore, we show how the water surface is directly estimated from sensor data. The approach is verified using scans captured with an industrial manipulator to achieve reproducable scanner trajectories with different incident angles. We show that the proposed method is effective for refractive correction and that it can be applied directly to the raw sensor data without requiring any external markers or targets.
-
Schauer, J., Nüchter, A.: The Peopleremover --- Removing Dynamic Objects From 3-D Point Cloud Data by Traversing a Voxel Occupancy Grid. IEEE Robotics and Automation Letters (RAL). 3, 1679--1686 (2018).
Even though it would be desirable for most postprocessing purposes to obtain a point cloud without moving objects in it, it is often impractical or downright impossible to free a scene from all nonstatic clutter. Outdoor environments contain pedestrians, bicycles, and motor vehicles which cannot easily be stopped from entering the sensor range and indoor environments like factory production lines cannot be evacuated due to production losses during the time of the scan. In this letter, we present a solution to this problem that we call the "peopleremover." Given a registered set of 3-D point clouds, we build a regular voxel occupancy grid and then traverse it along the lines of sight between the sensor and the measured points to find the differences in volumetric occupancy between the scans. Our approach works for scan slices from mobile mapping as well as for the more general scenario of terrestrial scan data. The result is a clean point cloud free of dynamic objects.
-
Schilling, K., Tzschichholz, T., Motroniuk, I., Aumann, A., Mammadov, I., Ruf, O., Schmidt, C., Appel, N., Kleinschrodt, A., Montenegro, S., Nüchter, A.: TOM: A formation for photogrammetric earth observation by three cubesats. Proceedings IAA-ASS-18 : International Academy of Astronautics. p. 1--14. Univelt, Inc., Changsha, China (2018).
-
Borrmann, D., Nüchter, A., Wiemann, T.: Workshop: Large-Scale 3D Point Cloud Processing for Mixed and Augmented Reality. IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct '08). p. xxxv (2018).
The rapid development of 3D scanning technology combined with state-of-the-art mapping algorithms allows to capture 3D point clouds with high resolution and accuracy. The high amount of data collected with LiDAR, RGB-D cameras or generated through SfM approaches makes the direct use of the recorded data for realistic rendering and simulation problematic. Therefore, these point clouds have to be transformed into representations that fulfill the computational requirements for VR and AR setups. In this tutorial participants will be introduced to state-of-the-art methods in point cloud processing and surface reconstruction with open source software to learn the benefits for AR and VR applications by interleaved presentations, software demonstrations and software trials. The focus lies on 3D point cloud data structures (range images, octrees, k-d trees) and algorithms, and their implementation in C/C++. Surface reconstruction using Marching Cubes and other meshing methods will play another central role. Reference material for subtopics like 3D point cloud registration and SLAM, calibration, filtering, segmentation, meshing, and large scale surface reconstruction will be provided. Participants are invited to bring their Linux, MacOS or Windows laptops to gain hands-on experience on practical problems occuring when working with large scale 3D point clouds in VR and AR applications.
-
Lehtola, V., Kaartinen, H., Nüchter, A.: Autonomous 3D Modelling of Indoor Spaces. GIM International. 31, 20--23 (2017).
Mobile scanning can be an equally accurate yet more cost-effective solution than traditional terrestrial laser scanning done with tripods. To succeed, however, mobile scanners not only require a suitable combination of sensors, but also reliable and continuous knowledge about where the scanners are located and the direction in which they are pointing during scanning. There are multiple ways to achieve this, which has led to the development of various scientific and commercial solutions. This article compares several mobile scanning solutions for 3D modelling of indoor spaces and highlights their strengths and weaknesses.
-
Leung, K.Y.K., Lühr, D., Houshiar, H., Inostroza, F., Borrmann, D., Adams, M., Nüchter, A., del Solar, J.R.: Chilean underground mine dataset. International Journal of Robotics Research (IJRR). 36, 16--23 (2017).
This article presents a robotic dataset collected from the largest underground copper mine in the world. The sensor measurements from a 3D scanning lidar, a 2D radar, and stereo cameras were recorded from an approximately two kilometer traverse of a production-active tunnel. The equipment used and the data collection process is discussed in detail, along with the format of the data. This dataset is suitable for research in robotic navigation, as well as simultaneous localization and mapping. The download instructions are available at the following website http://dataset.amtc.cl.
-
Lehtola, V., Kaartinen, H., Nüchter, A., Kaijaluoto, R., Kukko, A., Litkey, P., Honkavaara, E., Rosnell, T., Vaaja, M., Virtanen, J.-P., et al.,: Comparison of the Selected State-Of-The-Art 3D Indoor Scanning and Point Cloud Generation Methods. Remote Sensing. 796 (2017).
Accurate three-dimensional (3D) data from indoor spaces are of high importance for various applications in construction, indoor navigation and real estate management. Mobile scanning techniques are offering an efficient way to produce point clouds, but with a lower accuracy than the traditional terrestrial laser scanning (TLS). In this paper, we first tackle the problem of how the quality of a point cloud should be rigorously evaluated. Previous evaluations typically operate on some point cloud subset, using a manually-given length scale, which would perhaps describe the ranging precision or the properties of the environment. Instead, the metrics that we propose perform the quality evaluation to the full point cloud and over all of the length scales, revealing the method precision along with some possible problems related to the point clouds, such as outliers, over-completeness and misregistration. The proposed methods are used to evaluate the end product point clouds of some of the latest methods. In detail, point clouds are obtained from five commercial indoor mapping systems, Matterport, NavVis, Zebedee, Stencil and Leica Pegasus: Backpack, and three research prototypes, Aalto VILMA, FGI Slammer and the Würzburg backpack. These are compared against survey-grade TLS point clouds captured from three distinct test sites that each have different properties. Based on the presented experimental findings, we discuss the properties of the proposed metrics and the strengths and weaknesses of the above mapping systems and then suggest directions for future research.
-
Koch, R., May, S., Nüchter, A.: Detection and Purging of Specular Reflective and Transparent Object Influences in 3D Range Measurements. Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 377--384. , Nafplio, Greece (2017).
3D laser scanners are favoured sensors for mapping in mobile service robotics at indoor and outdoor applications, since they deliver precise measurements at a wide scanning range. The resulting maps are detailed since they have a high resolution. Based on these maps robots navigate through rough terrain, fulfil advanced manipulation, and inspection tasks. In case of specular reflective and transparent objects, e.g., mirrors, windows, shiny metals, the laser measurements get corrupted. Based on the type of object and the incident angle of the incoming laser beam there are three results possible: a measurement point on the object plane, a measurement behind the object plane, and a measurement of a reflected object. It is important to detect such situations to be able to handle these corrupted points. This paper describes why it is difficult to distinguish between specular reflective and transparent surfaces. It presents a 3D- Reflection-Pre-Filter Approach to identify specular reflective and transparent objects in point clouds of a multi-echo laser scanner. Furthermore, it filters point clouds from influences of such objects and extract the object properties for further investigations. Based on an Iterative-Closest-Point-algorithm reflective objects are identified. Object surfaces and points behind surfaces are masked according to their location. Finally, the processed point cloud is forwarded to a mapping module. Furthermore, the object surface corners and the type of the surface is broadcasted. Four experiments demonstrate the usability of the 3D-Reflection-Pre-Filter. The first experiment was made in a empty room containing a mirror, the second experiment was made in a stairway containing a glass door, the third experiment was made in a empty room containing two mirrors, the fourth experiment was made in an office room containing a mirror. This paper demonstrate that for single scans the detection of specular reflective and transparent objects in 3D is possible. It is more reliable in 3D as in 2D. Nevertheless, collect the data of multiple scans and post-filter them as soon as the object was bypassed should pursued. This is why future work concentrates on implementing a post-filter module. Besides, it is the aim to improve the discrimination between specular reflective and transparent objects.
-
Schauer, J., Nüchter, A.: Digitizing automotive production lines without interrupting assembly operations through an automatic voxel-based removal of moving objects. Proceedings of the 13th IEEE International Conference on Control and Automation (ICCA '17). p. 701--706. , Ohrid, Macedonia (2017).
We present an efficient method to partition a point cloud gathered through kinematic laser scanning into static and dynamic points. The presented algorithm utilizes a voxel grid data structure and uses a ray intersection test to mark voxels as dynamic. The algorithm does not require any ego-motion estimations, computationally expensive object recognition or tracking of moving objects over time. It is easy to implement and can be executed on many cores in parallel. We show the viability of this approach by applying our algorithm to a dataset that we gathered by mounting a FARO Focus3D Laser scanner onto a skid which was then sent along a production line for consumer car chassis in a factory of the Volkswagen corporation. Since factory operators are interested in acquiring digital models of their production lines without suspending factory operations, the resulting point cloud will contain many dynamic objects like humans or other car bodies. We show how our algorithm is able to successfully remove these dynamic objects from the resulting point cloud with minimal errors. Our implementation is published under a free license as part of 3DTK.
-
Koch, R., May, S., Nüchter, A.: Effective Distinction Of Transparent And Specular Reflective Objects In Point Clouds Of A Multi-Echo Laser Scanner. Proceedings of the 18th IEEE International Conference on Advanced Robotics (ICAR '17). p. 566--571. , Hong Kong, China (2017).
A favoured sensor for mapping is a 3D laser scanner since it allows a wide scanning range, precise measurements, and is usable indoor and outdoor. Hence, a mapping module delivers detailed and high resolution maps which makes it possible to navigate safely. Difficulties result from transparent and specular reflective objects which cause erroneous and dubious measurements. At such objects, based on the incident angle, measurements result from the object surface, an object behind the transparent surface, or an object mirrored with respect to the reflective surface. This paper describes an enhanced Pre-Filter-Module to distinguish between these cases. Two experiments demonstrate the usability and show that for single scans the identification of mentioned objects in 3D is possible. The first experiment was made in an empty room with a mirror. The second experiment was made in a stairway which contains a glass door. Further, results show that a discrimination between a specular reflective and a transparent object is possible. Especially for transparent objects the detected size is restricted to the incident angle. That is why future work concentrates on implementing a post-filter module. Gained experience shows that collecting the data of multiple scans and postprocess them as soon as the object was bypassed will improve the map.
-
Koch, B., Leblebici, R., Martell, A., Jörissen, S., Schilling, K., Nüchter, A.: Evaluating continuous-time SLAM using a predefined trajectory provided by a robotic arm. Proceedings of the ISPRS Geospatial Week 2017, Laserscanning 2017. p. 17--23. , Wuhan, China (2017).
Recently published approaches to SLAM algorithms process laser sensor measurements and output a map as a point cloud of the environment. Often the actual precision of the map remains unclear, since SLAMalgorithms apply local improvements to the resulting map. Unfortunately, it is not trivial to compare the performance of SLAMalgorithms objectively, especially without an accurate ground truth. This paper presents a novel benchmarking technique that allows to compare a precise map generated with an accurate ground truth trajectory to a map with a manipulated trajectory which was distorted by different forms of noise. The accurate ground truth is acquired by mounting a laser scanner on an industrial robotic arm. The robotic arm is moved on a predefined path while the position and orientation of the end-effector tool are monitored. During this process the 2D profile measurements of the laser scanner are recorded in six degrees of freedom and afterwards used to generate a precise point cloud of the test environment. For benchmarking, an offline continuous-time SLAM algorithm is subsequently applied to remove the inserted distortions. Finally, it is shown that the manipulated point cloud is reversible to its previous state and is slightly improved compared to the original version, since small errors that came into account by imprecise assumptions, sensor noise and calibration errors are removed as well.
-
Pfitzner, C., May, S., Nüchter, A.: Evaluation of Features from RGB-D Data for Human Body Weight Estimation. Proceedings of the 20th World Congress of the International Federation of Automatic Control (IFAC WC '17). , Toulouse, France (2017).
Body weight is a crucial parameter when it comes to drug or radiation dosing. In case of emergency treatment time is short so that physicians estimate the body weight by the visual appearance of a patient. Further, visual body weight estimation might be a feature for person identification. This paper presents the anthropometric feature extraction from RGB-D sensor data, recorded from frontal view. The features are forwarded to an artificial neural network for weight estimation. Experiments with 233 people demonstrate the capability of different features for body weight estimation. To prove robustness against sensor modalities, a structured light sensor is used, as well as a time-of-flight sensor. An additional experiment including temperature features from a thermal camera improves the body weight estimation beyond.
-
Koch, R., May, S., Murmann, P., Nüchter, A.: Identification of Transparent and Specular Reflective Material in Laser Scans to Discriminate Affected Measurements for Faultless Robotic SLAM. Journal of Robotics and Autonomous Systems (JRAS). 87, 296--312 (2017).
Mapping with laser scanners is the state-of-the-art method applied in service, industrial, medical, and rescue robotics. Although a lot of research has been done, maps still suffer from interferences caused by transparent and specular reflective objects. Glass, mirrors, shiny or translucent surfaces cause erroneous measurements depending on the incident angle of the laser beam. In past experiments the Mirror Detector Approach was implemented to determine such measurements with a multi-echo laser scanner. Recognition values are based on their differences in recorded measurements in regard to the distance of the echoes. This paper describes the research to distinguish between reflective and transparent objects. The implemented Mirror Detector was specifically modified for recognition of said objects for which four experiments were conducted; one experiment to show the map of the original Mirror Detector; two experiments to investigate intensity characteristics based on angle, distance, and material; and one experiment to show an applied discrimination with the extended version of the Mirror Detector, the Reflection Classifier Approach. To verify the results, a comparison with existing models was performed. This study showed that shiny metals, like aluminium, etc., provide significant characteristics, while mirrors are to be characterized by a mixed model of glass and shiny metal. Transparent objects turned out to be challenging because their appearance in the sensor data strongly depends on the background. Nevertheless, these experiments show that discrimination of transparent and reflective materials based on the reflected intensity is possible and feasible.
-
Nüchter, A., Bleier, M., Schauer, J., Janotta, P.: Improving Google's Cartographer 3D Mapping by Continuous-Time SLAM. Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 543--549. , Nafplio, Greece (2017).
This paper shows how to use the result of Google's SLAM solution, called Cartographer, to bootstrap our continuous-time SLAM algorithm. The presented approach optimizes the consistency of the global point cloud, and thus improves on Google’s results. We use the algorithms and data from Google as input for our continuous-time SLAM software. We also successfully applied our software to a similar backpack system which delivers consistent 3D point clouds even in absence of an IMU.
-
Bleier, M., Nüchter, A.: Low-cost 3D Laser Scanning in Air or Water Using Self-calibrating Structured Light. Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 105--112. , Nafplio, Greece (2017).
In-situ calibration of structured light scanners in underwater environments is time-consuming and complicated. This paper presents a self-calibrating line laser scanning system, which enables the creation of dense 3D models with a single fixed camera and a freely moving hand-held cross line laser projector. The proposed approach exploits geometric constraints, such as coplanarities, to recover the depth information and is applicable without any prior knowledge of the position and orientation of the laser projector. By employing an off-the-shelf underwater camera and a waterproof housing with high power line lasers an affordable 3D scanning solution can be built. In experiments the performance of the proposed technique is studied and compared with 3D reconstruction using explicit calibration. We demonstrate that the scanning system can be applied to above-the-water as well as underwater scenes.
-
Chen, L., Fan, L., Xie, G., Huang, K., Nüchter, A.: Moving-Object Detection From Consecutive Stereo Pairs Using Slanted Plane Smoothing. IEEE Transactions on Intelligent Transportation Systems. 18, 1--10 (2017).
Detecting moving objects is of great importance for autonomous unmanned vehicle systems, and a challenging task especially in complex dynamic environments. This paper proposes a novel approach for the detection of moving objects and the estimation of their motion states using consecutive stereo image pairs on mobile platforms. First, we use a variant of the semi-global matching algorithm to compute initial disparity maps. Second, assisted by the initial disparities, boundaries in the image segmentation produced by simple linear iterative clustering are classified into coplanar, hinge, and occlusion. Moving points are obtained during ego-motion estimation by a modified random sample consensus) algorithm without resorting to time-consuming dense optical flow. Finally, the moving objects are extracted by merging superpixels according to the boundary types and their movements. The proposed method is accelerated on the GPU at 20 frames per second. The data which we use for testing and benchmarking is released, thus completing similar data sets. It includes 812 image pairs and 924 moving objects with ground truth for better algorithms evaluation. Experimental results demonstrate that the proposed method achieves competitive results in terms of moving-object detection and their motion state estimation in challenging urban scenarios.
-
Koch, R., Böttcher, L., Jahrsdörfer, M., Maier, J., Trommer, M., May, S., Nüchter, A.: Out of lab calibration of a rotating 2D scanner for 3D mapping. Proceedings of the SPIE optical metrology, Videometrics, Range Imaging, and Applications. pp. 10332 - 10332 - 8. , Munich, Germany (2017).
Mapping is an essential task in mobile robotics. To fulfil advanced navigation and manipulation tasks a 3D representation of the environment is required. Applying stereo cameras or Time-of-flight cameras (TOF cameras) are one way to archive this requirement. Unfortunately, they suffer from drawbacks which makes it difficult to map properly. Therefore, costly 3D laser scanners are applied. An inexpensive way to build a 3D representation is to use a 2D laser scanner and rotate the scan plane around an additional axis. A 3D point cloud acquired with such a custom device consists of multiple 2D line scans. Therefore the scanner pose of each line scan need to be determined as well as parameters resulting from a calibration to generate a 3D point cloud. Using external sensor systems are a common method to determine these calibration parameters. This is costly and difficult when the robot needs to be calibrated outside the lab. Thus, this work presents a calibration method applied on a rotating 2D laser scanner. It uses a hardware setup to identify the required parameters for calibration. This hardware setup is light, small, and easy to transport. Hence, an out of lab calibration is possible. Additional a theoretical model was created to test the algorithm and analyse impact of the scanner accuracy. The hardware components of the 3D scanner system are an HOKUYO UTM-30LX-EW 2D laser scanner, a Dynamixel servo-motor, and a control unit. The calibration system consists of an hemisphere. In the inner of the hemisphere a circular plate is mounted. The algorithm needs to be provided with a dataset of a single rotation from the laser scanner. To achieve a proper calibration result the scanner needs to be located in the middle of the hemisphere. By means of geometric formulas the algorithms determine the individual deviations of the placed laser scanner. In order to minimize errors, the algorithm solves the formulas in an iterative process. First, the calibration algorithm was tested with an ideal hemisphere model created in Matlab. Second, laser scanner was mounted differently, the scanner position and the rotation axis was modified. In doing so, every deviation, was compared with the algorithm results. Several measurement settings were tested repeatedly with the 3D scanner system and the calibration system. The results show that the length accuracy of the laser scanner is most critical. It influences the required size of the hemisphere and the calibration accuracy.
-
Bleier, M., Dias, A., Ferreira, A., Pidgeon, J., Almeida, J., Silva, E., Schilling, K., Nüchter, A.: Real-time 3D Mine Modelling in the !VAMOS! Project. In: Buxton, M. and Benndorf, J. (eds.) Real Time Mining. p. 91--102. Technische Universität Bergakademie Freiberg, Institut für Markscheidewesen und Geodäsie, Amsterdam, The Netherlands (2017).
The project Viable Alternative Mine Operating System (\textexclamdown VAMOS!) develops a new safe, clean and low visibility mining technique for excavating raw materials from submerged inland mines. During operations, the perception data of the mining vehicle can only be communicated to the operator via a computer interface. In order to assist remote control and facilitate assessing risks a detailed view of the mining process below the water surface is necessary. This paper presents approaches to real-time 3D reconstruction of the mining environment for immersive data visualisation in a virtual reality environment to provide advanced spatial awareness. From the raw survey data a more consistent 3D model is created using post-processing techniques based on a continuous-time simultaneous localization and mapping (SLAM) solution. Signed distance function (SDF) based mapping is employed to fuse the measurements from multiple views into a single representation and reduce sensor noise. Results of the proposed techniques are demonstrated on a dataset captured in an sub- merged inland mine.
-
Bleier, M., Dias, A., Ferreira, A., Pidgeon, J., Almeida, J.M., Silva, E., Schilling, K., Nüchter, A.: Signed Distance Function Based Surface Reconstruction of a Submerged Inland Mine Using Continuous-Time SLAM. Proceedings of the 20th World Congress of the International Federation of Automatic Control (IFAC WC '17). , Toulouse, France (2017).
The planning of mining operations in water filled open-pit mines requires detailed bathymetry to create a mine plan and assess the involved risks. This paper presents post- processing techniques for creating an improved 3D model from a survey carried out using an autonomous surface vehicle with a multibeam sonar and a GPS/INS navigation system. Inconsistencies of the created point cloud as a result of calibration errors or GPS signal loss are corrected using a continuous-time simultaneous localization and mapping (SLAM) solution. Signed distance function based mapping is employed to fuse the measurements from multiple runs into a consistent representation and reduce sensor noise. From the signed distance function model we reconstruct a 3D surface mesh. We use this terrain model to establish a virtual reality scene for immersive data visualization of the mining operations for testing and planing during development. Results of the proposed approach are demonstrated on a dataset captured in an abandoned submerged inland mine.
-
Struckmeier, O., Borrmann, D., Nüchter, A.: Teach-In für die 3D-Scan Akquise mit einem Roboter. Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2017, Jade Hochschule. p. 108--119 (2017).
Die 3D-Erfassung einer kompletten Umgebung mittels 3D-Laserscanner stellt abhängig von der zu scannenden Umgebung und dem Scanner einen hohen Zeitaufwand für einen menschlichen Operator dar. Der Scanner muss an die einzelnen Scanpositionen bewegt und dort verortet werden. Mit steigender Qualität der Messung nimmt zudem die Dauer der einzelnen Scanvorgänge zu. In diesem Beitrag wird ein Teach-In Ansatz vorgestellt und evaluiert, der den manuelle n Vorgang verkürzt. Dabei führt ein Roboter die vom Vermesser geplanten und eingespeicherten zeitintensiven Schritte automatisch durch und entlastet somit den Bediener.
-
Nüchter, A., Elseberg, J., Janotta, P.: Towards Mobile Mapping of Underground Mines. In: Buxton, M. and Benndorf, J. (eds.) Real Time Mining. p. 27--38. Technische Universität Bergakademie Freiberg, Institut für Markscheidewesen und Geodäsie, Amsterdam, The Netherlands (2017).
Mobile laser scanning systems automate the acquisition of 3D point clouds of environments. The mapping systems are commonly mounted on cars or ships. This paper presents a flexible mapping solution mounted on an underground vehicle, that is able to map underground mines in 3D in walking speeds. A clever choice of hard- and software enables the system to generate 3D maps without using GPS (global positioning system) information and without relying on highly expensive IMU (inertial measurement unit) systems.
-
Lauterbach, H.A., Borrmann, D., Nüchter, A.: Towards Radiometrical Alignment of 3D Point Clouds. Proceedings of the 7th ISPRS International Workshop 3D-ARCH 2017: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 419--424. , Nafplio, Greece (2017).
3D laser scanners are typically not able to collect color information. Therefore coloring is often done by projecting photos of an additional camera to the 3D scans. The capturing process is time consuming and therefore prone to changes in the environment. The appearance of the colored point cloud is mainly effected by changes of lighting conditions and corresponding camera settings. In case of panorama images these exposure variations are typically corrected by radiometrical aligning the input images to each other. In this paper we adopt existing methods for panorama optimization in order to correct the coloring of point clouds. Therefore corresponding pixels from overlapping images are selected by using geometrically closest points of the registered 3D scans and their neighboring pixels in the images. The dynamic range of images in raw format allows for correction of large exposure differences. Two experiments demonstrate the abilities of the approach.
-
Bleier, M., Nüchter, A.: Towards robust self-calibration for handheld 3D line laser scanning. Proceedings of LowCost 3D 2017. p. 31--36. , Hamburg, Germany (2017).
This paper studies self-calibration of a structured light system, which reconstructs 3D information using video from a static consumer camera and a handheld cross line laser projector. Intersections between the individual laser curves and geometric constraints on the relative position of the laser planes are exploited to achieve dense 3D reconstruction. This is possible without any prior knowledge of the movement of the projector. However, inaccurrately extracted laser lines introduce noise in the detected intersection positions and therefore distort the reconstruction result. Furthermore, when scanning objects with specular reflections, such as glossy painted or metalic surfaces, the reflections are often extracted from the camera image as erroneous laser curves. In this paper we investiagte how robust estimates of the parameters of the laser planes can be obtained despite of noisy detections.
-
Borrmann, D., Leutert, F., Maurovic, I., Seder, M., Nüchter, A.: Automatische Grundrisserstellung mittels Laserscandaten. Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2016, Jade Hochschule. p. 108--119 (2016).
In den letzten Jahren haben sich Laserscanner zum Stand der Technik bei der Erstellung von Gebäudemodellen entwickelt. Im Gegensatz zur vergleichsweise kurzen Aufnahmezeit bei der eigentlichen Erfassung der Umgebung mit dem Scanner nimmt die Nachbearbeitung der Daten einen deutlich höheren Zeitanteil ein. Eine manuelle Datenanalyse ist zeitaufwändig und fehleranfällig. Dies erhöht den Bedarf an automatischen Verfahren zur quantitativen Erfassung und Charakterisierung von Umgebungen. In diesem Beitrag präsentieren wir eine automatische Grundrisserstellung basierend auf dem 3D Toolkit (3DTK - http://www.threedtk.de). Nach der autonomen Akquise mit einem mobilen Roboter werden die Daten mittels des iterativen Verfahrens der nächsten Punkte (engl. Iterative Closest Point (ICP)) registriert. Anschließend erfolgt eine automatische Vektorisierung eines 2D-Schnitts der Umgebung basierend auf Verfahren aus der Bildverarbeitung. Der so erzeugte Grundriss dient als Grundlage für eine semantische Karte.
-
Nüchter, A.: Effiziente Speicherung großer Punktwolken -- Datenstrukturen für Algorithmen für mobile und terrestrische Laserscansysteme. Terrestrisches Laserscanning (TLS 2016) Beiträge zum 154. DVW-Seminar am 28. und 29. November in Fulda. p. 105--120. , Fulda, Germany (2016).
-
Lehtola, V.V., Virtanen, J.-P., Rönnholm, P., Nüchter, A.: LOCALIZATION CORRECTIONS FOR MOBILE LASER SCANNER USING LOCAL SUPPORT-BASED OUTLIER FILTERING. Proceedings of the ISPRS Congress 2016. p. 81--88. , Prague, Czech Republic (2016).
Following the pioneering work introduced in [Lehtola et al., ISPRS J. Photogramm. Remote Sens. 99, 2015, pp. 25–29], we extend the state-of-the-art intrinsic localization solution for a single two-dimensional (2D) laser scanner from one into (quasi) three dimensions (3D). By intrinsic localization, we mean that no external sensors are used to localize the scanner, such as inertial measurement devices (IMU) or global navigation satellite systems (GNSS). Specifically, the proposed method builds on a novel concept of local support-based filtering of outliers, which enables the use of six degrees-of-freedom (DoF) simultaneous localization and mapping (SLAM) for the purpose of enacting appropriate trajectory corrections into the previous one-dimensional solution. Moreover, the local support-based filtering concept is platform independent, and is therefore likely to be widely generalizable. The here presented overall method is yet limited into quasi-3D by its inability to recover trajectories with steep curvature, but in the future, it may be further extended into full 3D.
-
Lehtola, V.V., Virtanen, J.-P., Vaaja, M.T., Hyyppä, H., Nüchter, A.: Localization of a Mobile Laser Scanner via Dimensional Reduction. ISPRS Journal of Photogrammetry and Remote Sensing (JPRS). 121, 48--59 (2016).
We extend the concept of intrinsic localization from a theoretical one-dimensional (1D) solution onto a 2D manifold that is embedded in a 3D space, and then recover the full six degrees of freedom for a mobile laser scanner with a simultaneous localization and mapping algorithm (SLAM). By intrinsic localization, we mean that no reference coordinate system, such as global navigation satellite system (GNSS), nor inertial measurement unit (IMU) are used. Experiments are conducted with a 2D laser scanner mounted on a rolling prototype platform, VILMA. The concept offers potential in being extendable to other wheeled platforms.
-
Koch, P., May, S., Schmidpeter, M., Kühn, M., Pfitzner, C., Merkl, C., Koch, R., Fees, M., Martin, J., Nüchter, A.: Multi-Robot Localization and Mapping based on Signed Distance Functions. Journal of Intelligent and Robotic Systems. 83, 409--428 (2016).
This publication describes a 2D Simultaneous Localization and Mapping approach applicable to multiple mobile robots. The presented strategy uses data of 2D LIDAR sensors to build a dynamic representation based on Signed Distance Functions. Novelties of the approach are a joint map built in parallel instead of occasional merging of smaller maps and the limited drift localization which requires no loop closure detection. A multi-threaded software architecture performs registration and data integration in parallel allowing for drift-reduced pose estimation of multiple robots. Experiments are provided demonstrating the application with single and multiple robot mapping using simulated data, public accessible recorded data, two actual robots operating in a comparably large area as well as a deployment of these units at the Robocup rescue league.
-
Pfitzner, C., May, S., Nüchter, A.: Neural Network-based Visual Body Weight Estimation for Drug Dosage Finding. Proceedings of the SPIE 9784, Medical Imaging 2016: Image Processing. , San Diego, CA, USA (2016).
Body weight adapted drug dosages are important for emergency treatments. This paper describes an improved body weight estimation approach for emergency patients in a trauma room, based on images from a RGBD sensor and a thermal camera. The improvements are archived by several extensions: The sensor fusion of RGBD and thermal camera eases filtering and segmentation of the patient's body from the background. Robustness and accuracy is gained by an artificial neural network (ANN), which considers features from the sensors as input to calculate the patient's body weight, e.g. the patient's volume, surface and shape parameters. The ANN is trained offline with 30 percent of the patients data. Preliminary experiments with 69 real patients show an accuracy close to 90 percent for a threshold of ten percent relative error in real body estimation. Results are compared to the patient's self estimation, a physician's guess and an anthropometric method: If the patient is knowledgeable it is the best possibility for body weight adapted drug dosages with 97 percent accuracy. The treating physicians and the anthropometric estimation achieve an accuracy of approximately 70 percent. The here presented approach gets an accuracy of nearly 90 percent and would be the best solution if a patient can not provide his own body weight and can not be weighted on a scale. These preliminary results demonstrate a sufficient approach for an upcoming clinical trial with 1,000 patients for body weight estimation.
-
Schauer, J., Bedkowski, J., Majek, K., Nüchter, A.: Performance comparison between state-of-the-art point-cloud based collision detection approaches on the CPU and GPU. Proceedings of the 4th IFAC Symposium on Telematics Applications (TA '13). p. 54--59. , Porto Alegre, Brazil (2016).
We present two fundamentally different approaches to detect collisions between two point clouds and compare their performance on multiple datasets. A collision between points happens if they are closer to each other than a given threshold radius. One approach utilizes the main CPU with a k-d tree datastructure to efficiently carry out fixed range searches around points in 3D while the other mainly executes on a GPU using a regular grid decomposition technique implemented in the CUDA framework. We will show how massively parallel 3D range searches on a grid based datastructure on a GPU performs similarly well as a tree based approach on the CPU with orders of magnitude less parallelization. We also show how each method scales with varying input sizes and how they perform differently well depending on the spatial structure of the input data.
-
Konolige, K., Nüchter, A.: Range Sensors. In: Siciliano, B. and Khatib, O. (eds.) Handbook of Robotics. p. 783--810. Springer (2016).
-
Leutert, F., Borrmann, D., Schilling, K., Nüchter, A.: Spatial Projection of Thermal Data for Visual Inspection. Proceedings of the 14th IEEE International Conference on Control, Automation, Robotics and Vision (ICARCV '16). p. 1--6. , Phuket, Thailand (2016).
Since the advent of thermal imaging, devices with a high optical resolution that use detector arrays to capture the emitted radiance in the thermal infrared range of an entire scene simultaneously have developed as a standard in monitoring energy related. They have had a huge impact on the building industry and in manifacturing, where they are commonly used to monitor proecces that require stable temperature conditions. As beneficial as contactless measurements are, the subsequent localization of points of interest in the environment is often difficult. To overcome this problem we propose a portable system that combines thermal imaging with Augmented Reality (AR). The idea of the approach is to project the gathered temperature information back into the scene to facilitate visual inspection.
-
Houshiar, H., Nüchter, A.: 3D Point Cloud Compression using Conventional Image Compression for Efficient Data Transmission. Proceedings of the XXV International Symposium on Information, Communication and Automation Technologies (ICAT '15). p. 1--8. IEEE Xplore, Sarajevo, Bosnia (2015).
Modern 3D laser scanners make it easy to collect large 3D point clouds. In this paper we present the use of conventional image based compression methods for 3D point clouds. We map the point cloud onto panorama images to encode the range, reflectance and color value for each point. An encoding method is presented to map the floating point measured ranges on to a three channel image. The image compression methods are used to compress the generated panorama images. We present the results of several lossless compression methods and the lossy JPEG on point cloud compression. Lossless compression methods are designed to retain the original data. On the other hand lossy compression methods sacrifice the details for higher compression ratio. This produces artefacts in the recovered point cloud data. We study the effects of these artefacts on encoded range data. A filtration process is presented for determination of range outliers from uncompressed point clouds.
-
Nüchter, A., Borrmann, D., Elseberg, J., Redondo, D.: A Backpack-Mounted 3D Mobile Scanning System. Allgemeine Vermessungs-Nachrichten (AVN), Special Issue MoLAS 2014. 122, 301--307 (2015).
Mobile laser scanning systems automate the acquisition of 3D point clouds of environments. The mapping systems are commonly mounted on cars or ships. This paper presents a mapping solution mounted on a backpack. A clever choice of hard- and software enables the system to generate 3D maps without using GPS (global positioning system) information and without relying on expensive IMU (inertial measurement unit) systems. Therefore, it enables flexible indoor mapping.
-
Nüchter, A., Borrmann, D., Koch, P., Kühn, M., May, S.: A Man-Portable, IMU-free Mobile Mapping. Proceedings of the ISPRS Geospatial Week 2015, Laserscanning 2015. p. 17--23. , La Grande Motte, France (2015).
Mobile mapping systems are commonly mounted on cars, ships and robots. The data is directly geo-referenced using GPS data and expensive IMU (inertial measurement systems). Driven by the need for flexible, indoor mapping systems we present an inexpensive mobile mapping solution that can be mounted on a backpack. It combines a horizontally mounted 2D profiler with a constantly spinning 3D laser scanner. The initial system featuring a low-cost MEMS IMU was revealed and demonstrated at MoLaS: Technology Workshop Mobile Laser Scanning at Fraunhofer IPM in Freiburg in November 2014. In this paper, we present an IMU-free solution.
-
Houshiar, H., Elseberg, J., Borrmann, D., Nüchter, A.: A Study of Projections for Key Point Based Registration of Panoramic Terrestrial 3D Laser Scans. Journal of Geo-spatial Information Science. 18, 11--31 (2015).
This paper surveys state-of-the-art image features and descriptors for the task of 3D scan registration based on panoramic reflectance images. As modern terrestrial laser scanners digitize their environment in a spherical way, the sphere has to be projected to a two-dimensional image. To this end, we evaluate the equirectangular, the cylindrical, the Mercator, the rectilinear, the Pannini, the stereographic, and the z-axis projection. We show that the Mercator and the Pannini projection outperform the other projection methods.
-
Houshiar, H., Borrmann, D., Elseberg, J., Nüchter, A., Winkler, S., Näth, F.: CASTLE3D -- A Computer Aided System for Labelling Archaeological Excavations in 3D. Proceedings of the XXV International CIPA Symposium. p. 111--118. , Taipei, Taiwan (2015).
Documentation of archaeological excavation sites with conventional methods and tools such as hand drawings, measuring tape and archaeological notes is time consuming. This process is prone to human errors and the quality of the documentation depends on the qualification of the archaeologist on site. Use of modern technology and methods in 3D surveying and 3D robotics facilitate and improve this process. Computer-aided systems and databases improve the documentation quality and increase the speed of data acquisition. 3D laser scanning is the state of the art in modelling archaeological excavation sites, historical sites and even entire cities or landscapes. Modern laser scanners are capable of data acquisition of up to 1 million points per second. This provides a very detailed 3D point cloud of the environment. 3D point clouds and 3D models of an excavation site provide a better representation of the environment for the archaeologist and for documentation. The point cloud can be used both for further studies on the excavation and for the presentation of results. This paper introduces a Computer aided system for labelling archaeological excavations in 3D (CASTLE3D). Consisting of a set of tools for recording and georeferencing the 3D data from an excavation site, CASTLE3D is a novel documentation approach in industrial archaeology. It provides a 2D and 3D visualisation of the data and an easy-to-use interface that enables the archaeologist to select regions of interest and to interact with the data in both representations. The 2D visualisation and a 3D orthogonal view of the data provide cuts of the environment that resemble the traditional hand drawings. The 3D perspective view gives a realistic view of the environment. CASTLE3D is designed as an easy-to-use on-site semantic mapping tool for archaeologists. Each project contains a predefined set of semantic information that can be used to label findings in the data. Multiple regions of interest can be joined under one label. Further information such as color, orientation and archaeological notes are added to the label to improve the documentation. The available 3D information allows for easy measurements in the data. The full 3D information of a region of interest can be segmented from the entire data. By joining this data from different georeferenced views the full 3D shape of findings is stored. All the generated documentation in CASTLE3D is exported to an XML format and serves as input for other systems and databases. Apart from presenting the functionalities of CASTLE3D we evaluate its documentation process in a sample project. For this purpose we export the data to the Adiuvabit database (http://adiuvabit.de) where more information is added for further analysis. The documentation process is compared to traditional documentation methods and it is shown how the automated system helps in accelerating the documentation process and decreases errors to a minimum.
-
Schauer, J., Nüchter, A.: Collision detection between point clouds using an efficient \($k$\)-d tree implementation. Journal Advanced Engineering Informatics (JAdvEI). 29, 440--458 (2015).
Context: An important task in civil engineering is the detection of collisions of a 3D model with an environment representation. Existing methods using the structure gauge provide an insufficient measure because the model either rotates or because the trajectory makes tight turns through narrow passages. This is the case in either automotive assembly lines or in narrow train tunnels. Objective: Given two point clouds, one of the environment and one of a model and a trajectory with six degrees of freedom along which the model moves through the environment, find all colliding points of the environment with the model within a certain clearance radius. Method: This paper presents two collision detection (CD) methods called kd-CD and kd-CD-simple and two penetration depth (PD) calculation methods called kd-PD and kd-PD-fast. All four methods are based on searches in a k-d tree representation of the environment. The creation of the k-d tree, its search methods and other features will be explained in the scope of their use to detect collisions and calculate depths of penetration. Results: The algorithms are benchmarked by moving the point cloud of a train wagon with 2.5 million points along the point cloud of a 1144m long train track through a narrow tunnel with overall 18.92 million points. Points where the wagon collides with the tunnel wall are visually highlighted with their penetration depth. With a safety margin of 5cm kd-PD-simple finds all colliding points on its trajectory which is sampled into 19,392 positions in 77s on a standard desktop machine of 1.6GHz. Conclusion: The presented methods for collision detection and penetration depth calculation are shown to solve problems for which the structure gauge is an insufficient measure. The underlying k-d tree is shown to be an effective data structure for the required look-up operations.
-
Borrmann, D., Houshiar, H., Elseberg, J., Nüchter, A., Näth, F., Winkler, S.: Das Castle3D Framework zur fortlaufenden semantischen 3D-Kartierung von archäologischen Ausgrabungsstätten. Allgemeine Vermessungs-Nachrichten (AVN). 122, 233--246 (2015).
Das 3D Laserscanning ist Stand der Technik bei der Modellierung archäologischer Ausgrabungsstätten, historischer Anlagen und sogar ganzer Städte oder Landschaften. Die Dokumentation der Befunde auf einer Ausgrabungsstätte ist eine wesentliche archäologische Aufgabe. Ein automatisiertes System würde diesen Prozess beschleunigen und die Anzahl der Fehler auf ein Minimum reduzieren. Dieser Beitrag stellt einen neuen Ansatz in der Dokumentation industrieller Archäologie vor durch die Entwicklung einer standardisierten Herangehensweise an die computerunterstützte Dokumentation einer archäologischen Ausgrabungsstätten. Ausserdem wird eine Reihe von Tools zur Erfassung und Registrierung von 3D-Daten auf Ausgrabungsstätten vorgestellt, die den Hauptbestandteil der Arbeitskette abdecken. Wir stellen ein effizientes Werkzeug zur Verfügung für die Visualisierung der erworbenen 3D-Punktwolken im 3D- und 2D-Modus. Der Hauptzweck dieser Software ist Archäologen ein einfach zu bedienendes Tool für die semantische Kartierung vor Ort zu bieten. Es enthält Funktionen für die Auswahl und Kennzeichnung von Funden. Jedes Label kann mit weiteren Informationen versehen werden. Diese Daten werden im XML-Format exportiert und dienen als Eingabe für andere Systeme und Datenbanken.
-
Koch, R., May, S., Koch, P., Kühn, M., Nüchter, A.: Detection of Specular Reflections in Range Measurements for Faultless Robotic SLAM. Proceedings of ROBOT'2015: Second Iberian Robotics Conference, Advances in Robotics, Volume 1. p. 133--145. Springer, Lisbon, Portugal (2015).
Laser scanners are state-of-the-art devices used for mapping in service, industry, medical and rescue robotics. Although a lot of work has been done in laser-based SLAM, maps still suffer from interferences caused by objects like glass, mirrors and shiny or translucent surfaces. Depending on the surface's reflectivity, a laser beam is deflected such that returned measurements provide wrong distance data. At certain positions phantom-like objects appear. This paper describes a specular reflectance detection approach applicable to the emerging technology of multi-echo laser scanners in order to identify and filter reflective objects. Two filter stages are implemented. The first filter reduces errors in current scans on the fly. A second filter evaluates a set of laser scans, triggered as soon as a reflective surface has been passed. This makes the reflective surface detection more robust and is used to refine the registered map. Experiments demonstrate the detection and elimination of reflection errors. They show improved localization and mapping in environments containing mirrors and large glass fronts is improved.
-
Al khawaldah, M., Nüchter, A.: Enhanced frontier-based exploration for indoor environment with multiple robots. Advanced Robotics. 28, 657--669 (2015).
n this paper, the exploration and map-building of unknown environment by a team of mobile robots is intensively investigated. A new exploration technique is proposed to increase the exploration efficiency. In particular, the new technique has two main objectives: firstly, it aims at reducing the exploration time and the traveled distance by reducing the overlap which takes place when a certain area in the environment is explored by more than one robot. To achieve this, a new procedure to assign the next target location for each individual robot is proposed. And secondly, it aims at reducing computations complexity required by target selection and path planning tasks. More importantly, the proposed technique obviates the need for environment segmentation complex procedures which is adopted in some previous important research works. The new technique is intensively tested with different environments. The results showed the effectiveness of the proposed technique.
-
Lauterbach, H.A., Borrmann, D., Hess, R., Eck, D., Schilling, K., Nüchter, A.: Evaluation of a Backpack-Mounted 3D Mobile Scanning System. Remote Sensing. 7, 13753--13781 (2015).
Recently, several backpack-mounted systems, also known as personal laser scanning systems, have been developed. They consist of laser scanners or cameras that are carried by a human operator to acquire measurements of the environment while walking. These systems were first designed to overcome the challenges of mapping indoor environments with doors and stairs. While the human operator inherently has the ability to open doors and to climb stairs, the flexible movements introduce irregularities of the trajectory to the system. To compete with other mapping systems, the accuracy of these systems has to be evaluated. In this paper, we present an extensive evaluation of our backpack mobile mapping system in indoor environments. It is shown that the system can deal with the normal human walking motion, but has problems with irregular jittering. Moreover, we demonstrate the applicability of the backpack in a suitable urban scenario.
-
Borrmann, D., Hess, R., Eck, D., Houshiar, H., Nüchter, A., Schilling, K.: Evaluation of Methods for Robotic Mapping of Cultural Heritage Sites. Proceedings of the 2th IFAC conference on Embedded Systems, Computer Intelligence and Telematics (CESCIT '15). p. 105--110. , Maribor, Slovenia (2015).
In archaeological studies the use of new technologies has moved into focus in the past years creating new challenges such as the processing of the massive amounts of data. In this paper we present steps and processes for smart 3D modelling of environments by use of the mobile robot Irma3D. A robot that is equipped with multiple sensors, most importantly a photo camera and a laser scanner, enables the automation of most of the processes, including data acquisition and registration. The robot was tested in the Würzburg Residence. Methods for automatic 3D color reconstructions of cultural heritage sites are evaluated in this paper.
-
Bedkowski, J., Majek, K., Majek, P., Musialik, P., Pelka, M., Nüchter, A.: Intelligent Mobile System for Improving Spatial Design Support and Security Inside Buildings. Mobile Networks and Applications. 20, 1--14 (2015).
This paper concerns an intelligent mobile application for spatial design support and security domain. Mobility has two aspects in our research: The first one is the usage of mobile robots for 3D mapping of urban areas and for performing some specific tasks. The second mobility aspect is related with a novel Software as a Ser- vice system that allows access to robotic functionalities and data over the Ethernet, thus we demonstrate the use of the novel NVIDIA GRID technology allowing to virtualize the graphic processing unit. We introduce Complex Shape Histogram, a core component of our artificial intelligence engine, used for classifying 3D point clouds with a Support Vector Machine. We use Complex Shape Histograms also for loop closing detection in the simultaneous localization and mapping algorithm. Our intelligent mobile system is built on top of the Qualitative Spatio-Temporal Representa- tion and Reasoning framework. This framework defines an ontology and a semantic model, which are used for building the intelligent mobile user interfaces. We show experiments demonstrating advantages of our approach. In addition, we test our prototypes in the field after the end-user case studies demonstrating a relevant contribution for future intelligent mobile systems that merge mobile robots with novel data centers.
-
Demisse, G., Borrmann, D., Nüchter, A.: Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency. Journal of Intelligent and Robotic Systems. 77, 55--72 (2015).
In recent years 3D models of buildings are used in maintenance and inspection, preservation, and other building related applications. However, the usage of these models is limited because most models are pure representations with no or little associated semantics. In this paper we present a pipeline of techniques used for interior interpretation, object detection, and adding energy related semantics to windows of a 3D thermal model. A sequence of algorithms is presented for building the fundamental semantics of a 3D model. Among other things, these algorithms enable the system to differentiate between objects in a room and objects that are part of the room, e.g. floor, windows. Subsequently, the thermal information is used to construct a stochastic mathematical model– namely Markov Random Field– of the temperature distribution of the detected windows. As a result, the MAP(Maximum a posteriori) framework is used to further label the windows as either open, closed or damaged based upon their temperature distribution. The experimental results showed the robustness of the techniques. Furthermore, a strategy to optimize the free parameters is described, in cases where there is a sample training dataset.
-
Pfitzner, C., May, S., Merkl, C., Breuer, L., Köhrmann, M., Braun, J., Dirauf, F., Nüchter, A.: Libra3D: Body Weight Estimation for Emergency Patients in Clinical Environment with a 3D Structured Light Sensor. Proceedings of the IEEE International Conference Robotics and Automation (ICRA '15). , Seattle, WA, USA (2015).
his paper describes the application of a weight estimation method for emergency patients in clinical environments. The approach applies established algorithms for point cloud processing and filtering to data from a low-cost, structured light sensor. A patient's volume is estimated on the basis of their visible front surface. The approach is currently being tested in the workflow of the emergency room at the Universitätsklinikum Erlangen, Germany. Preliminary results show the accuracy of the approach in relation to other conservative means of weight measurements, for example, by physicians and anthropometric measurements.
-
Käshammer, P., Nüchter, A.: Mirror Identification and Correction of 3D Point Clouds. Proceedings of the 6th ISPRS International Workshop 3D-ARCH 2015: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 109--114. , Avila, Spain (2015).
In terrestrial laser scanning (TLS), the surface geometry of objects is scanned by laser beams and recorded digitally. This produces a discrete set of scan points, commonly referred to as a point cloud. The coordinates of the scan points are determined by measuring the angles and the time-of-flight relative to the origin (scanner position). However, if it comes to mirror surfaces laser beams are fully reflected, due to the high reflectivity. Mirrors do not appear in the point cloud at all. Instead, for every reflected beam, a incorrect scan point is created behind the actual mirror plane. Consequently, problems arise in multiple derived application fields such as 3D virtual reconstruction of complex architectures. The paper presents a new approach to automatically detect framed rectangular mirrors with known dimensions and to correct the 3D point cloud, using the calculated mirror plane.
-
Koch, P., May, S., Schmidpeter, M., Kühn, M., Pfitzner, C., Merkl, C., Koch, R., Fees, M., Martin, J., Nüchter, A.: Multi-Robot Localization and Mapping based on Signed Distance Functions. Proceedings of the IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC '15). p. 77--82. , Vila Real, Portugal (2015).
This publication describes a 2D Simultaneous Localization and Mapping approach applicable to multiple mobile robots. The presented strategy uses data of 2D LIDAR sensors to build a dynamic representation based on Signed Distance Functions. A multi-threaded software architecture performs registration and data integration in parallel allowing for drift-reduced pose estimation of multiple robots. Experiments are provided demonstrating the application with single and multiple robot mapping using simulated data, public accessible recorded data as well as two actual robots operating in a comparably large area.
-
Borrmann, D., Hess, R., Eck, D., Nüchter, A., Schilling, K.: Robotic Mapping of Cultural Heritage Sites. Proceedings of the 6th ISPRS International Workshop 3D-ARCH 2015: "3D Virtual Reconstruction and Visualization of Complex Architectures". p. 9--16. , Avila, Spain (2015).
In archaeological studies the use of new technologies has moved into focus in the past years creating new challenges such as the processing of the massive amounts of data. In this paper we present steps and processes for smart 3D modelling of environments by use of the mobile robot Irma3D. A robot that is equipped with multiple sensors, most importantly a photo camera and a laser scanner, enables the automation of most of the processes, including data acquisition and registration. The robot was tested in two scenarios, Ostia Antica and the Würzburg Residence. The paper describes the steps for creating 3D color reconstructions of these renown cultural heritage sites.
-
Gailis, J., Nüchter, A.: Towards Globally Consistent Scan Matching With Ground Truth Integration. Proceedings of the ISPRS International Conference on Photogrammetric Image Analysis (PIA '15). p. 59--64. , Munich, Germany (2015).
The scan matching based simultaneous localization and mapping method with six dimensional poses is capable of creating a three dimensional point cloud map of the environment, as well as estimating the six dimensional path that the vehicle has travelled. The essence of it is the registering and matching of sequentially acquired 3D laser scans, while moving along a path, in a common coordinate frame in order to provide 6D pose estimations at the respective positions, as well as create a three dimensional map of the environment. An approach that could drastically improve the reliability of acquired data is to integrate available ground truth information. This paper is about implementing such functionality as a contribution to 6D SLAM (simultaneous localization and mapping with 6 DoF) in the 3DTK – The 3D Toolkit software (Nüchter and Lingemann, 2011), as well as test the functionality of the implementation using real world datasets.
-
May, S., Koch, P., Koch, R., Merkl, C., Pfitzer, C., A.Nüchter,: A Gereralized 2D and 3D Multi-Sensor Data Integration Approach based on Signed Distance Functions for Multi-Modal Robotic Mapping. Proceedings of th 19th International Workshop on Vision, Modeling and Visualization (VMV '14). , Darmstadt, Germany (2014).
This paper describes a data integration approach for arbitrary 2D/3D depth sensing units exploiting assets of the signed distance function. The underlying framework generalizes the KinectFusion approach with an objectoriented model respecting different sensor modalities. For instance, measurements of 2D/3D laser range finders and RGB-D cameras can be integrated into the same representation. Exemplary, an environment is reconstructed with a 3D laser range finder, while adding fine details from objects of interest by closer inspection with an RGB-D sensor. A typical application of this approach is the exploration in rescue environments, where large-scale mapping is performed on the basis of long-range laser range finders while hollows are inspected with lightweight sensors attached to a manipulator arm.
-
Borrmann, D., Nüchter, A., DJakulovi'c, M., Maurovi'c, I., Petrovi'c, I., Osmankovi'c, D., Velagi'c, J.: A mobile robot based system for fully automated thermal 3D mapping. Journal Advanced Engineering Informatics (JAdvEI). 28, 425--440 (2014).
It is hard to imagine living in a building without electricity and a heating or cooling system these days. Factories and data centers are equally dependent on a continuous functioning of these systems. As beneficial as this development is for our daily life, the consequences of a failure are critical. Malfunctioning power supplies or temperature regulation systems can cause the close-down of an entire factory or data center. Heat and air conditioning losses in buildings lead to a large waste of the limited energy resources and pollute the environment unnecessarily. To detect these flaws as quickly as possible and to prevent the negative consequences constant monitoring of power lines and heat sources is necessary. To this end, we propose a fully automatic system that creates 3D thermal models of indoor environments. The proposed system consists of a mobile platform that is equipped with a 3D laser scanner, an RGB camera and a thermal camera. A novel 3D exploration algorithm ensures efficient data collection that covers the entire scene. The data from all sensors collected at different positions is joined into one common reference frame using calibration and scan matching. In the post-processing step a model is built and points of interest are automatically detected. A viewer is presented that aids experts in analyzing the heat flow and localizing and identifying heat leaks. Results are shown that demonstrate the functionality of the system.
-
Elseberg, J., Borrmann, D., Schauer, J., Nüchter, A., Koriath, D., Rautenberg, U.: A sensor skid for precise 3D modeling of production lines. Proceedings of the Commision V Symposium Close-range imaging, ranging and applications. p. 117--122. , Riva del Garda, Italy (2014).
Motivated by the increasing need of rapid characterization of environments in 3D, we designed and built a sensor skid that automates the work of an operator of terrestrial laser scanners. The system combines terrestrial laser scanning with kinematic laser scanning and uses a novel semi-rigid SLAMmethod. It enables us to digitize factory environments without the need to stop production. The acquired 3D point clouds are precise and suitable to detect objects that collide with items moved along the production line.
-
Li, Q., Chen, L., Li, M., Shaw, S.-L., Nüchter, A.: A Sensor-Fusion Drivable-Region and Lane-Detection System for Autonomous Vehicle Navigation in Challenging Road Scenarios. IEEE Transactions on Vehicular Technology. 63, 540--555 (2014).
Autonomous vehicle navigation is challenging since various types of road scenarios in real urban environments have to be considered, particularly when only perception sensors are used, without position information. This paper presents a novel real-time optimal-drivable-region and lane detection system for autonomous driving based on the fusion of light detection and ranging (LIDAR) and vision data. Our system uses a multisensory scheme to cover the most drivable areas in front of a vehicle. We propose a feature-level fusion method for the LIDAR and vision data and an optimal selection strategy for detecting the best drivable region. Then, a conditional lane detection algorithm is selectively executed depending on the automatic classification of the optimal drivable region. Our system successfully handles both structured and unstructured roads. The results of several experiments are provided to demonstrate the reliability, effectiveness, and robustness of the system.
-
Nüchter, A., Rusu, R.B., Holz, D., Munoz, D.: Editorial: Semantic Perception, Mapping and Exploration. Journal of Robotics and Autonomous Systems (JRAS), Special Issue on Semantic Perception, Mapping and Exploration. 62, 1--2 (2014).
-
Schauer, J., Nüchter, A.: Efficient Point Cloud Collision Detection and Analysis in a Tunnel Environment using Kinematic Laser Scanning and k-d Tree Search. Proceedings of the Photogrammetric Computer Vision (PCV '14). p. 289--295. , Zürich, Switzerland (2014).
Measuring the structure gauge of tunnels and other narrow passages has so far been the only way to evaluate whether large vehicles can pass through them. But especially for very long vehicles like train wagons and their cargo, the structure gauge is an insufficient measure because the center part of the vehicle between two bogies will inevitably leave the swept volume of its cross section when moving along any other trajectory than a straight line perpendicular to its cross section. In addition, the vehicle as well as the cargo must keep a minimum safety margin from the environment at all points of its trajectory. This paper explores an automated method to check for possible collisions of a model represented by a 3D point cloud moving through the 3D point cloud of an environment. We were given environment data of a train track through a narrow tunnel where simply relying on the structure gauge would indicate that a given wagon would pass through without any collision even though in reality, the train wagon would collide with the inner tunnel wall inside a sharp turn of the tracks. The k-d tree based collision detection method presented in this paper is able to correctly highlight these collisions and indicate the penetration depth of each colliding point of the environment into the model of the train wagon. It can be generalized for any setup where two static point clouds have to be tested for intersection along a trajectory.
-
Houshiar, H., Borrmann, D., Nüchter, A.: Fortlaufende semantische 3D-Kartierung von archäologischen Ausgrabungsstätten. Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2014, Jade Hochschule. p. 268--277 (2014).
Das 3D Laserscanning ist Stand der Technik bei der Modellierung archäologischer Ausgrabungsstätten, historischer Anlagen und sogar ganzer Städte oder Landschaften. Die Dokumentation der Befunde auf einer Ausgrabungsstätte ist eine wesentliche archäologische Aufgabe. Ein automatisiertes System würde diesen Prozess beschleunigen und die Anzahl der Fehler auf ein Minimum reduzieren. Dieser Beitrag stellt einen neuen Ansatz in der Dokumentation industrieller Archäologie vor. Er besteht aus einer Reihe von Tools zur Erfassung und Registrierung von 3D-Daten auf Ausgrabungsstätten. Wir stellen ein effizientes Werkzeug zur Verfügung für die Visualisierung der erworbenen 3D-Punktwolken im 3D- und 2D-Modus. Der Hauptzweck dieser Software ist Archäologen ein einfach zu bedienendes Tool für die semantische Kartierung vor Ort zu bieten. Es enthält Funktionen für die Auswahl und Kennzeichnung von Funden. Jedes Label kann mit weiteren Informationen versehen werden. Diese Daten werden im XML-Format exportiert und dienen als Eingabe für andere Systeme und Datenbanken.
-
Bruder, G., Steinicke, F., Nüchter, A.: Immersive Point Cloud Virtual Environments (Poster). Proceedings of IEEE Symposium on 3D User Interfaces 3DUI Proceedings of IEEE Symposium on 3D User Interfaces (3DUI '14). p. 161--162 (2014).
Today's three-dimensional (3D) virtual environments (VEs) are usually based on textured polygonal 3D models, which represent the appearance and geometry of the virtual world. However, some application domains require other graphical paradigms, which are currently not adequately addressed by 3D user interfaces. We introduce a novel approach for a technical human-robot telepresence setup that allows a human observer to explore a VE, which is a 3D reconstruction of the real world based on point clouds. Such point cloud virtual environments (PCVEs) represent the external environment, and are usually acquired by 3D scanners. We present an application scenario, in which a mobile robot captures 3D scans of a terrestrial environment, which are automatically registered to a coherent PCVE. This virtual 3D reconstruction is displayed in an immersive virtual environment (IVE) in which a user can explore the PCVE. We explain and describe the technical setup, which opens up new vistas of presenting a VE as points rather than a polygonal representation.
-
Nüchter, A., Rusu, R.B., Holz, D., Munoz, D. eds: Journal of Robotics and Autonomous Systems (JRAS). (2014).
-
Al khawaldah, M., Nüchter, A.: Multi-Robot Cooperation for Efficient Exploration. AUTOMATIKA -- Journal for Control, Measurement, Electronics, Computing and Communications. 55, 276--286 (2014).
This paper addresses the problem of exploration of an unknown environment by developing effective exploration strategies for a team of mobile robots equipped with continuously rotating 3D scanners. The main aim of the new strategies is to reduce the exploration time of unknown environment. Unlike most of other published works, to save time, the laser scanners rotate and scan the environment while robots are in motion. Furthermore, the new strategies are able to explore large outdoor environments as a considerable reduction of the required computations, especially those required for path planning, have been achieved. Moreover, another new exploration strategy has been developed so that robots continuously replan the order to visit the remaining unexplored areas according to the new data (i.e. updated map) collected by the robot in question or by the other team members. This new extension led to further enhancements over the above mentioned ones, but with slightly higher computational costs. Finally, to assess our new exploration strategies with different levels of environment complexity, new set of experiments were conducted in environments where obstacles are distributed according to the Hilbert curve. The results of these experiments show the effectiveness of the proposed technique to effectively distribute the robots over the environment. More importantly, we show how the optimal number of robots is related to the environment complexity.
-
Bedkowski, J., Majek, K., Nüchter, A.: Nowy algorytm 6DSLAM wykorzystujacy semantyczne rozpoznanie otoczenia. 13 Krajowka Konferencja Robotyki Kudowa Zdroj Prace Naukowe, Elektronika z. 194 Post. Robotyki Tom II. p. 513--520 (2014).
W artykule przedstawiono modyfikację algorytmu 6DSLAM wykorzystującą semantyczne rozpoznawanie otoczenia. Zastosowanie semantycznego podejścia nie tylko poprawia w porównaniu do klasycznej metody dokładność tworzonej mapy metrycznej przez robota mobilnego, ale także umożliwia tworzenie takiej mapy w trudnych warunkach terenowych. Przedstawiono eksperymenty tworzenia mapy metrycznej/semantycznej budynków uwzględniając jazdę poziomą oraz kierunku pionowym po schodach. Porównano wynik z poprzednią implementacją algorytmu 6DSLAM. Nowe podejście poprawia spójność oraz dokładność mapy metrycznej. Zastosowanie semantycznego rozpoznawania otoczenia pozwala na rozszerzenie mapy metrycznej o nowe informacje o charakterze jakościowym, w tym przypadku rozróżniane są ściany, podłoga, sufit oraz punkty charakteryzujące się otoczeniem nieuporządkowanym.
-
Elseberg, J., Borrmann, D., Nüchter., A.: A Study of Scan Patterns for Mobile Mapping. Proceedings of the ISPRS Conference on "Serving Society with Geoinformatics" (ISPRS-SSG '13). p. 75--80. , Antalya, Turkey (2013).
Mobile terrestrial scanning systems automate terrestrial laser scanning. Continous scanning mobile terrestrial systems constantly spin the terrestrial laser scanner and thus combine terrestrial scanning with kinematic laser scanning. This paper presents a scan pattern analysis for these systems. We aim at finding the most advantageous combination of terrestrial and kinematic systems. The resulting 3D point cloud depends on the scan pattern and the trajectory and velocity of the mobile system.
-
Elseberg, J., Borrmann, D., Nüchter, A.: Algorithmic solutions for computing accurate maximum likelihood 3D point clouds from mobile laser scanning platforms. Remote Sensing. 5, 5871--5906 (2013).
Mobile laser scanning puts high requirements on the accuracy of the positioning systems and the calibration of the measurement system. We present a novel algorithmic approach for calibration with the goal of improving the measurement accuracy of mobile laser scanners. We describe a general framework for calibrating mobile sensor platforms that estimates all configuration parameters for any arrangement of positioning sensors, including odometry. In addition, we present a novel semi-rigid Simultaneous Localization and Mapping (SLAM) algorithm that corrects the vehicle position at every point in time along its trajectory, while simultaneously improving the quality and precision of the entire acquired point cloud. Using this algorithm, the temporary failure of accurate external positioning systems or the lack thereof can be compensated for. We demonstrate the capabilities of the two newly proposed algorithms on a wide variety of datasets.
-
Nüchter, A., Elseberg, J., Borrmann, D.: Automation in 3D Laser Scanning -- From an Automated Tripod towards Optimal 3D Point Clouds from Mobile Laser Scanning. GIM International. 27, (2013).
-
Gurau, C., Nüchter, A.: Challenges in Using Semantic Knowledge for 3D Object Classification. Proceedings of the KI 2013 Workshop on Visual and Spatial Cognition, KIK - KI & Kognition Workshop Series. , Koblenz, Germany (2013).
To cope with a wide variety of tasks, robotic systems need to perceive and understand their environments. In particular, they need a representation of individual objects, as well as contextual relations between them. Visual information is the primary data source used to make predictions and inferences about the world. There exists, however, a growing tendency to introduce high-level semantic knowledge to enable robots to reason about objects. We use the Semantic Web framework to represent knowledge and make inferences about sensor data, in order to detect and classify objects in the environment. The contribution of this work is the identification of several challenges that co-occur when combining sensor data processing with such a reasoning method.
-
Bkedkowski, J., Majek, K., Nüchter, A.: General Purpose Computing on Graphics Processing Units for Robotic Applications. Journal of Software Engineering for Robotics (JOSER). 4, 23--33 (2013).
This paper deals with research related with the improvements of state of the art algorithms used in robotic applications based on parallel computation. The main goal is to decrease the computational complexity of 3D cloud of points processing in such applications as: data filtering, normal vector estimation, data registration and calculation of point feature histogram. The presented results efficiently improve the existing implementations with minimal lost of accuracy. The main contribution is a regular grid decomposition originally implemented for nearest neighborhood search. This data structure is used almost for all presented methods, it provides an efficient method for decreasing the time of computation. The results are compared with well-known robotic frameworks such as PCL and 3DTK.
-
Dumitru, R.-C., Borrmann, D., Nüchter, A.: Interior Reconstruction using the 3D Hough Transform. Proceedings of the 5th ISPRS International Workshop 3D-ARCH 2013: "3D Virtual Reconstruction and Visualization of Complex Architectures". , Trento, Italy (2013).
Laser scanners are often used to create accurate 3D models of buildings for civil engineering purposes, but the process of manually vectorizing a 3D point cloud is time consuming and error-prone (Adan and Huber, 2011). Therefore, the need to characterize and quantify complex environments in an automatic fashion arises, posing challenges for data analysis. This paper presents a system for 3D modeling by detecting planes in 3D point clouds, based on which the scene is reconstructed at a high architectural level through removing automatically clutter and foreground data. The implemented software detects openings, such as windows and doors and completes the 3D model by inpainting.
-
Demisse, G., Borrmann, D., Nüchter, A.: Interpreting Thermal 3D Models of Indoor Environments for Energy Efficiency. Proceedings of the 16th IEEE International Conference on Advanced Robotics (ICAR '13). p. 1--8. , Montevideo, Uruguay (2013).
In recent years, 3D models of buildings are used in maintenance and inspection, preservation, and other building related applications. However, the usage of these models is limited, because most models are pure representations with no or little associated semantics. In this paper, we present a pipeline of techniques used for interior interpretation, object detection, and adding energy related semantics to windows of a 3D thermal model. A sequence of algorithms is presented for building the fundamental semantics of a 3D model. Furthermore, a Markov Random Field is used to model the temperature distribution of detected windows to further label the windows as either open, closed or damaged.
-
Nüchter, A., Elseberg, J., Borrmann, D.: Irma3D -- An Intelligent Robot for Mapping Applications. Proceedings of the 3rd IFAC Symposium on Telematics Applications (TA '13). p. 119--124. , Seoul, Korea (2013).
Motivated by the increasing need of rapid characterization of environments in 3D, we designed a robot system that automates the work of an operator of terrestrial laser scanners. The built system enables to work without using special targets or markers and thus enables the surveyors to save more than 75% of the time spent in the field. Another impulse for developing the platform is the demand for a remote multi-sensor inspection tool. The robot is capable of surveying remote sites or danger areas, such as plants, underground mines, tunnels, caves, or channels. The results are precise, multi-modal digital 3D maps. This paper presents the recently developed robot Irma3D, its hardware, the developed interconnected software modules, the associated sensor calibration methods and a few applications.
-
Zhang, L., Li, Q., Li, M., Mao, Q.Z., Nüchter, A.: Multiple Vehicle-like Target Tracking Based on Velodyne Lidar. Proceedings of the 6th IFAC Symposium on Intelligent Autonomous Vehicles (IAV '13). , Gold Coast, Australia (2013).
This paper proposes a novel multiple vehicle-like target tracking method based on a Velodyne HDL64E light detection and ranging (LiDAR) system. The proposed method combines multiple hypothesis tracking (MHT) algorithm with dynamic point cloud registration (DPCR), which is able to solve the multiple vehicle-like target tracking in highly dynamic urban environments without any auxiliary information from GPS or IMU. Specifically, to track targets consistently, the DPCR is developed to calculate accurately the pose of the ego-vehicle for the transformation of raw measurements taken in the moving coordinate systems into a static absolute coordinate system; while in turn, MHT helps to improve the performance of DPCR by discriminating and removing the dynamic points from the scene. Furthermore, the proposed MHT method is also able to solve the occlusion problem existing in the point cloud. Experiments on sets of urban environments prove that the presented method is effective and robust, even in highly dynamic environments.
-
Houshiar, H., Borrmann, D., Elseberg, J., Nüchter, A., Winkler, S., Näth, F.: On-Site Semantic mapping of Archeological Excavation Areas. Proceedings of the XXIV International CIPA Symposium. p. 163--168. , Strasbourg, France (2013).
3D laser scanning is the state of the art in modeling archaeological excavation sites, historical sites and even entire cities or landscapes. The documentation of findings on an excavation site is an essential archaeological task. Automated systems accelerate this process and decrease the amount of error to a minimum. This paper presents a new documentation approach in industrial archaeology. It consists of a set of tools for recording and registering 3D data from excavation sites. We provide an efficient tool for visualization of acquired 3D point clouds in 3D and 2D modes. The main purpose of this software is to provide an easy to use, on-site semantic mapping tool for archaeologists. It includes functions for selecting and labeling findings. Additional information can be provided for each label. This data is exported to an XML format and serves as input for other systems and databases.
-
Elseberg, J., Borrmann, D., Nüchter, A.: One Billion Points in the Cloud --- An Octree for Efficient Processing of 3D Laser Scans. ISPRS Journal of Photogrammetry & Remote Sensing (JPRS), Special issue on terrestrial 3D modelling. 76, 76--88 (2013).
Automated 3-dimensional modeling pipelines include 3D scanning, registration, data abstraction, and visualization. All steps in such a pipeline require the processing of a massive amount of 3D data, due to the ability of current 3D scanners to sample environments with a high density. The increasing sampling rates make it easy to acquire Billions of spatial data points. This paper presents algorithms and data structures for handling these data. We propose an efficient octree to store and compress 3D data without loss of precision. We demonstrate its usage for an exchange file format, fast point cloud visualization, sped-up 3D scan matching, and shape detection algorithms. We evaluate our approach using typical terrestrial laser scans.
-
Nüchter, A., Elseberg, J., Borrmann, D.: Optimale 3D-Punktwolken aus mobilen Laserscandaten. Photogrammetrie Laserscanning Optische 3D-Messtechnik, Beiträge der Oldenburger 3D-Tage 2013, Jade Hochschule. p. 186--193 (2013).
Mobiles Laserscanning stellt hohe Anforderungen an die Exaktheit der Positionierung und Kalibrierung des Messsystems, da sich bereits kleinste Ungenauigkeiten in den akquirierten 3D-Daten drastisch niederschlagen können. Dieser Beitrag präsentiert eine neuen Ansatz zur Kalibrierung und zur Lösung des SLAM-Problems mit dem Ziel, die Messgenauigkeit des mobilen Scansystems zu optimieren. Bei dem Kalibrierungsverfahren handelt es sich um eine allgemeingültige Methode, die die Positionierung und Orientierung sämtlicher Sensoren berechnet, einschliesslich der Odometrie. Der vorgestellte SLAM-Ansatz (Simultaneous Localization and Mapping) ist semi-rigide. Dabei wird die Trajektorie so deformiert und jede Pose dahingehend optimiert, dass die Gesamtqualität der 3D-Punktwolke optimal wird. Das SLAM-Verfahren kann kurzfristige Sensorausfälle oder Ungenauigkeiten ausgleichen und ermöglicht vielfältige neue Anwendungen.
-
Houshiar, H., Elseberg, J., Borrmann, D., Nüchter, A.: Panorama Based Point Cloud Reduction and Registration. Proceedings of the 16th IEEE International Conference on Advanced Robotics (ICAR '13). p. 1--8. , Montevideo, Uruguay (2013).
To reconstruct environments 3D point clouds acquired by laser scanners are registered. This is an important but also time consuming part of any mapping system for mobile robots. The time needed for mapping is drastically reduced when the size of the input data is reduced. This paper examines different ways of reducing the size of point clouds without losing vital information for the matching process. We present novel point cloud reduction methods on the basis of panorama images. It is shown that the reduced point clouds are ideally suited for feature based registration on panorama images. We evaluate the presented reduction methods based on their effect on the performance of the registration algorithm.
-
Schilling, K., Tzschichholz, T., Motroniuk, I., Aumann, A., Mammadov, I., Ruf, O., Schmidt, C., Appel, N., Kleinschrodt, A., Montenegro, S., Nüchter, A.: TOM: A formation for photogrammetric earth observation by three cubesats. Proceedings IAA-ASS-18 : International Academy of Astronautics. p. 1--14. Univelt, Inc., Changsha, China (2018).
Nüchter, A., Borrmann, D., Elseberg, J., Redondo, D.: A Backpack-Mounted 3D Mobile Scanning System. Allgemeine Vermessungs-Nachrichten (AVN), Special Issue MoLAS 2014. 122, 301--307 (2015).