22.09.2020
OptoFence II

Project goals

• Development of a fast telescope system for UAV detection and tracking
• Real-time deep learning object detection and tracking
• Camera-based control with high-speed auto focus

Description

Drones and other types of unmanned aerial vehicles (UAVs) have gained massive popularity not only in the professional but also in the private sector in recent years. Incidents such as the closure of London’s Gatwick Airport due to a drone sighting demonstrate that advances in UAV technology pose a threat to public safety. The early identification of incoming UAVs is of the highest priority for situational assessment.
Commercial drone detection systems use a multispectral approach for object detection and identification. For this purpose, the interaction of different sensors is used to be able to recognize and identify objects. The figure below shows an example where an object at a distance of 5 to 10 km is detected using radar. The problem, however, is that it is difficult to differentiate between a UAV and, for example, a bird. Optical sensors are used for this, which can clearly classify the object based on a recorded camera image. The operational distance of this optical component is currently limited to one to two kilometers, which only allows short reaction times in the event of a threat.

OptoFence II aims to develop a telescope-based optical platform to enable a larger identification area, which significantly increases situational awareness. The combination of a precise and fast mount, a high-quality telescope, a camera system and advanced methods of control systems and computer vision creates a versatile platform for the optical detection, tracking and identification of UAVs.

The basic concept is shown in Figure 2. A suitable pair of telescopes and cameras provide high-resolution images. In the next step, these are analyzed using modern deep learning algorithms to extract the position of the UAV in the individual images. For this purpose, an efficient software architecture was implemented in the project, which enables the detection and tracking of drones at up to 100 images per second. The data obtained serves as input for advanced control technology controllers to enable the telescope to follow the flight path of the object precisely. In addition, a specially implemented automatic focus tracking system enables the drone to be kept in sharp focus. The implemented system was intensively tested during field tests under various scenarios. It was demonstrated that small drones, such as the DJI Mavic 3, can be detected and tracked up to a distance of 5 km.

 

Use case

• UAV reconnaissance

Related Publications

• D. Ojdanić, A. Sinn, C. Naverschnigg, and G. Schitter, Feasibility Analysis of Optical UAV Detection Over Long Distances Using Robotic Telescopes, IEEE Transactions on Aerospace and Electronic Systems, vol. 59, iss. 5, pp. 5148-5157, 2023.
  [BibTex] [Download]

  @ARTICLE{2023_ojdanic_taes,
  author={Ojdani{\'c}, Denis and Sinn, Andreas and Naverschnigg, Christopher and Schitter, Georg},
  journal={IEEE Transactions on Aerospace and Electronic Systems},
  title={Feasibility Analysis of Optical UAV Detection Over Long Distances Using Robotic Telescopes},
  year={2023},
  volume={59},
  number={5},
  pages={5148-5157},
  doi={10.1109/TAES.2023.3248560},
  }

• D. Ojdanić, C. Naverschnigg, A. Sinn, D. Zelinskyi, and G. Schitter, Parallel Architecture for Low Latency UAV Detection and Tracking Using Robotic Telescopes, IEEE Transactions on Aerospace and Electronic Systems, vol. 60, iss. 4, pp. 5515-5524, 2024.
  [BibTex] [Download]

  @Article{2024_ojdanic_taes,
  author={Ojdanić, Denis and Naverschnigg, Christopher and Sinn, Andreas and Zelinskyi, Daniil and Schitter, Georg},
  journal={IEEE Transactions on Aerospace and Electronic Systems},
  title={Parallel Architecture for Low Latency UAV Detection and Tracking Using Robotic Telescopes},
  year={2024},
  volume={60},
  number={4},
  pages={5515-5524},
  doi={10.1109/TAES.2024.3396418},
  }

• D. Ojdanić, D. Zelinskyi, C. Naverschnigg, A. Sinn, and G. Schitter, High-speed telescope autofocus for UAV detection and tracking, Optics Express, vol. 32, iss. 5, p. 7147–7157, 2024.
  [BibTex] [Download]

  @Article{2024_ojdanic_oe,
  author = {Ojdani{\'c}, Denis and Zelinskyi, Daniil and Naverschnigg, Christopher and Sinn, Andreas and Schitter, Georg},
  journal = {Optics Express},
  title = {High-speed telescope autofocus for UAV detection and tracking},
  year = {2024},
  issn = {1094-4087},
  month = feb,
  number = {5},
  pages = {7147--7157},
  volume = {32},
  doi = {10.1364/oe.514859},
  publisher = {Optica Publishing Group},
  }

• D. Ojdanić, N. Paternoster, C. Naverschnigg, A. Sinn, and G. Schitter, Evaluation of the required optical resolution for deep learning-based long-range UAV detection, in Pattern Recognition and Tracking XXXV, 2024.
  [BibTex] [Download]

  @InProceedings{2024_ojdanic_spie,
  author = {Ojdani{\'c}, Denis and Paternoster, Niklas and Naverschnigg, Christopher and Sinn, Andreas and Schitter, Georg},
  booktitle = {Pattern Recognition and Tracking XXXV},
  title = {Evaluation of the required optical resolution for deep learning-based long-range UAV detection},
  year = {2024},
  month = jun,
  publisher = {SPIE},
  doi = {10.1117/12.3013251},
  }

• D. Ojdanić, C. Naverschnigg, A. Sinn, and G. Schitter, Algorithm evaluation for parallel detection and tracking of UAVs, in Optics, Photonics, and Digital Technologies for Imaging Applications VIII, 2024.
  [BibTex] [Download]

  @InProceedings{2024_ojdanic_spie1,
  author = {Ojdani{\'c}, Denis and Naverschnigg, Christopher and Sinn, Andreas and Schitter, Georg},
  booktitle = {Optics, Photonics, and Digital Technologies for Imaging Applications VIII},
  title = {Algorithm evaluation for parallel detection and tracking of UAVs},
  year = {2024},
  month = jun,
  publisher = {SPIE},
  doi = {10.1117/12.3017037},
  }

• D. Ojdanić, C. Naverschnigg, A. Sinn, and G. Schitter, Deep learning-based long-distance optical UAV detection: color versus grayscale, in Pattern Recognition and Tracking XXXIV, 2023.
  [BibTex] [Download]

  @InProceedings{2023_ojdanic_spie,
  author = {Ojdani{\'c}, Denis and Naverschnigg, Christopher and Sinn, Andreas and Schitter, Georg},
  booktitle = {Pattern Recognition and Tracking XXXIV},
  title = {Deep learning-based long-distance optical UAV detection: color versus grayscale},
  year = {2023},
  month = jun,
  publisher = {SPIE},
  doi = {10.1117/12.2663318},
  }

• D. Ojdanić, B. Gräf, A. Sinn, H. W. Yoo, and G. Schitter, Camera-guided real-time laser ranging for multi-UAV distance measurement, Appl. Opt., vol. 61, iss. 31, p. 9233–9240, 2022.
  [BibTex] [Download]

  @article{ojdanic2022AppliedOptics,
  author = {Denis Ojdani\'{c} and Benjamin Gr\"{a}f and Andreas Sinn and Han Woong Yoo and Georg Schitter},
  journal = {Appl. Opt.},
  keywords = {Laser beams; Laser light; Laser ranging; Optical components; Optical testing; Remote sensing},
  number = {31},
  pages = {9233--9240},
  publisher = {Optica Publishing Group},
  title = {Camera-guided real-time laser ranging for multi-UAV distance measurement},
  volume = {61},
  month = {Nov},
  year = {2022},
  doi = {10.1364/AO.470361},
  }

• D. Ojdanic, A. Sinn, C. Schwaer, and G. Schitter, UAV Detection and Tracking with a Robotic Telescope System, in Proceedings of the Advanced Intelligent Mechatronics Conference 2021, 2021.
  [BibTex]

  @inproceedings{TUW-299533,
  author = {Ojdanic, Denis and Sinn, Andreas and Schwaer, Christian and Schitter, Georg},
  title = {UAV Detection and Tracking with a Robotic Telescope System},
  booktitle = {Proceedings of the Advanced Intelligent Mechatronics Conference 2021},
  year = {2021},
  note = {Posterpr{\"a}sentation: 2021 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Delft, Netherlands; 2021-07-12 -- 2021-07-16}
  }

Project partners

• ASA Astrosystems GmbH
• Austrian Federal Ministry of Defence

Funding

This project is funded by the Austrian defense research programme FORTE of the Federal Ministry of Agriculture, Regions and Tourism (BMLRT).

  27.03.2020
Precision robotic in-line metrology for freeform surfaces

Project focus

• Measurement platform with 6DoF and arbitrary operation orientation
• Integrated control strategies for dual-actuated positioning and automated measurement procedures
• Conduct high precision measurements on a moving object directly in a production line

Description

High precision in-line measurements are considered as a key factor for future industrial production.  Particularly for surface measurements of free forms directly in a production line, flexible and versatile measurement systems are required. Robot-based measurement systems enable the required flexibility and are thus considered as a key factor. High-precision measurement systems, which operate at the micrometer level, are typically highly sensitive to vibrations and, therefore, usually applied in vibration-free laboratory environments. This is, however in complete contrary to the vibration-prone environment of an industrial manufacturing plant. Relative motion between the measurement tool and the sample in 6 DOFs is caused by environmental floor vibrations, disturbing measurements at the micrometer level.

Within the scope of this project, a vibration compensating measurement module is developed. This module is used as an end effector and coarsely aligned by the robot arm. The actuated platform, which carries the high precision measurement tool, compensates for disturbing vibrations in 6 DoFs by means of feedback control. A stiff link between the measurement tool and the sample is established, maintaining a constant alignment of the tool with respect to the sample surface.

The feasibility of the system concept has been shown for horizontal operation orientation in the project aim4np. The goal of this project is to enable contact-less high precision robot-based measurements of free form surfaces by a measurement module with orientation-independent system performance.

 

Platform Design and Integration

To mechanically decouple the measurement platform from the robot arm, (quasi-)zero stiffness actuation is desired. Therefore, eight voice coil actuators are utilized to levitate and actuated the platform in 6 DoFs within the airgap between coils and magnet housings. By use of FEM analysis tools, the structural stiffness of the platform is analyzed, aiming for a high position control bandwidth and vibration compensation capability. Good decoupling of the DoFs is ensured by a balanced design, avoiding unintended torsional motion through actuation and keeping the subsequent control effort at a low level. The system is deliberately over-actuated, providing high peak forces, and the arbitrary operation orientation of the entire system. An integrated sensor system determines the actual position of the moving platform with respect to the supporting frame, while tracking sensors are used to measure the distance and orientation between platform and sample.

 

Precision Motion Control

Model-based control techniques are applied to ensure a robust and high-performance positioning of the electro-magnetically actuated platform. A system model is built to decouple the MIMO plant, resulting in the implementation of a SISO controller for each DoF. A transition scheme is researched to operate the system in two different control modes, the stabilization of the platform with respect to the supporting frame and the tracking control of a sample surface.

 

Compact scanning confocal chromatic sensor as precision 3-D measurement tool

By manipulating the optical path of a 1-D confocal chromatic sensor with a high performance 2-D fast steering mirror, 3-D imaging is enabled. Using a CMOS camera, calibration data is acquired for a precise image reconstruction. The developed 3-D tool is capable of performing fast overview images and high resolution scans of arbitrary region of interests. The lateral scan range is about 350µm x 250µm, with scan speeds of up to 1 frame per second. The achieved lateral and axial resolution is 2.5µm and 7nm, respectively.

Applications

• Inline measurement systems
• Robot-based precision measurement and manufacturing systems

Project Partners

• Micro-Epsilon Messtechnik GmbH & Co. KG
• ATENSOR Engineering and Technology Systems GmbH

Funding

• Christian Doppler Research Association
• The Austrian Federal Ministry for Digital and Economic Affairs
• The National Foundation for Research, Technology and Development

  26.03.2020
Hybrid reluctance actuators for high precision motion

Project focus

• Hybrid reluctance actuators (HRAs) realize a motor constant higher than comparable voice coil actuators for motion control with nanometer resolution.
• HRAs’ magnetic force cancels the stiffness of guiding flexures for large motion and energy efficiency.
• Integrated mechatronic system design enables a robust PID controller to realize a high bandwidth.

Description

Next-generation high-quality motion systems require high-precision actuators with higher energy efficiency and larger force to improve the system throughput. Particularly, actuators with a motor constant higher than comparable voice coil actuators are highly desired. This project investigates hybrid reluctance actuators (HRAs) with guiding flexures as a promising candidate of next-generation systems. The following table and video show the currently ahcieved performance:

Table 1: Evaluated performance of a HRA

Actuation principle

A hybrid reluctance actuator (HRA) uses a permanent magnet and coils to generate magnetic fluxes through the ferromagnetic stator and mover. As Fig. 1(a) illustrates, the direction of the coil’s flux is the opposite of the magnet’s flux in the variable right gap. These fluxes cancel each other. Consequently, the total flux in the variable right gap is weaker than that in the variable left gap. The unbalanced flux results in the lateral actuation force.

HRAs generate a bidirectional force. It is relatively proportional to the coil current. The motor constant of HRAs can be a few times larger than that of a comparable voice coil actuators. However, unlike voice coil actuators, HRAs exhibit position-dependent magnetic force, which is modeled by a nonlinear negative stiffness.

Stiffness cancellation for long-range motion and high energy efficiency

A challenge of flexure-guided systems in general is that positioning requires a constant force against the flexure stiffness, which is significant around the stroke end. For energy efficiency, a HRA is designed such that its negative stiffness cancels the flexure stiffness. The simulation in Fig. 1(b) shows that the negative stiffness decreases the force required for a quasi-static positioning from more than +/-80 N to less than +/-15 N. The experiment in Fig. 1(c) reveals that the power required for the positioning around the stroke end at 1 mm is less than that around 0.7mm, demonstrating the effectiveness of the stiffness cancellation.

For high precision motion, the HRA is designed such that a robust PID controller with a position sensor compensates for the nonlinear negative stiffness for closed-loop stability within the entire actuation rage of 2 mm. The closed-loop system achieves a positioning resolution of 2.5 nm and a bandwidth of 640 Hz. Furthermore, data-driven learning control is combined for accurate scanning motion. The experimental results in Fig. 2 show triangular scanning motions with an amplitude of 1.6 mm at 1 Hz and with an amplitude of 10 um at 100 Hz, where the tracking error is 10 nm (rms). Overall, the HRA realizes a large dynamic range with high energy efficiency.

Magnetic flux feedback control for system linearity

HRAs exhibit nonlinearities, which include not only the negative stiffness, but also the magnetic hysteresis seen in a B-H curve. They impair scanning motions and can make HRAs open-loop unstable with a commonly used current amplifiers. Such nonlinearities are compensated by replacing conventional current amplifiers by a proposed flux amplifier where feedback control regulates a magnetic flux within the frequency band of interest.

As illustrated in Fig. 3, the flux loop includes the B-H hysteresis between the coil current and the resulting flux. Furthermore, the position-dependent flux that results in the negative stiffness is regarded as a disturbance. Consequently, the flux feedback controller rejects the nonlinearities. In fact, experimental results in Fig. 4 demonstrate that the proposed flux amplifier significantly decreases the motion error of a triangular scanning in comparison with a conventional current amplifier, even without real-time motion control.

Applications

• Nanopositioner / Micropositioner
• Dual stage actuator
• Scanner
• Vibration isolation

Related publications

Journal articles

• S. Ito, F. Cigarini, and G. Schitter, Flux-controlled Hybrid Reluctance Actuator for High-precision Scanning Motion, IEEE Transactions on Industrial Electronics, vol. 67, iss. 11, pp. 9593-9600, 2020.
  [BibTex] [Download]

  @Article{TUW-282490,
  author = {Ito, Shingo and Cigarini, Francesco and Schitter, Georg},
  title = {Flux-controlled Hybrid Reluctance Actuator for High-precision Scanning Motion},
  journal = {IEEE Transactions on Industrial Electronics},
  year={2020},
  volume={67},
  number={11},
  pages={9593-9600},
  doi = {10.1109/TIE.2019.2952829},
  keywords = {Actuators, Nanopositioning, Magnetic variables control, Motion control},
  }

• S. Ito, S. Troppmair, B. Lindner, F. Cigarini, and G. Schitter, Long-range Fast Nanopositioner Using Nonlinearities of Hybrid Reluctance Actuator for Energy Efficiency, IEEE Transactions on Industrial Electronics, vol. 66, iss. 4, 2019.
  [BibTex] [Download]

  @Article{TUW-271193,
  author = {Ito, Shingo and Troppmair, Stefan and Lindner, Bernhard and Cigarini, Francesco and Schitter, Georg},
  title = {Long-range Fast Nanopositioner Using Nonlinearities of Hybrid Reluctance Actuator for Energy Efficiency},
  journal = {IEEE Transactions on Industrial Electronics},
  year = {2019},
  volume = {66},
  number = {4},
  doi = {10.1109/TIE.2018.2842735},
  keywords = {Actuators, motion control, nanoposition- ing, magnetic circuits},
  numpages = {9},
  }

• F. Cigarini, S. Ito, S. Troppmair, and G. Schitter, Comparative Finite Element Analysis of a Voice Coil Actuator and a Hybrid Reluctance Actuator, IEEJ Journal of Industry Applications, vol. 8, iss. 2, 2019.
  [BibTex]

  @Article{TUW-276121,
  author = {Cigarini, Francesco and Ito, Shingo and Troppmair, Stefan and Schitter, Georg},
  title = {Comparative Finite Element Analysis of a Voice Coil Actuator and a Hybrid Reluctance Actuator},
  journal = {IEEJ Journal of Industry Applications},
  year = {2019},
  volume = {8},
  number = {2},
  doi = {10.1541/ieejjia.8.192},
  keywords = {Electromagnetic actuators, high-precision motion, finite element analysis},
  numpages = {8}
  }

• S. Ito, S. Troppmair, F. Cigarini, and G. Schitter, High-speed Scanner with Nanometer Resolution Using a Hybrid Reluctance Force Actuator, IEEJ Journal of Industry Applications, vol. 8, iss. 2, 2019.
  [BibTex]

  @Article{TUW-272944,
  author = {Ito, Shingo and Troppmair, Stefan and Cigarini, Francesco and Schitter, Georg},
  title = {High-speed Scanner with Nanometer Resolution Using a Hybrid Reluctance Force Actuator},
  journal = {IEEJ Journal of Industry Applications},
  year = {2019},
  volume = {8},
  number = {2},
  doi = {10.1541/ieejjia.8.170},
  keywords = {High-precision actuators, hybrid reluctance actuator, nanopositioning, motion control.},
  numpages = {7}
  }

Conference papers

• F. Cigarini, S. Ito, J. Konig, A. Sinn, and G. Schitter, Compensation of Hysteresis in Hybrid Reluctance Actuator for High-Precision Motion, in Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), 2019.
  [BibTex]

  @InProceedings{TUW-281659,
  author = {Cigarini, Francesco and Ito, Shingo and Konig, Julian and Sinn, Andreas and Schitter, Georg},
  title = {Compensation of Hysteresis in Hybrid Reluctance Actuator for High-Precision Motion},
  booktitle = {Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019)},
  year = {2019},
  volume = {52/15},
  note = {Vortrag: Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), Wien; 2019-09-04 -- 2019-09-06},
  doi = {10.1016/j.ifacol.2019.11.721},
  journal = {IFAC-PapersOnLine/Elsevier},
  keywords = {Hybrid reluctance actuator, magnetic hysteresis, magnetic flux control},
  numpages = {6}
  }

• S. Ito, B. Lindner, and G. Schitter, Sample-tracking Vibration Isolation with Hybrid Reluctance Actuators for Inline Metrology, in Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), 2019.
  [BibTex]

  @InProceedings{TUW-281655,
  author = {Ito, Shingo and Lindner, Bernhard and Schitter, Georg},
  title = {Sample-tracking Vibration Isolation with Hybrid Reluctance Actuators for Inline Metrology},
  booktitle = {Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019)},
  year = {2019},
  volume = {52/15},
  note = {Vortrag: Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), Wien; 2019-09-04 -- 2019-09-06},
  doi = {10.1016/j.ifacol.2019.11.731},
  journal = {IFAC-PapersOnLine/Elsevier},
  keywords = {Vibration control, Micro and Nano Mechatronic Systems, Motion Control Systems},
  numpages = {6}
  }

• F. Cigarini, S. Ito, S. Troppmair, and G. Schitter, Comparison of a Voice Coil and a Hybrid Reluctance Actuator via FEM, in Proceedings of the IEEJ International Workshop on Sensing, Actuation, Motion Control and Optimization (SAMCON), 2018.
  [BibTex]

  @InProceedings{TUW-268854,
  author = {Cigarini, Francesco and Ito, Shingo and Troppmair, Stefan and Schitter, Georg},
  title = {Comparison of a Voice Coil and a Hybrid Reluctance Actuator via FEM},
  booktitle = {Proceedings of the IEEJ International Workshop on Sensing, Actuation, Motion Control and Optimization (SAMCON)},
  year = {2018},
  note = {Vortrag: IEEJ International Workshop on Sensing, Actuation, Motion Control and Optimization (SAMCON), Tokyo (Japan); 2018-03-06 -- 2018-03-09},
  keywords = {Electromagnetic actuators, high-precision applications, finite element method},
  numpages = {6}
  }

• S. Ito, S. Troppmair, F. Cigarini, and G. Schitter, Hybrid Reluctance Actuator for High-speed Scanning with Nanometer Resolution, in Proceedings of the IEEJ International Workshop on Sensing, Actuation, Motion Control and Optimization (SAMCON), 2018.
  [BibTex]

  @InProceedings{TUW-268844,
  author = {Ito, Shingo and Troppmair, Stefan and Cigarini, Francesco and Schitter, Georg},
  title = {Hybrid Reluctance Actuator for High-speed Scanning with Nanometer Resolution},
  booktitle = {Proceedings of the IEEJ International Workshop on Sensing, Actuation, Motion Control and Optimization (SAMCON)},
  year = {2018},
  note = {Vortrag: IEEJ International Workshop on Sensing, Actuation, Motion Control and Optimization (SAMCON), Tokyo (Japan); 2018-03-06 -- 2018-03-09},
  numpages = {6}
  }

 

Project Partners

• Micro-Epsilon Messtechnik GmbH & Co. KG
• ATENSOR Engineering and Technology Systems GmbH

Funding

• Christian Doppler Research Association
• The Austrian Federal Ministry for Digital and Economic Affairs
• The National Foundation for Research, Technology and Development

  21.02.2020
Application of Curved Tapes on 3D Objects

Project focus

• Application of pre-cut tapes on freeform 3D surface
• Draping process with a defined normal force
• Traverse a tool along a surface-based path on a freeform 3D surface

Description

Robotic handling of deformable materials like, e. g. textiles, carbon and glass fiber fabrics, foils and paper, has been a challenging task in the industry for a long time, and has received great attention in research recently. Flexible automation on a human level has not yet been fully achieved for many handling tasks like, e. g., grasping, de-stacking and draping. The main challenges for the automated handling of deformable materials are their characteristics, in particular the nonlinear material behavior, the anisotropy, the diverse outer contours, the high dimensionality of the object state, and the wide range of material properties like porosity and density. In this context, many specialized mechatronic gripper solutions were implemented for grasping and transporting. However, high-mix and low-volume tasks require more flexible solutions.

This project presents a flexible approach for impedance-controlled wrinkle-free application of (curved) pre-cut adhesive tapes on freeform 3D surfaces. For this application, a surface-based path following control concept is developed, which provides a local coordinate frame based on the surface normal and the path tangent. Furthermore, the algorithms and solutions emerging from this application also represent a solution for a more general robotic problem, i. e. traversing a tool along a curved path on a freeform 3D surface with defined kinematic constraints.

The tape application process consists of the following steps and is depicted in Fig. 1:

• (a) 2D tape application path: A planar tape application path is created on the shape of the planar tape
• (b) 3D path mapping: The geometry and curvature of the planar tape application path is mapped onto the CAD model of the 3D object, which creates a 3D tape application path.
• (c) Robot starting pose: The tape application process is simulated to determine a suitable starting joint configuration for the robot which results in a collisionfree execution and has a reasonable reserve with respect to the joint limits.
• (d) Preparation and execution: The pre-cut tape is placed in the required starting position on the feed of the application tool and the robot performs the impedance-controlled tape application process.

The surface-based path following control concept is demonstrated on a 3D-printed object, as shown in Fig. 2: The paths of the pre-cut tapes in (a) are mapped onto the discrete surface of the 3D object in (b). The tape application result is depicted in (c).

Videos

Selected publications

• T. Weingartshofer, M. Schwegel, C. Hartl-Nesic, T. Glück, and A. Kugi, Collaborative Synchronization of a 7-Axis Robot, in Proceedings of the 8th IFAC Symposium on Mechatronic Systems MECHATRONICS 2019, Vienna, Austria, 2019, pp. 507-512.
  [BibTex]

  @InProceedings{Weingartshofer2019,
  author = {Weingartshofer, T. and Schwegel, M. and Hartl-Nesic, C. and Gl\"uck, T. and Kugi, A.},
  title = {Collaborative Synchronization of a 7-Axis Robot},
  booktitle = {Proceedings of the 8th IFAC Symposium on Mechatronic Systems MECHATRONICS 2019},
  year = {2019},
  volume = {52},
  number = {15},
  month = {9},
  pages = {507-512},
  doi = {10.1016/j.ifacol.2019.11.726},
  address = {Vienna, Austria},
  issn = {2405-8963},
  }

• C. Hartl-Nesic, B. Bischof, T. Glück, and A. Kugi, Pfadfolgeregelung mit Konzepten für den Pfadfortschritt: Ein Assemblierungsszenario, at – Automatisierungstechnik, vol. 68, iss. 1, p. 44–57, 2020.
  [BibTex]

  @Article{Hartl-Nesic2020,
  author = {Hartl-Nesic, Christian and Bischof, Bernhard and Gl{\"u}ck, Tobias and Kugi, Andreas},
  title = {Pfadfolgeregelung mit Konzepten für den Pfadfortschritt: Ein Assemblierungsszenario},
  journal = {at -- Automatisierungstechnik},
  year = {2020},
  volume = {68},
  number = {1},
  pages = {44--57},
  issn = {2196-677X},
  doi = {10.1515/auto-2019-0114},
  }

• C. Hartl-Nesic, Surface-Based Path Following Control on Freeform 3D Objects, A. Kugi, K. Schlacher, and W. Kemmetmüller, Eds., Düren: Shaker Verlag, 2020, vol. 49.
  [BibTex]

  @Book{HartlNesic2020,
  author = {Hartl-Nesic, C.},
  title = {Surface-Based Path Following Control on Freeform 3D Objects},
  editor = {A. Kugi and K. Schlacher and W. Kemmetm\"uller},
  isbn = {978-3-8440-7637-0},
  publisher = {Shaker Verlag},
  series = {Modellierung und Regelung komplexer dynamischer Systeme},
  volume = {49},
  address = {D\"uren},
  organization = {Institute f{\"u}r Automatisierungs- und Regelungstechnik (TU Wien) und Regelungstechnik und Prozessautomatisierung (JKU Linz)},
  year = {2020},
  }

• C. Hartl-Nesic, T. Glück, and A. Kugi, Surface-Based Path Following Control: Application of Curved Tapes on 3-D Objects, IEEE Transactions on Robotics, vol. 37, iss. 2, p. 615–626, 2021.
  [BibTex] [Download]

  @Article{HartlNesic2021,
  author = {Hartl-Nesic, Ch. and Gl{\"u}ck, T. and Kugi, A.},
  title = {Surface-Based Path Following Control: Application of Curved Tapes on 3-D Objects},
  doi = {10.1109/TRO.2020.3033721},
  issn = {1552-3098},
  number = {2},
  pages = {615--626},
  volume = {37},
  journal = {IEEE Transactions on Robotics},
  year = {2021},
  }

Applications

• Textile/garment industry
• Fiber reinforced plastics industry
• Food industry

  22.01.2020
Mathematical modelling, optimization and quality monitoring for side-trimming of heavy steel plates

Project focus

• Analysis and modeling of the side-trimming process of heavy steel plates
• Optimization of the process and the machine design
• Root cause analysis and minization of common quality defects
• Development of a camera- and laser-based quality monitoring system

Description

Side-trimming is an essential process step in the steel industry. The objective of this shearing process is to obtain rectangularly shaped heavy plates after the rolling process. Rolling-cut trimming shears perform this task stepwise and purely mechanically by a rolling-cuts of arc-shaped trimming blades. Due to plate thicknesses up to 45mm, the process forces can reach values of 5MN. Although rolling-cut shearing is the most common method for side-trimming of heavy steel plates, it may entail quality defects like broken edge surfaces, unevenly trimmed edges, or burr. Furthermore, the trimming process might cause a deviation of the plate contour from its ideal rectangular shape.

In this research project, it was found that the lateral process forces acting on the cutting blades have a significant influence on the resulting quality of the trimmed edges. An objective of this project is therefore to obtain a constant lateral process force by optimization of the machine design and the drive kinematics. As a consequence, both quality defects of the trimmed edges and wear of the cutting blades would be reduced.

Another strand of this research project is the development of an automatic quality monitoring system. This system evalutates the quality and the geometric accuracy of each trimmed edge using machine vision and laser-sensor technology. The generated and stored data sets can be used for optimizing the adjustable process parameters and for predictive maintenance of the cutting blades.

Selected publications

• A. Zeiler, A. Steinboeck, A. Kugi, and M. Jochum, Lateral Forces in Rolling-Cut Shearing and Their Consequences on Common Edge Defects, Journal of Manufacturing Science and Engineering, vol. 141, iss. 4, p. 41001-1–41001-9, 2019.
  [BibTex] [Download]

  @Article{Zeiler2019,
  author = {Zeiler, A. and Steinboeck, A. and Kugi, A. and Jochum, M.},
  title = {Lateral Forces in Rolling-Cut Shearing and Their Consequences on Common Edge Defects},
  journal = {Journal of Manufacturing Science and Engineering},
  year = {2019},
  volume = {141},
  number = {4},
  pages = {41001-1--41001-9},
  doi = {10.1115/1.4042578},
  }

• A. Zeiler, A. Steinboeck, M. Vincze, M. Jochum, and A. Kugi, Vision-based inspection and segmentation of trimmed steel edges, in Proceedings of the 18th IFAC Symposium on Control, Optimization and Automation in Mining, Mineral and Metal Processing, Stellenbosch, South Africa, 2019, p. 165–170.
  [BibTex]

  @InProceedings{Zeiler2019a,
  author = {Zeiler, A. and Steinboeck, A. and Vincze, M. and Jochum, M. and Kugi, A.},
  title = {Vision-based inspection and segmentation of trimmed steel edges},
  booktitle = {Proceedings of the 18th IFAC Symposium on Control, Optimization and Automation in Mining, Mineral and Metal Processing},
  year = {2019},
  month = {8},
  pages = {165--170},
  doi = {10.1016/j.ifacol.2019.09.182},
  address = {Stellenbosch, South Africa},
  }

Applications

• Shearing processes
• Automation of rolling mills
• Continuous production processes

  23.10.2019
Omron

The purpose of this project is to develop a software for estimating 6D pose and grasping points of non-rigid objects in bin-picking scenarios. This software would be used with industrial robots for picking individually different shaped objects like toys, vegetables, and fruits those are randomly piled in boxes or on belt conveyors. Multiple objects of one type are in a box or on a conveyor.

 

FUNDING:

Company Funding

  23.10.2019
Sasha

The assistance-robot Sasha, made by Toyota, has the typename HSR, standing for „Human Support Robot“. He is used both in private and industrial surroundings. His purpose is to keep order. Therefore he has to detect and assign objects. This is possible through the wide-angle and stereo cameras. His Laser Range Sensor helps with his orientation. The V4R-Team works on improving the image processing, the object recognition and the grasping for objects. A Robot able to manage similar tasks is called Kenny, developed by the V4R-Team within the SQUIRREL Project.

 

FUNDING:

Company Funding

  23.10.2019
Aeolus

The goal of the project with Aeolus is to advance current robotic capabilities to pick up unknown objects in conditions of clutter and fetching known objects, with emphasis on the diversity of objects addressed, and on the reliability of grasping.  During object pick-up, the robot may not need to maintain a constraint on the pose of grasped objects, while for fetching known objects includes delivering them to a human user while maintaining constraints on object pose.  Part of the project is also to acquire object models through interaction with a person, finding, remembering and updating the location of objects in a home-like environment, and physically executing the fetch and object transfer.

FUNDING:

Company Funding

  09.10.2019
InDex

Robot In-hand Dexterous manipulation by extracting data from human manipulation of objects to improve robotic autonomy and dexterity

The InDex project aims to understand how humans perform in-hand object manipulation and to replicate the observed skilled movements with dexterous artificial hands, merging the concepts of deep reinforcement and transfer learning to generalise in-hand skills for multiple objects and tasks. In addition, an abstraction and representation of previous knowledge will be fundamental for the reproducibility of learned skills to different hardware. Learning will use data across multiple modalities that will be collected, annotated, and assembled into a large dataset. The data and our methods will be shared with the wider research community to allow testing against benchmarks and reproduction of results. The core objectives are: (i) to build a multi-modal artificial perception architecture that extracts data of object manipulation by humans; (ii) the creation of a multimodal dataset of in-hand manipulation tasks such as regrasping, reorienting and finely repositioning; (iii) the development of an advanced object modelling and recognition system, including the characterisation of object affordances and grasping properties, in order to encapsulate both explicit information and possible implicit object usages; (iv) to autonomously learn and precisely imitate human strategies in handling tasks; and (v) to build a bridge between observation and execution, allowing deployment that is independent of the robot.

 

 

FUNDING:

The project is funded by FWF – Austrian Science Foundation & CHIST-ERA.

 

 

 

 

 

 

  09.10.2019
HEAP

Human-Guided Learning and Benchmarking of Robotic Heap Sorting

In HEAP we focus on advancing the state-of-the-art for sorting a heap of unknown, irregular objects and provide appropriate benchmarks. In our scenarios, we deal with unknown, broken or deformed object instances such as concrete, metal pipes and other plastic/metal parts of possibly complex shape. A major goal of this project is to make challenging manipulation tasks easily accessible and reproducible, and to allow for a comparative evaluation of different approaches using a standardized robotic platform and an open source simulation framework. Our intention in providing such a benchmark framework is to (i) evaluate state-of-the-art grasping and manipulation algorithms in these complex heap sorting setups and (ii) to define new challenges in terms of object recognition and manipulation that need to be solved by the community.

FUNDING:

The project is funded by FWF – Austrian Science Foundation & CHIST-ERA.