High resolution long range Lidar for autonomous driving (LiDcAR)

Project focus

  • Evaluation and control of scanning systems for Lidar applications
  • Precision metrology for MEMS scanning mirrors
  • Modeling and identification of Complex dynamics of MEMS scanning mirror
  • Advanced sensing and control for MEMS scanning mirror
  • Performance assessments of the MEMS scanning systems for automotive lidar

Description

Lidar is an acronym for light detection and ranging, in analogy to radar. Lidar has received much attention in the automotive industry as a key component for high level automated driving systems. Compared to other sensing techniques such as stereo cameras and radar, lidar can provide high resolution and highly accurate 3D measurements of the surroundings and robust detection in various weather conditions. The lidar sensors are expected to ensure the safety in automated driving applications such as collision detection, blind spot monitoring, object and pedestrian recognition, and terrain mapping. Currently Google, BMW, Ford, Volvo and other autonomous car developers include lidar sensors in their systems for safe driving.

(Left) Line scan lidar based on a 1D resonant MEMS mirror and a detector array. (Right) Example of a 1D resonant MEMS mirror with electrostatic comb-drive actuation.

The main challenges of this project are in the development and evaluation of the scanning system in the lidar sensor. Currently a polygon scanner, MEMS mirror and optical phase arrays are studied for the potential scanning system. The scanning system for the lidar for autonomous driving requires extreme robustness in harsh environmental conditions, i.e. large temperature range, humidity, dust, shock, and vibration. The scanning system in lidar should provide seamless operation to ensure the safety of the drivers and pedestrians. In addition, the performance of the lidar sensor is defined and measured precisely to ensure the safety requirements with sufficient accuracy.

This project aimed to develop precise and robust scanning systems for lidars. The research includes investigation of the various scanning techniques as well as the methodology to evaluate the performance of the lidar and its scanning system. The developed lidar and evaluation platform was mounted on a vehicle and tested in the field.

Precision metrology of MEMS scanning mirrors

For accurate evaluation of MEMS scanning mirrors, systematic errors of the measurement setup have to be identified and avoided. Hence, a self-calibrated test bench is developed and analyzed regarding its uncertainties of MEMS mirror angle detection. The basic principle is that a laser beam gets deflected by the MEMS mirror and is detected by a position sensitive detector (PSD), where the mirror angle can be recalculated. As alignment uncertainties such as an inaccurate placement of the MEMS mirror in its package cause estimation errors, a semi-automated calibration procedure is developed based on a CCD camera and a motorized stage. The achieved accuracy error of 0.025° at 15° mechanical angle suffices the requirements for automotive Lidar. As a crucial component of the test bench, the PSDs are analyzed regarding the achievable bandwidth while also temperature drifts as well as their compensation schemes are investigated.

(Left) MEMS test bench with semi-automated calibration and alignment capabilities. (Right) Uncertainty analysis of MEMS mirror angle detection and component-wise contribution.

Accurate modeling of the MEMS scanning mirror

Accurate modelling and identification of resonant MEMS mirrors dynamics is crucial for control design and allows to identify weaknesses and strengths of the individual MEMS mirror designs. In the presence of nonlinearities, conventional linear system theory fails to predict the behavior of the system as it gets state dependent. In particular, MEMS mirrors exhibit several nonlinearities, such as amplitude dependent damping, angle dependent stiffness as well as actuation forces, which depend on the angular comb-drive capacitance derivative and the driving voltage squared. Hence, an identification method for a general single degree of freedom model was developed to estimate the system parameters solely by measurement data with a minimum on prior assumptions. For advanced control design, a linearized model based on period-to-period energy conservation was developed, which represents the local MEMS mirror dynamics of phase, frequency and amplitude at a nominal operation point. As an extension, slow flow analysis was applied to the identified nonlinear model, which allows a fast calculation of the global and local behavior of the MEMS mirror.

(Left) Comparison of measured and simulated frequency response. (Right) Comparison of measured and simulated frequency response of the local dynamics at the nominal operation point.

Advanced sensing and control for MEMS mirrors

A precise sensing of the MEMS mirror movement is crucial for robust feedback control. As the size of the MEMS mirror is in the range of view millimeters, dedicated sensors can hardly be applied without complicating the design or increasing production cost. Hence, self-sensing approaches are developed, where the same comb-drives serve as actuators and sensors. The comb-drive capacitance of resonant MEMS mirrors usually has a triangular shape with its maximum at zero angle and deceases in both angular directions in a deterministic manner. Therefore, the measurement of the capacitance directly reveals the mirror angle. For square wave excitation of the MEMS mirror, the charge on the comb-drive capacitance is direct proportional to the capacitance and therefore serves as an angle sensor. However, the measurement time is limited to the time where the driving voltage is on, which lead to the development of a switched input observer to estimate the MEMS mirror angle also between the measurement windows, i.e. where the driving voltage is off.

(Left) Switched input observer scheme for self-sensing mirror trajectory estimation. (Right) Convergence of the switched input observer and comparison with the true angle measurement (PSD). The observer obtains the sense signal only, while the PSD is only for evaluation.

A precise zero crossing measurement is obtained by the displacement current flowing through the comb-drives during a constant driving voltage as depicted in Fig. 4. The current signal shows a sharp zero crossing when the mirror passes zero angle, which can be precisely detected by a simple comparator.

(Left) Example of trajectory, driving signal and current signal. (Right) Scheme of the DAsPLL.

For performance evaluation of the phase detection, the Digital Asynchronous PLL (DAsPLL) was developed, which operates the MEMS mirror with immediate phase error compensation and showed a center pixel uncertainty error of less than 0.3 mdeg at 58° field of view. Key feature of the DAsPLL is the asynchronous switching off of the driving voltage by bypassing the zero crossing comparator detection signal without clock speed limitations of the FPGA.

Also a conventional PLL structure was analyzed, where the driving frequency is adjusted according to the measured phase errors with an appropriate control law. The derived linear models of the MEMS mirror allowed a simple and robust control design and revealed that due to the nonlinear stiffness of the MEMS mirror, a proportional gain is crucial as only integral control leads to large settling times of the system or even instability.

(Left) Conventional PLL structure. (Right) Root locus of closed loop system model and dependency on proportional and integral gain of the PLL controller.

Applications

  • Lidar for autonomous driving
  • Precision measurement system

Related Publications and Patents

Journal articles

  • D. Brunner, H. W. Yoo, and G. Schitter, Linear Modeling and Control of Comb-Actuated Resonant MEMS Mirror with Nonlinear Dynamics, IEEE Transactions on Industrial Electronics, vol. 68, iss. 4, p. 3315–3323, 2021.
    [BibTex] [Download]
    @article{TUW-288425,
    author = {Brunner, David and Yoo, Han Woong and Schitter, Georg},
    title = {Linear Modeling and Control of Comb-Actuated Resonant MEMS Mirror with Nonlinear Dynamics},
    journal = {IEEE Transactions on Industrial Electronics},
    year = {2021},
    volume = {68},
    number = {4},
    pages = {3315--3323},
    doi = {10.1109/TIE.2020.2982124},
    keywords = {Energy conservation, laser radar, linearization techniques, microelectromechanical system (MEMS), micromirrors, nonlinear systems, parameter estimation, phase locked loops (PLL), system identification.}
    }
  • H. W. Yoo, S. Albert, and G. Schitter, Accurate Analytic Model of a Parametrically Driven Resonant MEMS Mirror with a Fourier Series Based Torque Approximation, Journal of Microelectromechanical Systems, vol. 29, iss. 6, p. 1431–1442, 2020.
    [BibTex] [Download]
    @article{TUW-290470,
    author = {Yoo, Han Woong and Albert, Stephan and Schitter, Georg},
    title = {Accurate Analytic Model of a Parametrically Driven Resonant MEMS Mirror with a Fourier Series Based Torque Approximation},
    journal = {Journal of Microelectromechanical Systems},
    year = {2020},
    volume = {29},
    number = {6},
    pages = {1431--1442},
    doi = {10.1109/JMEMS.2020.3024752},
    keywords = {Mirrors , Micromechanical devices , Torque , Analytical models , Mathematical model , Capacitance , Fourier series}
    }
  • D. Brunner, H. W. Yoo, and G. Schitter, Precise phase control of resonant MOEMS mirrors by comb-drive current feedback, Mechatronics, vol. 71, p. 102420, 2020.
    [BibTex] [Download]
    @article{TUW-290167,
    author = {Brunner, David and Yoo, Han Woong and Schitter, Georg},
    title = {Precise phase control of resonant MOEMS mirrors by comb-drive current feedback},
    journal = {Mechatronics},
    year = {2020},
    volume = {71},
    pages = {102420},
    doi = {10.1016/j.mechatronics.2020.102420},
    keywords = {Comb-drive, Digital asynchronous phase locked loop (DAsPLL), Micro-opto-electro-mechanical system (MOEMS), Nonlinear systems, Precision control, Resonant scanning mirror}
    }
  • H. W. Yoo, N. Druml, D. Brunner, C. Schwärzl, T. Thurner, M. Hennecke, and G. Schitter, MEMS-based lidar for autonomous driving, E&I Elektrotechnik und Informationstechnik, vol. 135, iss. 6, p. 408–415, 2018.
    [BibTex] [Download]
    @Article{TUW-273071,
    author = {Yoo, Han Woong and Druml, N. and Brunner, David and Schw{\"a}rzl, Christian and Thurner, Thomas and Hennecke, Markus and Schitter, Georg},
    title = {MEMS-based lidar for autonomous driving},
    journal = {E{\&}I Elektrotechnik und Informationstechnik},
    year = {2018},
    volume = {135},
    number = {6},
    pages = {408--415},
    keywords = {lidar; MEMS scanning mirror; autonomous driving; metrology platform},
    doi = {10.1007/s00502-018-0635-2},
    }

Conference papers

  • R. Schroedter, H. W. Yoo, D. Brunner, and G. Schitter, Capacitive Charge-based Self-Sensing for Resonant Electrostatic MEMS mirrors, in Proceedings of the 21st IFAC World Congress, 2020.
    [BibTex] [Download]
    @inproceedings{TUW-292461,
    author = {Schroedter, Richard and Yoo, Han Woong and Brunner, David and Schitter, Georg},
    title = {Capacitive Charge-based Self-Sensing for Resonant Electrostatic MEMS mirrors},
    booktitle = {Proceedings of the 21st IFAC World Congress},
    year = {2020},
    numpages = {6},
    keywords = {Resonant MEMS mirror, electrostatic comb drive, capacitive charge sensing, nonlinear observer with switched input},
    note = {Vortrag: 21st IFAC World Congress, Berlin; 2020-07-12 -- 2020-07-17}
    }
  • H. W. Yoo and G. Schitter, Complex Valued State Space Model for Weakly Nonlinear Duffing Oscillator with Noncollocated External Disturbance, in Proceedings of the 21st IFAC World Congress, 2020.
    [BibTex] [Download]
    @inproceedings{TUW-290466,
    author = {Yoo, Han Woong and Schitter, Georg},
    title = {Complex Valued State Space Model for Weakly Nonlinear Duffing Oscillator with Noncollocated External Disturbance},
    booktitle = {Proceedings of the 21st IFAC World Congress},
    year = {2020},
    numpages = {7},
    keywords = {Duffing oscillator, complex valued model, perturbation theory, disturbance model, noncollocated disturbance, MEMS mirror, automotive lidar},
    note = {Vortrag: 21st IFAC World Congress, Berlin; 2020-07-12 -- 2020-07-17}
    }
  • D. Brunner, H. W. Yoo, and G. Schitter, Digital Asynchronous Phase Locked Loop for Precision Control of MOEMS Scanning Mirror, in Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), 2019.
    [BibTex] [Download]
    @InProceedings{TUW-282514,
    author = {Brunner, David and Yoo, Han Woong and Schitter, Georg},
    title = {Digital Asynchronous Phase Locked Loop for Precision Control of MOEMS Scanning Mirror},
    booktitle = {Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019)},
    year = {2019},
    volume = {52/15},
    note = {Vortrag: Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), Wien; 2019-09-04 -- 2019-09-06},
    doi = {10.1016/j.ifacol.2019.11.647},
    journal = {IFAC-PapersOnLine/Elsevier},
    keywords = {Digital asynchronous phase locked loop (DAsPLL), Micro-Opto-Electro-Mechanical System (MOEMS), Resonant scanning mirror, Comb-drive, Nonlinear systems.},
    numpages = {6},
    }
  • H. W. Yoo, D. Brunner, T. Thurner, and G. Schitter, MEMS Test Bench and its Uncertainty Analysis for Evaluation of MEMS Mirrors, in Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), 2019.
    [BibTex] [Download]
    @InProceedings{TUW-283447,
    author = {Yoo, Han Woong and Brunner, David and Thurner, Thomas and Schitter, Georg},
    title = {MEMS Test Bench and its Uncertainty Analysis for Evaluation of MEMS Mirrors},
    booktitle = {Proceedings of the Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019)},
    year = {2019},
    volume = {52/15},
    note = {Vortrag: Joint Conference 8th IFAC Symposium on Mechatronic Systems (MECHATRONICS 2019), and 11th IFAC Symposium on Nonlinear Control Systems (NOLCOS 2019), Wien; 2019-09-04 -- 2019-09-06},
    doi = {10.1016/j.ifacol.2019.11.648},
    journal = {IFAC-PapersOnLine/Elsevier},
    keywords = {Metrology, Uncertainty analysis, Characterization, Microelectromechanical systems (MEMS), MEMS mirror},
    numpages = {6},
    }
  • D. Brunner, H. W. Yoo, T. Thurner, and G. Schitter, Data based Modelling and Identification of Nonlinear SDOF MOEMS Mirror, in MOEMS and Miniaturized Systems XVIII, 2019.
    [BibTex] [Download]
    @InProceedings{TUW-282507,
    author = {Brunner, David and Yoo, Han Woong and Thurner, Thomas and Schitter, Georg},
    title = {Data based Modelling and Identification of Nonlinear SDOF MOEMS Mirror},
    booktitle = {MOEMS and Miniaturized Systems XVIII},
    year = {2019},
    volume = {10931},
    note = {Vortrag: SPIE Photonics West 2019, San Francisco, California, USA; 2019-02-02 -- 2019-02-07},
    doi = {10.1117/12.2508429},
    journal = {SPIE},
    keywords = {Micro-Opto-Electro-Mechanical System (MOEMS), Resonant scanning mirror, Data based identification, Generalized nonlinear SDOF model, Actuated decay, Comb-drive torque},
    numpages = {11},
    }
  • H. W. Yoo, D. Brunner, T. Thurner, and G. Schitter, Compensation for Temperature Dependency of 1D Position Sensitive Detector, in Advances in Display Technologies IX, 10942, 2019.
    [BibTex] [Download]
    @InProceedings{TUW-283433,
    author = {Yoo, Han Woong and Brunner, David and Thurner, Thomas and Schitter, Georg},
    title = {Compensation for Temperature Dependency of 1D Position Sensitive Detector},
    booktitle = {Advances in Display Technologies IX},
    year = {2019},
    address = {10942},
    publisher = {SPIE},
    note = {Vortrag: SPIE Photonics West 2019, San Francisco, California, USA; 2019-02-02 -- 2019-02-07},
    doi = {10.1117/12.2508403},
    keywords = {Position sensitive detector (PSD), High temperature operation, Leakage current, Point drift measurement, Reliability test}
    }

Patents

  • H. W. Yoo, M. E. Hennecke, G. Schitter, and T. Thurner, Real time gating and signal routing in laser and detector arrays for lidar applications, U.S. Patent, iss. US 2020/0200877 A1, 2020.
    [BibTex]
    @patent{TUW-292557,
    author = {Yoo, Han Woong and Hennecke, Markus Edward and Schitter, Georg and Thurner, Thomas},
    title = {Real time gating and signal routing in laser and detector arrays for lidar applications},
    journal = {U.S. Patent},
    number = {US 2020/0200877 A1},
    year = {2020},
    note = {eingereicht: 2019-02-01, erteilt: 2020-06-25}
    }
  • H. W. Yoo, M. E. Hennecke, G. Schitter, and T. Thurner, Echtzeit-gating und Signal Wegleitung in Laser- und Detektorarrays für LiDAR-Anwendungen, Deutsches Patent- und Markenamt, iss. DE 10 2019 220 289 A1, 2020.
    [BibTex]
    @patent{TUW-292552,
    author = {Yoo, Han Woong and Hennecke, Markus Edward and Schitter, Georg and Thurner, Thomas},
    title = {Echtzeit-gating und Signal Wegleitung in Laser- und Detektorarrays f{\"u}r LiDAR-Anwendungen},
    journal = {Deutsches Patent- und Markenamt},
    number = {DE 10 2019 220 289 A1},
    year = {2020},
    note = {eingereicht: 2019-12-19, erteilt: 2020-06-25}
    }
  • T. Thurner, D. Brunner, M. E. Hennecke, G. Schitter, and H. W. Yoo, LiDAR-Sensoren und Verfahren für LiDAR-Sensoren, Deutsches Patent- und Markenamt, iss. DE 10 2018 128 164 A1, 2020.
    [BibTex]
    @patent{TUW-292555,
    author = {Thurner, Thomas and Brunner, David and Hennecke, Markus Edward and Schitter, Georg and Yoo, Han Woong},
    title = {LiDAR-Sensoren und Verfahren f{\"u}r LiDAR-Sensoren},
    journal = {Deutsches Patent- und Markenamt},
    number = {DE 10 2018 128 164 A1},
    year = {2020},
    howpublished = {Patent},
    note = {eingereicht: 2018-11-12, erteilt: 2020-05-14}
    }
  • T. Thurner, D. Brunner, M. E. Hennecke, G. Schitter, and H. W. Yoo, Lidar sensors and methods for lidar sensors, U.S. Patent, iss. US 2020/0150246 A1, 2020.
    [BibTex]
    @patent{TUW-292556,
    author = {Thurner, Thomas and Brunner, David and Hennecke, Markus Edward and Schitter, Georg and Yoo, Han Woong},
    title = {Lidar sensors and methods for lidar sensors},
    journal = {U.S. Patent},
    number = {US 2020/0150246 A1},
    year = {2020},
    howpublished = {Patent},
    note = {eingereicht: 2019-11-11, erteilt: 2020-05-14}
    }

Press Release

Project partners

Funding