Stay tuned for future updates!
Camera Perception
Radar Perception
LiDAR Perception
Sensor Fusion with LiDAR & Camera
AutowareAuto High-level Architecture of Perception Component
Camera Perception (Temporary Demo, AutowareClass2020)
Recap: Camera Model
\(D\): aperture diameter
\(f\): focal length
\(N\): aperture f-number (e.g., f/2.8, f/4)
$$D = \frac{f}{N}$$
A larger aperture diameter (smaller N) allows more light to enter the lens, affecting exposure and depth of field (DoF, not FoV directly). A smaller \(N\) (larger aperture) leads to a shallow DoF.
$$FoV = 2arctan(\frac{sensor\_dimension}{2f})$$
A longer focal length reduces the field of view (FoV).Recap: Camera Matrix
\(K\): intrinsic camera parameters
\([R|t]\): extrinsic camera parameters
\(s\): extrinsic camera parameters
\(f_x,f_y\): focal lengths of the camera in the x- and y-direction in pixels
\(c_x,c_y\): principal point coordinates, \(c_x\) and \(c_y\) are close to the center of the image
\(p_i\): 2D image point in homogeneous coordinates \([u, v, 1]^T\)
\(p_w\): 3D image point in the world in homogeneous coordinates \([x, y, z, 1]^T\)
$$P = K[R|t] = \begin{bmatrix} f_x & s & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix}[R|t]$$
$$p_i = \begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = Pp_w = K[R|t]p_w = \begin{bmatrix} f_x & s & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix} \begin{bmatrix} r_{11} & r_{12} & r_{13} & t_x \\ r_{21} & r_{22} & r_{23} & t_y \\ r_{31} & r_{32} & r_{33} & t_z \end{bmatrix} \begin{bmatrix} x \\ y \\ z \\ 1 \end{bmatrix}$$
Recap: Camera Len DistortionRadar Perception (Temporary Demo, AutowareClass2020)
Radar can measure the distance, angle, velocity, and cross-section to/of objects to detect and classify different objects.
LiDAR Perception
Autoware.Auto uses ray-based ground filtering because it is fast and deterministic.
Object Detection & Recognition with LiDAR and Camera on GEM e4
Reference
Back to top of the page
