ESVO2: Direct Visual-Inertial Odometry with Stereo Event Cameras