This study focuses on the critical aspect of robust state estimation for the safe navigation of an Autonomous Vehicle (AV). Existing literature primarily employs two prevalent techniques for state estimation, namely filtering-based and graph-based approaches. Factor Graph (FG) is a graph-based approach, constructed using Values and Factors for Maximum Aposteriori (MAP) estimation, that offers a modular architecture that facilitates the integration of inputs from diverse sensors. However, most FG-based architectures in current use require explicit knowledge of sensor parameters and are designed for single setups. To address these limitations, this research introduces a novel plug-and-play FG-based state estimator capable of operating without predefined sensor parameters. This estimator is suitable for deployment in multiple sensor setups, offering convenience and providing comprehensive state estimation at a high frequency, including mean and covariances. The proposed algorithm undergoes rigorous validation using various sensor setups on two different vehicles: a quadricycle and a shuttle bus. The algorithm provides accurate and robust state estimation across diverse scenarios, even when faced with degraded Global Navigation Satellite System (GNSS) measurements or complete outages. These findings highlight the efficacy and reliability of the algorithm in real-world AV applications.

Vehicle teleoperation has numerous potential applications, including serving as a backup solution for autonomous vehicles, facilitating remote delivery services, and enabling hazardous remote operations. However, complex urban scenarios, limited situational awareness, and network delay increase the cognitive workload of human operators and degrade teleoperation performance. To address this, the successive reference pose tracking (SRPT) approach was introduced in earlier work, which transmits successive reference poses to the remote vehicle instead of steering commands. The operator generates reference poses online with the help of a joystick steering and an augmented display, potentially mitigating the detrimental effects of delays. However, it is not clear which minimal set of sensors is essential for the SRPT vehicle teleoperation control loop. This paper tests the robustness of the SRPT approach in the presence of state estimation inaccuracies, environmental disturbances, and measurement noises. The simulation environment, implemented in Simulink, features a 14-dof vehicle model and incorporates difficult maneuvers such as tight corners, double-lane changes, and slalom. Environmental disturbances include low adhesion track regions and strong cross-wind gusts. The results demonstrate that the SRPT approach, using either estimated or actual states, performs similarly under various worst-case scenarios, even without a position sensor requirement. Additionally, the designed state estimator ensures sufficient performance with just an inertial measurement unit, wheel speed encoder, and steer encoder, constituting a minimal set of essential sensors for the SRPT vehicle teleoperation control loop.

Vehicle teleoperation has potential applications in fallback solutions for autonomous vehicles, remote delivery services, and hazardous operations. However, network delays and limited situational awareness can compromise teleoperation performance and increase the cognitive workload of human operators. To address these issues, we previously introduced the novel successive reference pose tracking (SRPT) approach, which transmits successive reference poses to the vehicle instead of steering commands. This paper compares the stability and performance of SRPT with Smith predictor-based approaches for direct vehicle teleoperation in challenging scenarios. The Smith predictor approach is further categorized, one with Lookahead driver and second with Stanley driver. Simulations are conducted in a Simulink environment, considering variable network delays and different vehicle speeds, and include maneuvers such as tight corners, slalom, low-adhesion roads, and strong crosswinds. The results show that the SRPT approach significantly improves stability and reference tracking performance, with negligible effect of network delays on path tracking. Our findings demonstrate the effectiveness of SRPT in eliminating the detrimental effect of network delays in vehicle teleoperation.

Teleoperation provides human operator sophisticated perceptual and cognitive skills into an over the network control loop. It gives hope of addressing some challenges related to vehicular autonomy which is based on artificial intelligence by providing a backup plan. Variable network time delays in data transmission is the major problem in teleoperating a vehicle. On 4G network, variability of these delays is high. Due to this, both video streaming and driving commands encounter variable time delay. This paper presents an approach of providing the human operator a forecast video stream which replicates future perspective of vehicle field of view accounting the delay present in the network. Regarding the image transformation, perspective projection technique is combined with correction given by smith predictor in the control loop. This image transformation accounts current time delay and tries to address both issues, time delays as well as its variability. For experiment sake, only frontward field of view is forecast. Performance is evaluated by performing online vehicle teleoperation on street edge case maneuvers and later comparing the path deviation with and without perspective projection.