We propose the joint speech translation and recognition (JSTAR) model that leverages the fast-slow cascaded encoder architecture for simultaneous end-to-end automatic speech recognition (ASR) and speech translation (ST). The model is transducer-based and uses a multi-objective training strategy that optimizes both ASR and ST objectives simultaneously. This allows JSTAR to produce high-quality streaming ASR and ST results. We apply JSTAR in a bilingual conversational speech setting with smart-glasses, where the model is also trained to distinguish speech from different directions corresponding to the wearer and a conversational partner. Different model pre-training strategies are studied to further improve results, including training of a transducer-based streaming machine translation (MT) model for the first time and applying it for parameter initialization of JSTAR. We demonstrate superior performances of JSTAR compared to a strong cascaded ST model in both BLEU scores and latency.

The E-8C Joint Surveillance Target Attack Radar System is a joint Air Force - Army program. The Joint STARS uses a multi-mode side looking radar to detect, track, and classify moving ground vehicles in all conditions deep behind enemy lines. The Air Force is massively speeding up a new networked surveillance system intended to collect, organize and disseminate pressing attack information in extremely high-risk environments including enemy stealth fighters, advanced air defenses and armed drones. "We do not want to recap JSTARS but create that same capability that protects soldiers and marines on the move. We want to replicate the technology, yet make it survivable," William Roper, Assistant Secretary of the Air Force, Acquisition, Technology & Logistics, told an audience recently at an Air Force Association Symposium.