In this paper, we explore a novel federated multimodal instruction tuning task(FedMIT), which is significant for collaboratively fine-tuning MLLMs on different types of multimodal instruction data on distributed devices. To solve the new task, we propose a federated multimodal instruction tuning framework(Pilot). Our framework integrates two stages of "adapter on adapter" into the connector of the vision encoder and the LLM. In stage 1, we extract task-specific features and client-specific features from visual information. In stage 2, we build the cross-task Mixture-of-Adapters(CT-MoA) module to perform cross-task interaction. Each client can not only capture personalized information of local data and learn task-related multimodal information, but also learn general knowledge from other tasks. In addition, we introduce an adaptive parameter aggregation strategy for text training parameters, which optimizes parameter aggregation by calculating weights based on the euclidean distance between parameters, so that parameter aggregation can benefit from positive effects to the greatest extent while effectively reducing negative effects. Our framework can collaboratively exploit distributed data from different local clients to learn cross-task knowledge without being affected by the task heterogeneity during instruction tuning. The effectiveness of our method is verified in two different cross-task scenarios.

This post is brought to you by NVIDIA and CIO. The views and opinions expressed herein are those of the author and do not necessarily represent the views and opinions of NVIDIA. CIOs seeking big wins in high business-impacting areas where there's significant room to improve performance should review their data science, machine learning (ML), and AI projects. A recent IDC report on AI projects in India[1] reported that 30-49% of AI projects failed for about one-third of organizations, and another study from Deloitte casts 50% of respondents' organizational performance in AI as starters or underachievers. That same study found 94% of respondents say AI is critical to success over the next five years. Executives see the AI opportunity for competitive differentiation and are looking for leaders to deliver successful outcomes.

CIOs seeking big wins in high business-impacting areas where there's significant room to improve performance should review their data science, machine learning (ML), and AI projects. A recent IDC report on AI projects in India[1] reported that 30-49% of AI projects failed for about one-third of organizations, and another study from Deloitte casts 50% of respondents' organizational performance in AI as starters or underachievers. That same study found 94% of respondents say AI is critical to success over the next five years. Executives see the AI opportunity for competitive differentiation and are looking for leaders to deliver successful outcomes. ML and AI are still relatively new practice areas, and leaders should expect ongoing learning and an improving maturity curve.

The U.S. Navy and U.S. Air Force are working on a new generation of training technologies to prepare their fighter aircraft for new Russian and Chinese air threats posed by the Su-57 fighter and J-20 fifth-generation stealth aircraft, respectively. Over the next two years, the U.S. Air Force plans to use a cutting-edge computer technology called the P5 Combat Training System (P5CTS), made by a firm called Cubic Mission and Performance Solutions. Information from Cubic describes the P5 as an encryption solution intended to improve U.S. Air Force and U.S. Navy pilot training for advanced, high-threat combat scenarios using advanced computer simulations, wireless networks, and artificial intelligence (AI)-enabled data organization. Interestingly, the P5 pod can be seen in the now-famous Top Gun Maverick movie on a F/A-18 fighter. "Over the course of the last 13 years, we've learned some critical lessons about integrating fast movers with virtual environments to provide a realistic presentation to the aircrew in their cockpits. Having a wireless network that allows you to sustain that environment without interruptions. In other words, a low, flat latency is a very important feature," said Cubic's training expert, Paul Averna.

We benchmark the current existing methods to jointly learn non-Cartesian k-space trajectory and reconstruction: PILOT, BJORK, and compare them with those obtained from the recently developed generalized hybrid learning (HybLearn) framework. We present the advantages of using projected gradient descent to enforce MR scanner hardware constraints as compared to using added penalties in the cost function. Further, we use the novel HybLearn scheme to jointly learn and compare our results through a retrospective study on fastMRI validation dataset.

Even during quarantine, there was a lot of back and forth and "fact" checking with the Lucasfilm brand team. Frazier said Lucasfilm is practically a co-developer, at least in terms of the amount of input and assets they provided for art and audio files. Last year's "Jedi: Fallen Order" by Respawn Entertainment and EA was also similarly collaborative and canonical, taking place just after the prequel trilogy. Frazier said he's a huge fan of the "Star Wars" animated series "Rebels," and was eager to tackle his first project for the brand.

The government plans to establish a licensing system for operating drones when the flights are beyond the operator's line of sight, government sources said Monday. The proposal comes as the government hopes for increased usage of unmanned vehicles for purposes such as delivering daily necessities and medicine, or assisting security patrols in areas with an aging population, the sources said. The license, which the government hopes will be introduced in fiscal 2022, will be age-restricted, and will require operators to pass both a written and practical examination. The licenses will be only valid for a certain period of time and will have to be renewed. Illegal drone use will lead to the cancellation or suspension of a license.

Last year, after breaking the Guinness World Record for the Key West to Cuba run, we wondered what was next for the #77 Lucas Oil SilverHook ocean racing powerboat? We found the answer in the 50th anniversary of the Trinidad & Tobago Great Race, one of the most grueling races in the world. The 115-mile endurance course starts in Trinidad's Port of Spain, where you head north and then east near the island before popping into the Atlantic Ocean for a 50-mile sprint to the finish in Store Bay, Tobago. Because of the logistical difficulties of racing on foreign shores, we were the first American entry in 29 years. We knew we would face stiff competition from Jumbie, Cat Killer, Mr. Solo and other local rivals that know the course well.

Automakers are moving at a frenzied pace to add more and more intelligence to their vehicles. However, as cars get smarter and smarter consumers struggle to understand the latest innovations. To help eliminate some of the confusion, the Society of Automotive Engineers (SAE) International has developed a scale to describe the six different levels of automation for self-driving cars. These are the cars we all know and love. The driver actually steps on the gas to go faster, steps on the brake to slow down and uses the steering wheel to turn. Even if the driver is assisted by warning signals, this still is generally viewed as Level 0. Example: Your parent's station wagon For Level 1, the driver is still in control of the overall operation and safety of the vehicle.

STEPPING OFF THE PLANE in Russia for the first time in 2013, I collided with a wall of blunt language and was intrigued beyond repair. Five years, countless classes and ten visits to Moscow later, I still claim a distinctly below-average capacity for the Russian tongue and its dense, foreboding components. To fill these gaps ahead of my next adventure abroad, I turned to technology. Late last year, Brooklyn's Waverly Labs released the Pilot ($299, waverlylabs.com), These eavesdropping devices use a cloud-based machine learning technology to pipe dozens of different languages into your brain in your mother tongue.