Goto

Collaborating Authors

 lavender


Diffusion Instruction Tuning

Jin, Chen, Tanno, Ryutaro, Saseendran, Amrutha, Diethe, Tom, Teare, Philip

arXiv.org Artificial Intelligence

We introduce Lavender, a simple supervised fine-tuning (SFT) method that boosts the performance of advanced vision-language models (VLMs) by leveraging state-of-the-art image generation models such as Stable Diffusion. Specifically, Lavender aligns the text-vision attention in the VLM transformer with the equivalent used by Stable Diffusion during SFT, instead of adapting separate encoders. This alignment enriches the model's visual understanding and significantly boosts performance across in- and out-of-distribution tasks. Lavender requires just 0.13 million training examples, 2.5% of typical large-scale SFT datasets, and fine-tunes on standard hardware (8 GPUs) in a single day. It consistently improves state-of-the-art open-source multimodal LLMs (e.g., Llama-3.2-11B, MiniCPM-Llama3-v2.5), achieving up to 30% gains and a 68% boost on challenging out-of-distribution medical QA tasks. By efficiently transferring the visual expertise of image generators with minimal supervision, Lavender offers a scalable solution for more accurate vision-language systems. All code, training data, and models will be shared at https://astrazeneca.github.io/vlm/.


Why AI is facing its 'Oppenheimer moment' - as the world faces killer robot arms race

Daily Mail - Science & tech

Regulators have warned that AI is facing its'Oppenheimer moment,' as the world is facing a killer robot arms race. The statements were made at a conference in Vienna Monday as nations are using AI-powered drones and databases to search out targets on battlefields. The Israeli military was found to use an AI dubbed Lavender to create a kill list and Ukraine is using high-powered drones to unleash endless streams of ammunition. 'This is the Oppenheimer Moment of our generation,' said Austrian Foreign Minister Alexander Schallenberg. 'Now is the time to agree on international rules and norms.'


Inside Israel's Bombing Campaign in Gaza

The New Yorker

Since the war began in Gaza, more than six months ago, the Israeli magazine 972 has published some of the most penetrating reporting on the Israel Defense Forces' conduct. In November, 972, along with the Hebrew publication Local Call, found that the I.D.F. had expanded the number of "legitimate" military targets, leading to a huge increase in civilian casualties. Then earlier this month, 972 and Local Call released a long feature called "Lavender: The AI Machine Directing Israel's Bombing Spree in Gaza." The story revealed how the Israeli military had used the program to identify suspected militants, which in practice meant that tens of thousands of Palestinians had their homes marked as legitimate targets for bombing, with minimal human oversight. The I.D.F. also said that, according to its rules, "analysts must conduct independent examinations" to verify the identification of targets.


White House investigating reports Israel used AI to identify bombing targets in Gaza and create a 'kill list' of 37,000 Palestinians suspected of being militants

Daily Mail - Science & tech

The White House revealed it is looking into reports the Israeli army has been using an AI system to populate its'kill list' of alleged Hamas terrorists, hours after President Joe Biden's call with Benjamin Netanyahu. The report cited six Israeli intelligence officers, who admitted to using an AI called'Lavender' to classify as many as 37,000 Palestinians as suspected militants -- marking these people and their homes as acceptable targets for air strikes. White House national security spokesperson John Kirby told CNN on Thursday that the reports had not been verified, but they were investigating. Israel has vehemently denied the AI's role with an army spokesperson describing the system as'auxiliary tools that assist officers in the process of incrimination.' However, during the call Biden reportedly threatened that he would condition the US' support for the attack in Gaza if the Israeli government didn't protect civilians and aid workers from offensive assaults.


Israeli army used controversial 'Lavender' AI system to create 'kill list' of Palestinian militants and bomb 37,000 targets, report claims

Daily Mail - Science & tech

The Israeli army has been using an AI system to populate its'kill list' of alleged Hamas terrorists, leading to the deaths of women and children, a new report claims. The report cited six Israeli intelligence officers, who admitted to using an AI called'Lavender' to classify as many as 37,000 Palestinians as suspected militants -- marking these people and their homes as acceptable targets for air strikes. Israel has vehemently denied the AI's role with an army spokesperson describing the system as'auxiliary tools that assist officers in the process of incrimination.' Lavender was trained on data from Israeli intelligence's decades-long surveillance of Palestinian populations, using the digital footprints of known militants as a model for what signal to look for in the noise, according to the report. The intel sources noted that human officers scanned each AI-chosen target for about '20 seconds' before giving their'stamp' of approval, despite an internal study that had determined Lavender AI misidentified people 10 percent of the time. Israel quietly delegated the identification of Hamas terrorists, Palestinian civilians and aide workers to an artificial intelligence, 'Lavender,' a new report revealed.


'AI-assisted genocide': Israel reportedly used database for Gaza kill lists

Al Jazeera

The Israeli military's reported use of an untested and undisclosed artificial intelligence-powered database to identify targets for its bombing campaign in Gaza has alarmed human rights and technology experts who said it could amount to "war crimes". The Israeli-Palestinian publication 972 Magazine and Hebrew-language media outlet Local Call reported recently that the Israeli army was isolating and identifying thousands of Palestinians as potential bombing targets using an AI-assisted targeting system called Lavender. "That database is responsible for drawing up kill lists of as many as 37,000 targets," Al Jazeera's Rory Challands, reporting from occupied East Jerusalem, said on Thursday. The unnamed Israeli intelligence officials who spoke to the media outlets said Lavender had an error rate of about 10 percent. "But that didn't stop the Israelis from using it to fast-track the identification of often low-level Hamas operatives in Gaza and bombing them," Challands said.


'The machine did it coldly': Israel used AI to identify 37,000 Hamas targets

The Guardian

The Israeli military's bombing campaign in Gaza used a previously undisclosed AI-powered database that at one stage identified 37,000 potential targets based on their apparent links to Hamas, according to intelligence sources involved in the war. In addition to talking about their use of the AI system, called Lavender, the intelligence sources claim that Israeli military officials permitted large numbers of Palestinian civilians to be killed, particularly during the early weeks and months of the conflict. Their unusually candid testimony provides a rare glimpse into the first-hand experiences of Israeli intelligence officials who have been using machine-learning systems to help identify targets during the six-month war. Israel's use of powerful AI systems in its war on Hamas has entered uncharted territory for advanced warfare, raising a host of legal and moral questions, and transforming the relationship between military personnel and machines. "This is unparalleled, in my memory," said one intelligence officer who used Lavender, adding that they had more faith in a "statistical mechanism" than a grieving soldier.


Vec2Vec: A Compact Neural Network Approach for Transforming Text Embeddings with High Fidelity

Gao, Andrew Kean

arXiv.org Artificial Intelligence

Vector embeddings have become ubiquitous tools for many language-related tasks. A leading embedding model is OpenAI's text-ada-002 which can embed approximately 6,000 words into a 1,536-dimensional vector. While powerful, text-ada-002 is not open source and is only available via API. We trained a simple neural network to convert open-source 768-dimensional MPNet embeddings into text-ada-002 embeddings. We compiled a subset of 50,000 online food reviews. We calculated MPNet and text-ada-002 embeddings for each review and trained a simple neural network to for 75 epochs. The neural network was designed to predict the corresponding text-ada-002 embedding for a given MPNET embedding. Our model achieved an average cosine similarity of 0.932 on 10,000 unseen reviews in our held-out test dataset. We manually assessed the quality of our predicted embeddings for vector search over text-ada-002-embedded reviews. While not as good as real text-ada-002 embeddings, predicted embeddings were able to retrieve highly relevant reviews. Our final model, Vec2Vec, is lightweight (<80 MB) and fast. Future steps include training a neural network with a more sophisticated architecture and a larger dataset of paired embeddings to achieve greater performance. The ability to convert between and align embedding spaces may be helpful for interoperability, limiting dependence on proprietary models, protecting data privacy, reducing costs, and offline operations.


SpiroBot : Medicinal Plant Recognition

#artificialintelligence

A simple identification approach to obtain all the health benefits your body needs! Medicinal plants contain natural compounds that can have therapeutic effects on the human body. These compounds, such as alkaloids, flavonoids, and terpenes, have been used for centuries to treat various ailments and have been shown to have anti-inflammatory, antioxidant, and antimicrobial properties. The use of medicinal plants can be a natural alternative to conventional pharmaceuticals and can have fewer side effects. However, it's important to note that the use of medicinal plants should be done under the guidance of a healthcare professional to ensure their safety and effectiveness.


On CRM: Chatbots Are Becoming A $10 Billion Market

#artificialintelligence

My dog died a few years ago. She was only about 7 years old and had a lot of health problems. But one thing I don't miss: my pharmacy bill. That's because Lavender (my wife chose the name, not me) was on no less than seven medications. And a day rarely went by without me a text message from my pharmacist at the CVS near me.