Large language models (LLMs) have recently shown significant potential in various biological tasks such as protein engineering and molecule design. These tasks typically involve black-box discrete sequence optimization, where the challenge lies in generating sequences that are not only biologically feasible but also adhere to hard fine-grained constraints. However, LLMs often struggle with such constraints, especially in biological contexts where verifying candidate solutions is costly and time-consuming. In this study, we explore the possibility of employing LLMs as highly-constrained bilevel optimizers through a methodology we refer to as Language Model Optimization with Margin Expectation (LLOME). This approach combines both offline and online optimization, utilizing limited oracle evaluations to iteratively enhance the sequences generated by the LLM. We additionally propose a novel training objective - Margin-Aligned Expectation (MargE) - that trains the LLM to smoothly interpolate between the reward and reference distributions. Lastly, we introduce a synthetic test suite that bears strong geometric similarity to real biophysical problems and enables rapid evaluation of LLM optimizers without time-consuming lab validation. Our findings reveal that, in comparison to genetic algorithm baselines, LLMs achieve significantly lower regret solutions while requiring fewer test function evaluations. However, we also observe that LLMs exhibit moderate miscalibration, are susceptible to generator collapse, and have difficulty finding the optimal solution when no explicit ground truth rewards are available. Large language models (LLMs) have recently shown significant promise on various biophysical optimization tasks, such as protein engineering and molecule design. These tasks are often formulated as black-box discrete sequence optimization problems, wherein a solver must attempt to output a discrete sequence x X that is feasible (i.e., a biologically plausible sequence) and that fulfills a number of strict constraints, such as containing specific motifs. Yet despite their many successes, LLMs often struggle to generate outputs that fulfill hard fine-grained constraints [31].

Federated Learning (FL) is an emerging paradigm that enables intelligent agents to collaboratively train Machine Learning (ML) models in a distributed manner, eliminating the need for sharing their local data. The recent work (arXiv:2106.02969) introduces a family of Federated Newton Learn (FedNL) algorithms, marking a significant step towards applying second-order methods to FL and large-scale optimization. However, the reference FedNL prototype exhibits three serious practical drawbacks: (i) It requires 4.8 hours to launch a single experiment in a sever-grade workstation; (ii) The prototype only simulates multi-node setting; (iii) Prototype integration into resource-constrained applications is challenging. To bridge the gap between theory and practice, we present a self-contained implementation of FedNL, FedNL-LS, FedNL-PP for single-node and multi-node settings. Our work resolves the aforementioned issues and reduces the wall clock time by x1000. With this FedNL outperforms alternatives for training logistic regression in a single-node -- CVXPY (arXiv:1603.00943), and in a multi-node -- Apache Spark (arXiv:1505.06807), Ray/Scikit-Learn (arXiv:1712.05889). Finally, we propose two practical-orientated compressors for FedNL - adaptive TopLEK and cache-aware RandSeqK, which fulfill the theory of FedNL.

In this paper, we investigate Extractive Question Answering (EQA) with Large Language Models (LLMs) under domain drift, i.e., can LLMs generalize to domains that require specific knowledge such as medicine and law in a zero-shot fashion without additional in-domain training? To this end, we devise a series of experiments to explain the performance gap empirically. Our findings suggest that: (a) LLMs struggle with dataset demands of closed domains such as retrieving long answer spans; (b) Certain LLMs, despite showing strong overall performance, display weaknesses in meeting basic requirements as discriminating between domain-specific senses of words which we link to pre-processing decisions; (c) Scaling model parameters is not always effective for cross domain generalization; and (d) Closed-domain datasets are quantitatively much different than open-domain EQA datasets and current LLMs struggle to deal with them. Our findings point out important directions for improving existing LLMs.

We consider the Gaussian kernel density estimator with bandwidth $\beta^{-\frac12}$ of $n$ iid Gaussian samples. Using the Kac-Rice formula and an Edgeworth expansion, we prove that the expected number of modes on the real line scales as $\Theta(\sqrt{\beta\log\beta})$ as $\beta,n\to\infty$ provided $n^c\lesssim \beta\lesssim n^{2-c}$ for some constant $c>0$. An impetus behind this investigation is to determine the number of clusters to which Transformers are drawn in a metastable state.

Rep. Chris Smith, R-N.J., opens up about the aerial systems spotted in the Garden State on'The Story.' A U.S. Coast Guard official said one of its vessels was trailed by dozens of drones off the coast of New Jersey recently, according to Rep. Chris Smith, R-N.J. Smith, a guest on "The Story with Martha MacCallum" Tuesday, said he spent Monday night on the beach in Ocean County and spoke to several people, including a U.S. Coast Guard commanding officer stationed in Barnegat Light. Smith learned from the Coast Guard commander that the night before, "one of their 47-foot vessels, boats, was trailed very closely by more than a dozen of these drones." "Now, that to me, is very, very, not just suspicious, provocative, and this could be a foreign power, whether it be [Vladimir] Putin, or it could be Xi Jinping in China, or the Middle East, we can't rule any of that out," the congressman said. Photos taken in the Bay Shore section of Toms River of what appear to be large drones hovering in the area at high altitudes in New Jersey on Sunday, Dec. 8, 2024.

At least five people have been killed in Israeli attacks on several towns in southern Lebanon, the country's Health Ministry has said, amid a fragile ceasefire between Israel and Hezbollah. "An Israeli enemy drone strike on the town of Ainata killed one person and wounded another," the ministry said. An "Israeli strike on the town of Bint Jbeil killed three people," while a third "on Beit Lif killed one person", it added. There was no immediate comment from the Israeli military on the attacks. Israel's army escalated its attacks on Lebanon in late September after more than 11 months of cross-border exchanges of fire with the Lebanese armed group Hezbollah, which began firing rockets towards Israel after the Palestinian group Hamas's attack on southern Israel on October 7, 2023.

Can a Big Tech company be sued in Kenya for alleged abuses at an outsourcing company working on its behalf? That's the question at the heart of two lawsuits that are attempting to set a new precedent in Kenya, which is the prime destination for tech companies looking to farm out digital work to the African continent. The two-year legal battle stems from allegations of human rights violations at an outsourced Meta content moderation facility in Nairobi, where employees hired by a contractor were paid as little as 1.50 per hour to view traumatic content, such as videos of rapes, murders, and war crimes. The suits claim that despite the workers being contracted by an outsourcing company, called Sama, Meta essentially supervised and set the terms for the work, and designed and managed the software required for the task. Both companies deny wrongdoing and Meta has challenged the Kenyan courts' jurisdiction to hear the cases.

Apple has begun rolling iOS 18.2 and iPadOS 18.2 to iPhones and iPads. The updates bring with them major enhancements to the company's suite of AI features, and are likely the final software releases Apple has planned for 2024. More Apple Intelligence features are available through macOS 15.2. However, note access to all of the AI features mentioned below is limited to users in the US, Australia, Canada, New Zealand, South Africa and the UK for now, with support additionally limited to devices with their language set to English. Provided you own an iPhone 15 Pro, 16 or 16 Pro, one of the highlights of iOS 18.2 is Image Playground, which is available both as a standalone app and Messages extension.

Hosted by Ben Byford, The Machine Ethics Podcast brings together interviews with academics, authors, business leaders, designers and engineers on the subject of autonomous algorithms, artificial intelligence, machine learning, and technology's impact on society. In this episode we're chatting to Caitlin about gender and AI, that technology isn't neutral, using technology for good, diversity creation and exploitation, lived experience expertise, co-creating technologies and AI life cycle, importance of success metrics, international treaties on AI, and more… Alliance is a leader of the UN's Generation Equality Action Coalition Technology & Innovation for Gender Equality. Caitlin was co-chair of the Expert Group for the UN Commission on the Status of Women (CSW67) in 2023 with its first ever priority theme of Technology & Innovation. Caitlin leads the Human Rights Toolbox initiative, an educational platform that supports a global community working for a human rights-based approach to AI – with equity & inclusion at the core of the code. Women at the Table are a leader of the fr feminist AI research Network, with Hubs in Latin America & the Caribbean, Middle East & North Africa, SouthEastAsia, and sister network in Africa, and serves as Civil Society lead for the World Benchmarking Alliance's Collective Impact Coalition for Ethical AI.

The Global Change Analysis Model (GCAM) simulates complex interactions between the coupled Earth and human systems, providing valuable insights into the co-evolution of land, water, and energy sectors under different future scenarios. Understanding the sensitivities and drivers of this multisectoral system can lead to more robust understanding of the different pathways to particular outcomes. The interactions and complexity of the coupled human-Earth systems make GCAM simulations costly to run at scale - a requirement for large ensemble experiments which explore uncertainty in model parameters and outputs. A differentiable emulator with similar predictive power, but greater efficiency, could provide novel scenario discovery and analysis of GCAM and its outputs, requiring fewer runs of GCAM. As a first use case, we train a neural network on an existing large ensemble that explores a range of GCAM inputs related to different relative contributions of energy production sources, with a focus on wind and solar. We complement this existing ensemble with interpolated input values and a wider selection of outputs, predicting 22,528 GCAM outputs across time, sectors, and regions. We report a median $R^2$ score of 0.998 for the emulator's predictions and an $R^2$ score of 0.812 for its input-output sensitivity.