In natural human-to-human conversations, participants often receive feedback signals from one another based on their follow-up reactions. These reactions can include verbal responses, facial expressions, changes in emotional state, and other non-verbal cues. Similarly, in human-machine interactions, the machine can leverage the user's follow-up utterances as feedback signals to assess whether it has appropriately addressed the user's request. Therefore, we propose using the likelihood of follow-up utterances as rewards to differentiate preferred responses from less favored ones, without relying on human or commercial LLM-based preference annotations. Our proposed reward mechanism, ``Follow-up Likelihood as Reward" (FLR), matches the performance of strong reward models trained on large-scale human or GPT-4 annotated data on 8 pairwise-preference and 4 rating-based benchmarks. Building upon the FLR mechanism, we propose to automatically mine preference data from the online generations of a base policy model. The preference data are subsequently used to boost the helpfulness of the base model through direct alignment from preference (DAP) methods, such as direct preference optimization (DPO). Lastly, we demonstrate that fine-tuning the language model that provides follow-up likelihood with natural language feedback significantly enhances FLR's performance on reward modeling benchmarks and effectiveness in aligning the base policy model's helpfulness.

Residents and officials from multiple US states are demanding answers about mysterious drone sightings that have been blamed on everything from foreign governments to alien UFOs. Numerous'SUV-sized' craft first appeared in New Jersey in mid-November, and have since spread to New York, Pennsylvania and Connecticut. Drone sightings have also been reported in states such as Texas, Oklahoma and California as well as foreign countries such as Germany. But it's unclear whether these reports are related to the activity plaguing the Northeast. In New Jersey, the drones sometimes appear in groups and often remain in the same place for hours at a time, according to eyewitnesses.

Intelligence analysts have revealed why they believe Russia is behind the mysterious drones invading the skies over New Jersey. US Army general Darryl Williams described a situation that mirrors what has unfolded at American/NATO bases across Europe that are known to supply arms to Ukraine. And retired police lieutenant and intelligence analyst Tim McMillan told DailyMail.com Lt McMillan and other experts have noted that the New Jersey sightings circled around Picatinny Arsenal, home of the US Army's CCDC Armaments Center, which is responsible for manufacturing and supplying Ukraine with artillery ammunition. These experts suggest that Russia could be carrying out an intelligence-gathering mission known as'ferreting', meant to intentionally trigger and test their foreign rival's airspace defense procedures and response time.

Gravitational-wave observatories like LIGO are large-scale, terrestrial instruments housed in infrastructure that spans a multi-kilometer geographic area and which must be actively controlled to maintain operational stability for long observation periods. Despite exquisite seismic isolation, they remain susceptible to seismic noise and other terrestrial disturbances that can couple undesirable vibrations into the instrumental infrastructure, potentially leading to control instabilities or noise artifacts in the detector output. It is, therefore, critical to characterize the seismic state of these observatories to identify a set of temporal patterns that can inform the detector operators in day-to-day monitoring and diagnostics. On a day-to-day basis, the operators monitor several seismically relevant data streams to diagnose operational instabilities and sources of noise using some simple empirically-determined thresholds. It can be untenable for a human operator to monitor multiple data streams in this manual fashion and thus a distillation of these data-streams into a more human-friendly format is sought. In this paper, we present an end-to-end machine learning pipeline for features-based multivariate time series clustering to achieve this goal and to provide actionable insights to the detector operators by correlating found clusters with events of interest in the detector.

You've seen the Game awards nominations. Our own Guardian games of the year list is still a wee while away, but while you're waiting – with bated breath, I'm sure – here's an appetiser: Pushing Buttons' alternative awards. Need to recover your hearts while adventuring through a bunch of eerie rifts that are tearing Hyrule apart? Simply conjure a bed out of thin air, make sure you're out of enemy reach and have a wee nap. Need to make your way across a bridgeable gap?

The latest footage of bizarre drones in New Jersey captured several craft orbiting each other over Somerset County, while at least 12 counties have reported sightings. The video, released this week, shows three'mystery drones in the air' as two move extremely close as if they are interacting with each other and the third hovered for'about 15 minutes.' New Jersey Governor Phil Murphy said Monday night that the drones are'very sophisticated, explaining: 'The minute we get eyes on them [the drones], they go dark.' 'I don't blame people for being frustrated,' Gov Murphy continued, adding that he had spent most of Sunday coordinating on the issue with both the White House and the US Department of Homeland Security in the hope of getting answers. He said that the state received 49 sighting reports on Sunday night alone, with hundreds of locals sharing experiences on social media platforms. On Monday, Picatinny Arsenal, the Army facility in Morris County, confirmed it has had 11 sightings of'UFOs' over in its airspace in recent weeks.

We present phi-4, a 14-billion parameter language model developed with a training recipe that is centrally focused on data quality. Unlike most language models, where pre-training is based primarily on organic data sources such as web content or code, phi-4 strategically incorporates synthetic data throughout the training process. While previous models in the Phi family largely distill the capabilities of a teacher model (specifically GPT-4), phi-4 substantially surpasses its teacher model on STEM-focused QA capabilities, giving evidence that our data-generation and post-training techniques go beyond distillation. Despite minimal changes to the phi-3 architecture, phi-4 achieves strong performance relative to its size -- especially on reasoning-focused benchmarks -- due to improved data, training curriculum, and innovations in the post-training scheme.

Statistical downscaling is a technique used in climate modeling to increase the resolution of climate simulations. High-resolution climate information is essential for various high-impact applications, including natural hazard risk assessment. However, simulating climate at high resolution is intractable. Thus, climate simulations are often conducted at a coarse scale and then downscaled to the desired resolution. Existing downscaling techniques are either simulation-based methods with high computational costs, or statistical approaches with limitations in accuracy or application specificity. We introduce Generative Bias Correction and Super-Resolution (GenBCSR), a two-stage probabilistic framework for statistical downscaling that overcomes the limitations of previous methods. GenBCSR employs two transformations to match high-dimensional distributions at different resolutions: (i) the first stage, bias correction, aligns the distributions at coarse scale, (ii) the second stage, statistical super-resolution, lifts the corrected coarse distribution by introducing fine-grained details. Each stage is instantiated by a state-of-the-art generative model, resulting in an efficient and effective computational pipeline for the well-studied distribution matching problem. By framing the downscaling problem as distribution matching, GenBCSR relaxes the constraints of supervised learning, which requires samples to be aligned. Despite not requiring such correspondence, we show that GenBCSR surpasses standard approaches in predictive accuracy of critical impact variables, particularly in predicting the tails (99% percentile) of composite indexes composed of interacting variables, achieving up to 4-5 folds of error reduction.

As waves of loud, car-sized mystery drones continue to buzz over New Jersey, one family reported that the craft changed time on their car's clock. The family of Morris County locals said they were following one of these seemingly terrestrial UFOs in their vehicle, only to experience the odd effect on their car's electronics as the unexplained craft'hovered above them.' 'The clock in their car changed time,' according to one Fox News reporter who spoke to the unnamed family. 'They say the clock went back to normal after they drove off.' While local law enforcement in Morris County has issued a statement asserting that'there is no known threat to public safety' at this time -- the Federal Aviation Administration (FAA) has issued a ban on drone flights over sensitive areas in state.

Recent advances in multimodal training have significantly improved the integration of image understanding and generation within a unified model. This study investigates how vision-language models (VLMs) handle image-understanding tasks, specifically focusing on how visual information is processed and transferred to the textual domain. We compare VLMs that generate both images and text with those that output only text, highlighting key differences in information flow. We find that in models with multimodal outputs, image and text embeddings are more separated within the residual stream. Additionally, models vary in how information is exchanged from visual to textual tokens. VLMs that only output text exhibit a distributed communication pattern, where information is exchanged through multiple image tokens. In contrast, models trained for image and text generation rely on a single token that acts as a narrow gate for the visual information. We demonstrate that ablating this single token significantly deteriorates performance on image understanding tasks. Furthermore, modifying this token enables effective steering of the image semantics, showing that targeted, local interventions can reliably control the model's global behavior.