Technology Engineering Synthetic skin reveals hidden'Mona Lisa' when exposed to heat The octopus-inspired material could lead to better camouflage technology for the military and beyond. Breakthroughs, discoveries, and DIY tips sent six days a week. Octopuses and their cephalopod cousins have long fascinated biologists with their seemingly supernatural shapeshifting. The cephalopods rapidly change color and texture, blending into their surroundings and evading predators. This natural camouflage is a remarkable bit of biology that engineers have tried to replicate, albeit with limited success.

As one of the few Chain-of-Thought (CoT) reasoning models--and notably the first open-source implementation of its kind--DeepSeek-R1 has demonstrated remarkable improvements in the performance of complex reasoning tasks. Experimental results show that DeepSeek-R1 not only achieves CoT reasoning but also significantly reduces computational resource requirements [1]. Furthermore, it has outperformed comparable models, such as ChatGPT-o1, in certain benchmark tests, showcasing exceptional performance advantages. However, while the CoT approach significantly enhances reasoning capabilities, it also brings forth security concerns that warrant attention. Due to the influence of scaling laws, the volume of data used during the training of LLMs has reached unprecedented levels. Although extensive methods have been employed to sanitize the data during collection and filtering [2], technical limitations and resource constraints have resulted in a considerable amount of harmful content remaining in the training data.

Pretraining large language models is a complex endeavor influenced by multiple factors, including model architecture, data quality, training continuity, and hardware constraints. In this paper, we share insights gained from the experience of training DMaS-LLaMa-Lite, a fully open source, 1.7-billion-parameter, LLaMa-based model, on approximately 20 billion tokens of carefully curated data. We chronicle the full training trajectory, documenting how evolving validation loss levels and downstream benchmarks reflect transitions from incoherent text to fluent, contextually grounded output. Beyond pretraining, we extend our analysis to include a post-training phase focused on instruction tuning, where the model was refined to produce more contextually appropriate, user-aligned responses. We highlight practical considerations such as the importance of restoring optimizer states when resuming from checkpoints, and the impact of hardware changes on training stability and throughput. While qualitative evaluation provides an intuitive understanding of model improvements, our analysis extends to various performance benchmarks, demonstrating how high-quality data and thoughtful scaling enable competitive results with significantly fewer training tokens. By detailing these experiences and offering training logs, checkpoints, and sample outputs, we aim to guide future researchers and practitioners in refining their pretraining strategies. The training script is available on Github at https://github.com/McGill-DMaS/DMaS-LLaMa-Lite-Training-Code. The model checkpoints are available on Huggingface at https://huggingface.co/collections/McGill-DMaS/dmas-llama-lite-6761d97ba903f82341954ceb.

It is common to reject undesired outputs of Large Language Models (LLMs); however, current methods to do so require an excessive amount of computation, or severely distort the distribution of outputs. We present a method to balance the distortion of the output distribution with computational efficiency, allowing for the generation of long sequences of text with difficult-to-satisfy constraints, with less amplification of low probability outputs compared to existing methods. We show through a series of experiments that the task-specific performance of our method is comparable to methods that do not distort the output distribution, while being much more computationally efficient. Language models sometimes generate undesirable outputs, such as syntactically-incorrect code, hallucinated PII, or profanity. These conditions, which we collectively refer to as errors for the remainder of the paper, can be detected with incremental parsers, regular expression matching, or even simple substring searches. However, once detection occurs, there are several competing methods for mitigating errors in the output. One set of methods, constrained generation (Beurer-Kellner et al., 2024; Geng et al., 2024; Melcer et al., 2024), avoids errors by disabling the generation of any token that immediately leads to such an error. While this method is effective, it can lead to the amplification of low-probability outputs. Another class of methods avoids errors without any amplification of low-probability outputs, at the cost of additional computation. Rejection sampling is the simplest such method; i.e. if the output contains an error, simply generate another sample until the output is acceptable. Adaptive Sampling with Approximate Expected Futures (ASAp) (Park et al., 2024) provides a performance improvement over rejection sampling while maintaining the output distribution by effectively sampling without replacement, but there are still many situations in which it may converge too slowly. In our experiments, we show that our method obtains task-specific performance on par with ASAp, while converging significantly faster when the constraints are difficult to satisfy. We first describe autoregressive language models and their properties.

The boundary between what's real and what's not is becoming ever thinner thanks to a new AI tool from Microsoft. Called VASA-1, the technology transforms a still image of a person's face into an animated clip of them talking or singing. Lip movements are'exquisitely synchronised' with audio to make it seem like the subject has come to life, the tech giant claims. In one example, Leonardo da Vinci's 16th century masterpiece'The Mona Lisa' starts rapping crudely in an American accent. However, Microsoft admits the tool could be'misused for impersonating humans' and is not releasing it to the public.

In recent years, large language models have achieved state-of-the-art performance across various NLP tasks. However, investigations have shown that these models tend to rely on shortcut features, leading to inaccurate predictions and causing the models to be unreliable at generalization to out-of-distribution (OOD) samples. For instance, in the context of relation extraction (RE), we would expect a model to identify the same relation independently of the entities involved in it. For example, consider the sentence "Leonardo da Vinci painted the Mona Lisa" expressing the created(Leonardo da Vinci, Mona Lisa) relation. If we substiute "Leonardo da Vinci" with "Barack Obama", then the sentence still expresses the created relation. A robust model is supposed to detect the same relation in both cases. In this work, we describe several semantically-motivated strategies to generate adversarial examples by replacing entity mentions and investigate how state-of-the-art RE models perform under pressure. Our analyses show that the performance of these models significantly deteriorates on the modified datasets (avg. of -48.5% in F 1), which indicates that these models rely to a great extent on shortcuts, such as surface forms (or patterns therein) of entities, without making full use of the information present in the sentences.

This paper analyses a visual archive of drawings produced by an interactive robotic art installation where audience members narrated their dreams into a system powered by CLIPdraw deep learning (DL) model that interpreted and transformed their dreams into images. The resulting archive of prompt-image pairs were examined and clustered based on concept representation accuracy. As a result of the analysis, the paper proposes four groupings for describing and explaining CLIP-generated results: clear concept, text-to-text as image, indeterminacy and confusion, and lost in translation. This article offers a glimpse into a collection of dreams interpreted, mediated and given form by Artificial Intelligence (AI), showcasing oftentimes unexpected, visually compelling or, indeed, the dream-like output of the system, with the emphasis on processes and results of translations between languages, sign-systems and various modules of the installation. In the end, the paper argues that proposed clusters support better understanding of the neural model.

The Mona Lisa is small. Less than three feet tall and about two feet wide, it hangs tiny in the biggest exhibition room at France's Louvre Museum. And in the past two or so weeks, some vigilante AI artists have decided that it should be bigger--much bigger. They're making that happen using a beta tool in Adobe Photoshop called "generative fill." It launched late last month and allows users to fill in, augment, or expand an image using AI--think ChatGPT but for Photoshop.

Late last month, during yet another inexplicable rebranding exercise, HBO's Max streaming service changed the way it organizes film credits. Rather than separate out discrete production categories for users to peruse, Max's credits lumped writers and directors together under an ominous header, dubbing them "creators." The recategorization enraged writers, filmmakers, and the Directors Guild of America. Within a few hours, Max's parent company, Warner Bros., apologized for the move, calling it "an oversight in the technical transition from HBO Max to Max." The change--made by a company with a market cap that is approaching $30 billion during a contentious writers' strike--felt petty and vindictive to Hollywood professionals.

Have you ever wondered what famous historical paintings would look like if they were created today? Thanks to the power of artificial intelligence (AI), it's now possible to reimagine these masterpieces in ways that were previously impossible. Using advanced algorithms like those found in Midjourney v5, AI can analyze and manipulate existing artwork to create new and innovative pieces. For example, AI can be used to alter the color palette or style of a painting, giving it a modern twist while still maintaining the essence of the original work. Let's take a closer look at how AI can be used to reimagine these famous paintings: The Persistence of Time is a surrealist masterpiece that is famous for its melting clocks.