Writing complete stories is considered a hallmark display of human intelligence, and thus researchers in artificial intelligence (AI) and natural language generation (NLG) have long used it as a pinnacle task for their research (Klein et al., 1973; Meehan, 1977; Turner, 1993; Dehn, 1981; Liu and Singh, 2002; McIntyre and Lapata, 2009). Creative writing and storytelling present unique challenges for automatic language generation: story arcs extend over thousands of words, stories typically contain multiple characters with their own distinctive personas and voices, and well-written stories have an authorial voice that is consistent and identifiable. At the same time, lies and fabrications-common generation flaws which are a liability in tasks like machine translation and automatic summarization-can be an asset in the creative domain. In recent years, the field of NLG has progressed by leaps and bounds due to the development of neural language models capable of learning the structure of language by ingesting billions of written words (Chowdhery et al., 2022; Zhang et al., 2022; Brown et al., 2020). There has been considerable work in applying these advancements toward the development of AI-powered tools for creative writing, but nearly all previous research in this space has evaluated their methods either with amateur writers or with crowd workers paid to assess performance on narrowly defined tasks (Clark et al., 2018; Roemmele and Gordon, 2015; Nichols et al., 2020). While these sorts of evaluations are valuable as preliminary assessments, we believe it is also crucial to solicit feedback from actual domain experts in creative writing: professional writers, educators, and language experts. Skilled writers comprise a unique user group with a different set of needs and expectations than amateurs.

A major concern of Machine Learning (ML) models is their opacity. They are deployed in an increasing number of applications where they often operate as black boxes that do not provide explanations for their predictions. Among others, the potential harms associated with the lack of understanding of the models' rationales include privacy violations, adversarial manipulations, and unfair discrimination. As a result, the accountability and transparency of ML models have been posed as critical desiderata by works in policy and law, philosophy, and computer science. In computer science, the decision-making process of ML models has been studied by developing accountability and transparency methods. Accountability methods, such as adversarial attacks and diagnostic datasets, expose vulnerabilities of ML models that could lead to malicious manipulations or systematic faults in their predictions. Transparency methods explain the rationales behind models' predictions gaining the trust of relevant stakeholders and potentially uncovering mistakes and unfairness in models' decisions. To this end, transparency methods have to meet accountability requirements as well, e.g., being robust and faithful to the underlying rationales of a model. This thesis presents my research that expands our collective knowledge in the areas of accountability and transparency of ML models developed for complex reasoning tasks over text.

Large language models (LLMs) have led to a series of breakthroughs in natural language processing (NLP), owing to their excellent understanding and generation abilities. Remarkably, what further sets these models apart is the massive amounts of world knowledge they internalize during pretraining. While many downstream applications provide the model with an informational context to aid its performance on the underlying task, how the model's world knowledge interacts with the factual information presented in the context remains under explored. As a desirable behavior, an LLM should give precedence to the context whenever it contains task-relevant information that conflicts with the model's memorized knowledge. This enables model predictions to be grounded in the context, which can then be used to update or correct specific model predictions without frequent retraining. By contrast, when the context is irrelevant to the task, the model should ignore it and fall back on its internal knowledge. In this paper, we undertake a first joint study of the aforementioned two properties, namely controllability and robustness, in the context of LLMs. We demonstrate that state-of-the-art T5 and PaLM (both pretrained and finetuned) could exhibit poor controllability and robustness, which do not scale with increasing model size. As a solution, we propose a novel method - Knowledge Aware FineTuning (KAFT) - to strengthen both controllability and robustness by incorporating counterfactual and irrelevant contexts to standard supervised datasets. Our comprehensive evaluation showcases the utility of KAFT across model architectures and sizes.

Providing better language tools for low-resource and endangered languages is imperative for equitable growth. Recent progress with massively multilingual pretrained models has proven surprisingly effective at performing zero-shot transfer to a wide variety of languages. However, this transfer is not universal, with many languages not currently understood by multilingual approaches. It is estimated that only 72 languages possess a "small set of labeled datasets" on which we could test a model's performance, the vast majority of languages not having the resources available to simply evaluate performances on. In this work, we attempt to clarify which languages do and do not currently benefit from such transfer. To that end, we develop a general approach that requires only unlabelled text to detect which languages are not well understood by a cross-lingual model. Our approach is derived from the hypothesis that if a model's understanding is insensitive to perturbations to text in a language, it is likely to have a limited understanding of that language. We construct a cross-lingual sentence similarity task to evaluate our approach empirically on 350, primarily low-resource, languages.

About three weeks ago, I received an email with an access code for an app that allowed me to take rides in a robotaxi. The app comes from Cruise, a startup that was acquired by General Motors in 2016 for just under a billion dollars. With it, I could now use the Cruise robotaxis, which have been operating in San Francisco since August 2021, for myself. What makes them special is that these robotaxis drive driverless. There is no one in the car when it picks up passengers. Thanks to a friend who had been given access to the app a few months earlier, I was able to make my first two trips as early as the beginning of July. I reported on that, especially because two coyotes had crossed our path.

Can continuous diffusion models bring the same performance breakthrough on natural language they did for image generation? To circumvent the discrete nature of text data, we can simply project tokens in a continuous space of embeddings, as is standard in language modeling. Through qualitative and quantitative evaluation, we show that our text diffusion models generate samples comparable with those produced by standard autoregressive language models -- while being in theory more efficient on accelerator hardware at inference time. Our work paves the way for scaling up diffusion models for text, similarly to autoregressive models, and for improving performance with recent refinements to continuous diffusion. Continuous diffusion models (Sohl-Dickstein et al., 2015) have taken the world of image generation by storm, advancing the state of the art further than ever before (Rombach et al., 2021; Ramesh et al., 2022). Diffusion for language is indeed an attractive prospect. Compared to autoregressive (AR) models (Bengio et al., 2000; Sutskever et al., 2011; Austin et al., 2021; Hoffmann et al., 2022), diffusion models can predict all tokens in a sequence at once. This allows for bidirectional, rather than causal attention-- increasing interactions between tokens, potentially leading to more coherent samples. Diffusion models can make a better usage of hardware accelerators during inference than AR models, since computations are parallelizable over the sequence axis.

Recent lay language generation systems have used Transformer models trained on a parallel corpus to increase health information accessibility. However, the applicability of these models is constrained by the limited size and topical breadth of available corpora. We introduce CELLS, the largest (63k pairs) and broadest-ranging (12 journals) parallel corpus for lay language generation. The abstract and the corresponding lay language summary are written by domain experts, assuring the quality of our dataset. Furthermore, qualitative evaluation of expert-authored plain language summaries has revealed background explanation as a key strategy to increase accessibility. Such explanation is challenging for neural models to generate because it goes beyond simplification by adding content absent from the source. We derive two specialized paired corpora from CELLS to address key challenges in lay language generation: generating background explanations and simplifying the original abstract. We adopt retrieval-augmented models as an intuitive fit for the task of background explanation generation, and show improvements in summary quality and simplicity while maintaining factual correctness. Taken together, this work presents the first comprehensive study of background explanation for lay language generation, paving the path for disseminating scientific knowledge to a broader audience. CELLS is publicly available at: https://github.com/LinguisticAnomalies/pls_retrieval.

The Cyberguy, Kurt Knutsson, discussed how Microsoft is set to have ten thousand employees in China and warns about rogue fraudulent apps on phone app stores, on'Fox & Friends Weekend.' Microsoft is buying Activision Blizzard – the video game company behind titles like "Guitar Hero," "Candy Crush," "World of Warcraft" and "Call of Duty" – in the largest tech acquisitions in history. Antitrust regulators are assessing whether the deal could hurt competition in the booming global video game industry, for numerous reasons including the fact that Microsoft already produces the widely used Xbox gaming console. Further consolidation in the tech industry elicits well deserved skepticism from regulators, but they should also consider how this deal could help America's greatest geopolitical adversary: China. To understand how, it's first important to understand Microsoft's long and close relationship with the People's Republic of China and its ruling Chinese Communist Party (CCP). Microsoft has operated in China for three decades, boasting on its website that its "most complete subsidiary and largest R&D center outside the United States is in China."

Time Series Forecasting has been an active area of research due to its many applications ranging from network usage prediction, resource allocation, anomaly detection, and predictive maintenance. Numerous publications published in the last five years have proposed diverse sets of objective loss functions to address cases such as biased data, long-term forecasting, multicollinear features, etc. In this paper, we have summarized 14 well-known regression loss functions commonly used for time series forecasting and listed out the circumstances where their application can aid in faster and better model convergence. We have also demonstrated how certain categories of loss functions perform well across all data sets and can be considered as a baseline objective function in circumstances where the distribution of the data is unknown. Our code is available at GitHub: https://github.com/aryan-jadon/Regression-Loss-Functions-in-Time-Series-Forecasting-Tensorflow.

Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. Chief diplomats from the world's leading democracies rallied together in a joint statement condemning global adversaries like Iran and North Korea and called on Russia and China to remember their security commitments to the United Nations. After two days of meetings, officials from the Group of 7 (G7) released a lengthy statement Friday in an address to its top geopolitical challengers, warning them to adhere to international laws. United States Secretary of States Antony Blinken and Foreign Minister Yoshimasa Hayashi of Japan, right, meet for bilateral talks at the G7 Foreign Ministers' Meeting in Muenster, Germany, Friday, Nov. 4, 2022.