Human language and logic abilities are computationally quantified within the well-studied grammar-automata hierarchy. We identify three hierarchical tiers and two corresponding transitions and show their correspondence to specific abilities in transformer-based language models (LMs). These emergent abilities have often been described in terms of scaling; we show that it is the transition between tiers, rather than scaled size itself, that determines a system's capabilities. Specifically, humans effortlessly process language yet require critical training to perform arithmetic or logical reasoning tasks; and LMs possess language abilities absent from predecessor systems, yet still struggle with logical processing. We submit a novel benchmark of computational power, provide empirical evaluations of humans and fifteen LMs, and, most significantly, provide a theoretically grounded framework to promote careful thinking about these crucial topics. The resulting principled analyses provide explanatory accounts of the abilities and shortfalls of LMs, and suggest actionable insights into the expansion of their logic abilities.

Weakly-Supervised Video Object Localization (WSVOL) involves localizing an object in videos using only video-level labels, also referred to as tags. State-of-the-art WSVOL methods like Temporal CAM (TCAM) rely on class activation mapping (CAM) and typically require a pre-trained CNN classifier. However, their localization accuracy is affected by their tendency to minimize the mutual information between different instances of a class and exploit temporal information during training for downstream tasks, e.g., detection and tracking. In the absence of bounding box annotation, it is challenging to exploit precise information about objects from temporal cues because the model struggles to locate objects over time. To address these issues, a novel method called transformer based CAM for videos (TrCAM-V), is proposed for WSVOL. It consists of a DeiT backbone with two heads for classification and localization. The classification head is trained using standard classification loss (CL), while the localization head is trained using pseudo-labels that are extracted using a pre-trained CLIP model. From these pseudo-labels, the high and low activation values are considered to be foreground and background regions, respectively. Our TrCAM-V method allows training a localization network by sampling pseudo-pixels on the fly from these regions. Additionally, a conditional random field (CRF) loss is employed to align the object boundaries with the foreground map. During inference, the model can process individual frames for real-time localization applications. Extensive experiments on challenging YouTube-Objects unconstrained video datasets show that our TrCAM-V method achieves new state-of-the-art performance in terms of classification and localization accuracy.

Inferring causal relationships in observational time series data is an important task when interventions cannot be performed. Granger causality is a popular framework to infer potential causal mechanisms between different time series. The original definition of Granger causality is restricted to linear processes and leads to spurious conclusions in the presence of a latent confounder. In this work, we harness the expressive power of recurrent neural networks and propose a deep learning-based approach to model non-linear Granger causality by directly accounting for latent confounders. Our approach leverages multiple recurrent neural networks to parameterise predictive distributions and we propose the novel use of a dual-decoder setup to conduct the Granger tests. We demonstrate the model performance on non-linear stochastic time series for which the latent confounder influences the cause and effect with different time lags; results show the effectiveness of our model compared to existing benchmarks.

The elemental constituent functions of human minds are not yet known, and the paths to identifying these basic "cognitive acts" are constrained at each end by biology and behavior. A coherent architecture derived bottom-up from brain circuits is proffered. We posit principles and pose questions about architectures, their composition, and their applicability to a described range of formidable tasks, with the primary intent of aiding in setting guideposts and challenges for ongoing architecture studies.

Vector Autoregression (VAR) is a forecasting algorithm that can be used when two or more time series influence each other. That is, the relationship between the time series involved is bi-directional. In this post, we will see the concepts, intuition behind VAR models and see a comprehensive and correct method to train and forecast VAR models in python using statsmodels. First, what is Vector Autoregression (VAR) and when to use it? Vector Autoregression (VAR) is a multivariate forecasting algorithm that is used when two or more time series influence each other.

Vast information from the neurosciences may enable bottom-up understanding of human intelligence; that is, derivation of function from mechanism. This article describes such a research program: simulation and analysis of the circuits of the brain has led to derivation of a detailed set of elemental and composed operations emerging from individual and combined circuits. The specific hypothesis is forwarded that these operations constitute the "instruction set" of the brain, that is, the basic mental operations from which all complex behavioral and cognitive abilities are constructed, establishing a unified formalism for description of human faculties ranging from perception and learning to reasoning and language, and representing a novel and potentially fruitful research path for the construction of human-level intelligence. Attempts to construct intelligent systems are strongly impeded by the lack of formal specifications of natural intelligence, which is defined solely in terms of observed and measured human (or animal) abilities, so candidate computational descriptions of human-level intelligence are necessarily underconstrained. This simple fact underlies Turing's proposed test for intelligence: lacking any specification to test against, the sole measures at that time were empirical observations of behavior, even though such behaviors may be fitted by multiple different hypotheses and simulated by many different proposed architectures.

A new comprehensive approach to nonlinear time series analysis and modeling is developed in the present paper. We introduce novel data-specific mid-distribution based Legendre Polynomial (LP) like nonlinear transformations of the original time series {Y (t)} that enables us to adapt all the existing stationary linear Gaussian time series modeling strategy and made it applicable for non-Gaussian and nonlinear processes in a robust fashion. The emphasis of the present paper is on empirical time series modeling via the algorithm LPTime. We demonstrate the effectiveness of our theoretical framework using daily S&P 500 return data between Jan/2/1963 - Dec/31/2009. Our proposed LPTime algorithm systematically discovers all the'stylized facts' of the financial time series automatically all at once, which were previously noted by many researchers one at a time.

As chief executive officer of one of the world's largest hedge funds, Luke Ellis prides himself on a healthy appetite for risk. "My job," he says, "is to not blink." About five years ago, he did, though--in a big way. What spooked him was an experiment at his firm, Man Group Plc. Engineers at the company's technology-centric AHL unit had been dabbling with artificial intelligence--a buzzy, albeit not widely used, technology at the time. The system they built evolved autonomously, finding moneymaking strategies humans had missed. The results were startlingly good, and now Ellis and fellow executives needed to figure out their next move. Man Group, which has about $96 billion under management, typically takes its most promising ideas from testing to trading real money within weeks. In the fast-moving world of modern finance, an edge today can be gone tomorrow.

Miguel Ferrer, an actor with a long list of credits ranging from "Twin Peaks" to his current role on CBS' "NCIS: Los Angeles," died of cancer on Thursday. A fixture on TV and in movies since the 1980s, Ferrer's reputation as a scene-stealer began with 1987's "RoboCop," where he played Bob Morton, the conniving corporate executive who designed the film's title cyborg. His other landmark role was as FBI agent Albert Rosenfield in David Lynch's landmark series "Twin Peaks," along with its corresponding film, "Fire Walk With Me." Ferrer reprised the role in the upcoming return of the series, which is set to debut in May on Showtime. "Great talent, better man," wrote "Twin Peaks" co-creator Mark Frost on Twitter. "Working & writing for him was a highlight in every part of my life."

David Freedman is the author of an excellent book: "Statistical Models: Theory and Practice" which discusses the issue of causation. It's a very unique stat book in that it really gets into the issue of model assumptions. It claims to be introductory but I believe that a semester or two of math stat as a pre-req would be helpful. In the time series context, you can run a VAR and then do tests for Granger Causality to see if one variable is really "causing" the other where "causing" is defined by Granger. R has a nice package called vars which makes building VAR models and doing testing extremely straightforward.