Bayesian Additive Regression Trees (BART) is a nonparametric Bayesian regression technique of rising fame. It is a sum-of-decision-trees model, and is in some sense the Bayesian version of boosting. In the limit of infinite trees, it becomes equivalent to Gaussian process (GP) regression. This limit is known but has not yet led to any useful analysis or application. For the first time, I derive and compute the exact BART prior covariance function. With it I implement the infinite trees limit of BART as GP regression. Through empirical tests, I show that this limit is worse than standard BART in a fixed configuration, but also that tuning the hyperparameters in the natural GP way yields a competitive method, although a properly tuned BART is still superior. The advantage of using a GP surrogate of BART is the analytical likelihood, which simplifies model building and sidesteps the complex BART MCMC. More generally, this study opens new ways to understand and develop BART and GP regression. The implementation of BART as GP is available in the Python package https://github.com/Gattocrucco/lsqfitgp .

Neural networks often assume independence among input data samples, disregarding correlations arising from inherent clustering patterns in real-world datasets (e.g., due to different sites or repeated measurements). Recently, mixed effects neural networks (MENNs) which separate cluster-specific 'random effects' from cluster-invariant 'fixed effects' have been proposed to improve generalization and interpretability for clustered data. However, existing methods only allow for approximate quantification of cluster effects and are limited to regression and binary targets with only one clustering feature. We present MC-GMENN, a novel approach employing Monte Carlo methods to train Generalized Mixed Effects Neural Networks. We empirically demonstrate that MC-GMENN outperforms existing mixed effects deep learning models in terms of generalization performance, time complexity, and quantification of inter-cluster variance. Additionally, MC-GMENN is applicable to a wide range of datasets, including multi-class classification tasks with multiple high-cardinality categorical features. For these datasets, we show that MC-GMENN outperforms conventional encoding and embedding methods, simultaneously offering a principled methodology for interpreting the effects of clustering patterns.

This paper provides an overview of the intricate relationship between social dynamics, technological advancements, and pioneering figures in the fields of cybernetics and artificial intelligence. It explores the impact of collaboration and interpersonal relationships among key scientists, such as McCulloch, Wiener, Pitts, and Rosenblatt, on the development of cybernetics and neural networks. It also discusses the contested attribution of credit for important innovations like the backpropagation algorithm and the potential consequences of unresolved debates within emerging scientific domains. It emphasizes how interpretive flexibility, public perception, and the influence of prominent figures can shape the trajectory of a new field. It highlights the role of funding, media attention, and alliances in determining the success and recognition of various research approaches. Additionally, it points out the missed opportunities for collaboration and integration between symbolic AI and neural network researchers, suggesting that a more unified approach may be possible in today's era without the historical baggage of past debates.

The history of deep learning goes back as far as 1943, when Walter Pitts and Warren McCulloch created a computer model based on the neural networks of the human brain. Today, if we asked a language model like GPT-3 to write an article about the history of deep learning, it might begin with that sentence. Many changes led from Pitts and McCulloch's early neural network to what we now call "deep learning": the addition of backpropagation (Yann LeCun and others), and the creation of "deep" networks with many "hidden layers" (Geoff Hinton and others) are perhaps the most important. And while early neural networks couldn't be programmed effectively (if at all) on the computers of their day, deep learning has now become commonplace. What was once couldn't even be implemented on the largest supercomputers run comfortably on your laptop.

Generalised Linear Models (GLMs McCullagh & Nelder 1989; Nelder & Wedderburn 1972) are frequently used in different applications to predict a univariate response due to the ease of interpretation of the parameter estimates as well as the large availability of software that facilitates simple analyses. A common assumption in GLMs is that the covariates specified (including potential interaction terms) have a linear relationship with the mean of the response after transformation through the link function. Extensions such as Generalised Additive Models (GAMs T. J. Hastie & Tibshirani 1990; Wood 2017) require the specification of the main and interaction effects via a sum of (potentially non-linear) predictors. In GAMs, the non-linear relationship is usually captured via basis expansions of the covariates and constrained by a smoothing parameter. However, in problems where the numbers of covariates and/or observations are large, the linearity assumption may not be verified and, more importantly, it may not be simple to specify the covariates and their interactions that impact most on the response.

See how institutions are integrating new technology to create smarter campuses. I've practically grown up in the data, application, information, and tech industry. For over 18 years, I've worked with both small organizations and major corporations like Gartner and Nelnet. With Nelnet Campus Commerce, I'm proud to help our partners consider the impact future technology will have on higher education. If you still remember the rotary phone, you're very different from today's "digital natives."

"Can we actually know the universe? My God, it's hard enough finding your way around Chinatown." "Know then thyself, presume not God to scan; The proper study of mankind is man." The field of AI is directed at the fundamental problem of how the mind works; its approach, among other things, is to try to simulate its working--in bits and pieces. History shows us that mankind has been trying to do this for certainly hundreds of years, but the blooming of current computer technology has sparked an explosion in the research we can now do. The center of AI is the wonderful capacity we call learning, which the field is paying increasing attention to. Learning is difficult and easy, complicated and simple, and most research doesn't look at many aspects of its complexity. However, we in the AI field are starting. Let us now celebrate the efforts of our forebears and rejoice in our own efforts, so that our successors can thrive in their research. This article is the substance, edited and ...

The first 50 years of AI are reviewed, and current controversies outlined. Scientific disputes include disagreements over the best research methodology, including classical AI, connectionism, hybrid systems, and situated and evolutionary robotics. Philosophical disputes concern (for instance) whether computation is necessary and sufficient for mentality, whether representations are essential for intelligence, whether consciousness can be explained objectively, and whether the Cartesian presuppositions of (most) AI should be replaced by a neo-Heideggerian approach. With respect to final verdicts, both juries (scientific and philosophical) are still out. But AI has aided theoretical psychology and revivified the philosophy of mind.

Walter Pitts was used to being bullied. He'd been born into a tough family in Prohibition-era Detroit, where his father, a boiler-maker, had no trouble raising his fists to get his way. One afternoon in 1935, they chased him through the streets until he ducked into the local library to hide. The library was familiar ground, where he had taught himself Greek, Latin, logic, and mathematics--better than home, where his father insisted he drop out of school and go to work. Outside, the world was messy. Inside, it all made sense. Not wanting to risk another run-in that night, Pitts stayed hidden until the library closed for the evening. Alone, he wandered through the stacks of books until he came across Principia Mathematica, a three-volume tome written by Bertrand Russell and Alfred Whitehead between 1910 and 1913, which attempted to reduce all of mathematics to pure logic. Pitts sat down and began to read. For three days he remained in the library until he had read each volume cover to cover--nearly 2,000 pages in all--and had identified several mistakes. Deciding that Bertrand Russell himself needed to know about these, the boy drafted a letter to Russell detailing the errors.

Robotic surgery sounds like the ultimate in safe, efficient and effective 21st-century health care. Instead of a surgeon's potentially fallible human hand, you have a robot with its precision-built mechanical arms able to perform micro-accurate procedures on tissues deep within the body. With robot-assisted surgery, the surgeon sits at a nearby console with a 3D view of the surgical site. If the surgeon's hand develops a tremor, the computer system knows to ignore it. The technology also means surgeons can use finer instruments that cause less damage to the body. In turn, this should reduce blood loss and the need for blood transfusions - and mean that patients recover more quickly.