Discrete structures are currently second-class in differentiable programming. Since functions over discrete structures lack overt derivatives, differentiable programs do not differentiate through them and limit where they can be used. For example, when programming a neural network, conditionals and iteration cannot be used everywhere; they can break the derivatives necessary for gradient-based learning to work. This limits the class of differentiable algorithms we can directly express, imposing restraints on how we build neural networks and differentiable programs more generally. However, these restraints are not fundamental. Recent work shows conditionals can be first-class, by compiling them into differentiable form as linear neurons. Similarly, this work shows iteration can be first-class -- by compiling to linear recurrent neurons. We present a minimal typed, higher-order and linear programming language with iteration called $\textsf{Cajal}\scriptstyle(\mathbb{\multimap}, \mathbb{2}, \mathbb{N})$. We prove its programs compile correctly to recurrent neurons, allowing discrete algorithms to be expressed in a differentiable form compatible with gradient-based learning. With our implementation, we conduct two experiments where we link these recurrent neurons against a neural network solving an iterative image transformation task. This determines part of its function prior to learning. As a result, the network learns faster and with greater data-efficiency relative to a neural network programmed without first-class iteration. A key lesson is that recurrent neurons enable a rich interplay between learning and the discrete structures of ordinary programming.

We don't program neural networks directly. Instead, we rely on an indirect style where learning algorithms, like gradient descent, determine a neural network's function by learning from data. This indirect style is often a virtue; it empowers us to solve problems that were previously impossible. But it lacks discrete structure. We can't compile most algorithms into a neural network -- even if these algorithms could help the network learn. This limitation occurs because discrete algorithms are not obviously differentiable, making them incompatible with the gradient-based learning algorithms that determine a neural network's function. To address this, we introduce $\textsf{Cajal}$: a typed, higher-order and linear programming language intended to be a minimal vehicle for exploring a direct style of programming neural networks. We prove $\textsf{Cajal}$ programs compile to linear neurons, allowing discrete algorithms to be expressed in a differentiable form compatible with gradient-based learning. With our implementation of $\textsf{Cajal}$, we conduct several experiments where we link these linear neurons against other neural networks to determine part of their function prior to learning. Linking with these neurons allows networks to learn faster, with greater data-efficiency, and in a way that's easier to debug. A key lesson is that linear programming languages provide a path towards directly programming neural networks, enabling a rich interplay between learning and the discrete structures of ordinary programming.

In the summer of 2009, the Israeli neuroscientist Henry Markram strode onto the TED stage in Oxford, England, and made an immodest proposal: Within a decade, he said, he and his colleagues would build a complete simulation of the human brain inside a supercomputer. They'd already spent years mapping the cells in the neocortex, the supposed seat of thought and perception. "It's a bit like going and cataloging a piece of the rain forest," Markram explained. "How many trees does it have? What shapes are the trees?"

You might say the brain is our most photogenic organ. We are, thanks to modern neuroimaging, living amid an explosion of brain data. Just consider: We can zoom into the brain's connectivity to the most minute, molecular level. We can trace individual cells as well as entire cell populations. We can turn neurons on and off just like a light switch.

In the summer of 2009, the Israeli neuroscientist Henry Markram endeavored onto the TED stage in Oxford, England, and introduced an immodest proposal: he and his colleagues would develop a full human brain simulation inside a supercomputer within a decade. They had been mapping the cells in the neocortex, the supposed seat of thought and perception, for years already. "It's a bit like going and cataloging one piece of rainforest," explained Markram. "How many trees it has? What features are the trees? "His team would now establish a virtual Silicon rainforest from which they hoped artificial intelligence would evolve organically.

In the summer of 2009, the Israeli neuroscientist Henry Markram strode onto the TED stage in Oxford, England, and made an immodest proposal: Within a decade, he said, he and his colleagues would build a complete simulation of the human brain inside a supercomputer. They'd already spent years mapping the cells in the neocortex, the supposed seat of thought and perception. "It's a bit like going and cataloging a piece of the rain forest," Markram explained. "How many trees does it have? What shapes are the trees?"

It's been more than a century since Spanish neuroanatomist Santiago Ramón y Cajal won the Nobel Prize for illustrating the way neurons allow you to walk, talk, think, and be. In the intervening hundred years, modern neuroscience hasn't progressed that much in how it distinguishes one kind of neuron from another. Sure, the microscopes are better, but brain cells are still primarily defined by two labor-intensive characteristics: how they look and how they fire. Which is why neuroscientists around the world are rushing to adopt new, more nuanced ways to characterize neurons. Sequencing technologies, for one, can reveal how cells with the same exact DNA turn their genes on or off in unique ways--and these methods are beginning to reveal that the brain is a more diverse forest of bristling nodes and branching energies than even Ramón y Cajal could have imagined.

Santiago Ramón y Cajal, a Spanish histologist and anatomist known today as the father of modern neuroscience, was also a committed psychologist who believed psychoanalysis and Freudian dream theory were "collective lies." When Freud published The Interpretation of Dreams in 1900, the science world swooned over his theory of the unconscious. Dreams quickly became synonymous with repressed desire. Puzzling dream images could unlock buried conflicts, the psychoanalyst said, given the correct interpretation. Cajal, who won the 1906 Nobel Prize for discovering neurons and, more remarkably, intuiting the form and function of synapses, set out to prove Freud wrong. To disprove the theory that every dream is the result of a repressed desire, Cajal began keeping a dream journal and collecting the dreams of others, analyzing them with logic and rigor. Translated here into English for the first time, the dreams of Santiago Ramón y Cajal offer insight into the mind of a great scientist. Cajal eventually deemed the project unpublishable.