Goto

Collaborating Authors

 Africa


The Goldilocks zone: Towards better understanding of neural network loss landscapes

arXiv.org Machine Learning

We explore the loss landscape of fully-connected neural networks using random, low-dimensional hyperplanes and hyperspheres. Evaluating the Hessian, $H$, of the loss function on these hypersurfaces, we observe 1) an unusual excess of the number of positive eigenvalues of $H$, and 2) a large value of $\mathrm{Tr}(H) / |H|$ at a well defined range of configuration space radii, corresponding to a thick, hollow, spherical shell we refer to as the \textit{Goldilocks zone}. We observe this effect for fully-connected neural networks over a range of network widths and depths on MNIST and CIFAR-10 with the $\mathrm{ReLU}$ non-linearity. The effect is not observed for the $\tanh$ non-linearity. Using our observations, we demonstrate a close connection between the Goldilocks zone, measures of local convexity/prevalence of positive curvature, and the suitability of a network initialization. We show that the high and stable accuracy reached when optimizing on random, low-dimensional hypersurfaces is directly related to the overlap between the hypersurface and the Goldilocks zone. We note that common initialization techniques initialize neural networks in this particular region of unusually high convexity, and offer a geometric intuition for their success. We take steps towards an analytic description of the general features of the loss function geometry, exploring its anisotropy and strong radial dependence. We support our theoretical results with experiments. Furthermore, we demonstrate that initializing a neural network at a number of points and selecting for high measures of local convexity such as $\mathrm{Tr}(H) / |H|$, number of positive eigenvalues of $H$, or low initial loss, leads to statistically significantly faster training on MNIST. Based on our observations, we hypothesize that the Goldilocks zone contains a high density of suitable initialization configurations.


NHS70: How Has Technology Changed Our Healthcare?

Forbes - Tech

Approximately 330,000 cataract operations are performed in England alone and it all started with the introduction of the intraocular lens. Sir Harold Ridley was the first to successfully implant an intraocular lens on 29 November 1949, at St Thomas' Hospital at London, however, it wasn't until the 1970s, following further developments in lens design and surgical techniques, that the lens found acceptance in cataract surgery. Laser eye surgery then followed on from the intraocular lens in the 1990s.


Tinder adds GIF-like video loops to spice up your dating profile

Engadget

If you're a dating app regular, you know that a photo only says so much about yourself. But do you really want to go to the trouble of recording a whole video for people who could swipe left before you've even spoken a word? Tinder thinks there's a better balance between the two. It's launching a Loops feature that (surprise) adds two-second looping videos to your profile alongside the usual still shots. You just have to trim an existing video to portray yourself as a fun-loving party person or tender romantic. The feature is initially available for iOS users in the US, UK, Canada and large chunks of Western Europe, Asia and the Middle East.


Artificial intelligence... for animals, Sustainable States, Click - BBC World News

#artificialintelligence

Researchers at the University of Wyoming are working with online community Snapshot Serengeti to develop artificial intelligence that can identify, count and describe wild animals in Tanzania. Over 3.2 million pictures of animals - captured by hidden cameras - have been identified by Snapshot Serengeti volunteers and the data collected is being used to power the AI algorithm, using deep neural networks.


Will Artificial Intelligence Make Citizen Scientists Obsolete? - Pacific Standard

#artificialintelligence

In Serengeti National Park, there are 225 hidden cameras constantly photographing the creatures that roam this Tanzanian wilderness. To date, these camera traps have captured more than three million images. Through Serengeti Snapshot, as the program is called, they've studied everything from the migrations of the region's herbivores to the surprising co-existence of lions, hyenas, and cheetahs. It's work that wouldn't have been possible without an army of 30,000 citizen scientists, who manually sorted the collection, identifying and naming the species in each frame. It's time-consuming work, and the volunteers are doing it for kicks.


Groupe PSA and Inria create an OpenLab dedicated to artificial intelligence - Automotive World

#artificialintelligence

Groupe PSA and Inria today announced the creation of an OpenLab dedicated to artificial intelligence. The studied areas will include autonomous and intelligent vehicles, mobility services, manufacturing, design development tools, the design itslelf and digital marketing as well as quality and finance. "Artificial intelligence will quickly become an efficiency factor for the group. The OpenLab will work on artificial intelligence algorithms enabling autonomous vehicles to drive in complex environments for example. It will also work on predictive maintenance, powertrain design optimisation and the modelling of complex systems such as cities, to offer mobility services adapted to people's needs" said Carla Gohin, Groupe PSA's Vice President for Research and Advanced Engineering. Inria's project teams will participate in this OpenLab bringing their high-level algorithmic expertise as part of a fruitful dialogue with Groupe PSA's experts on all the identified topics.


New Tokyo research center aims to boost Japan's 'Fourth Industrial Revolution'

The Japan Times

The newly established Center for the Fourth Industrial Revolution in the Japanese capital aims to update old regulations that hinder effective usage of cutting-edge technologies and accelerate social change by developing appropriate policy frameworks for a rapidly changing society, said Chizuru Suga, who heads the institution. "In short, what we are trying to do is like determining the size of a soccer goal before playing the game. We are trying to set up common policy frameworks that help people play a fair game," she said during a recent interview with The Japan Times. "Today's technology has been advancing so fast that no one could have been able to catch up with the most up-to-date movement and set the rules … to benefit as many people as possible," said Suga, originally from the Ministry of Economy, Trade and Industry (METI). The Tokyo facility, which opened Monday, is the first sister institution of the Center for the Fourth Industrial Revolution in San Francisco.



The NHS at 70: A timeline in pictures

BBC News

On 5 July 1948, 70 years ago on Thursday, the National Health Service (NHS) was born. We look at seven decades of the development of the NHS, alongside medical advancements, highlighting events that have been documented with archive photography. After the planning of an ambitious project to bring healthcare to everyone in the UK, the NHS was launched by Aneurin Bevan, the Health Secretary of the post-War Labour government. Bevan, seen below in 1945, was the chief architect of the plan to bring together hospitals, doctors, nurses, pharmacists, opticians and dentists under the umbrella of one organisation. The services were available to everyone, financed entirely from taxation with the central principle being that people would pay according to their means.


Logistic Regression, Neural Networks and Dempster-Shafer Theory: a New Perspective

arXiv.org Machine Learning

We revisit logistic regression and its nonlinear extensions, including multilayer feedforward neural networks, by showing that these classifiers can be viewed as converting input or higher-level features into Dempster-Shafer mass functions and aggregating them by Dempster's rule of combination. The probabilistic outputs of these classifiers are the normalized plausibilities corresponding to the underlying combined mass function. This mass function is more informative than the output probability distribution. In particular, it makes it possible to distinguish between lack of evidence (when none of the features provides discriminant information) from conflicting evidence (when different features support different classes). This expressivity of mass functions allows us to gain insight into the role played by each input feature in logistic regression, and to interpret hidden unit outputs in multilayer neural networks. It also makes it possible to use alternative decision rules, such as interval dominance, which select a set of classes when the available evidence does not unambiguously point to a single class, thus trading reduced error rate for higher imprecision.