While GPU clusters are the de facto choice for training large deep neural network (DNN) models today, several reasons including ease of workflow, security and cost have led to efforts investigating whether CPUs may be viable for inference in routine use in many sectors of the industry. But the imbalance between the compute capabilities of GPUs and CPUs is huge. Motivated by these considerations, we study a module which is a workhorse within modern DNN architectures, GEMM based Feed Forward Networks (FFNs), and assess the extent to which it can be made compute- (or FLOP-) lite. Specifically, we propose an alternative formulation (we call it LookupFFN) to GEMM based FFNs inspired by the recent studies of using Locality Sensitive Hashing (LSH) to approximate FFNs. Our formulation recasts most essential operations as a memory look-up, leveraging the trade-off between the two resources on any platform: compute and memory (since CPUs offer it in abundance). For RoBERTa language model pretraining, our formulation achieves similar performance compared to GEMM based FFNs, while dramatically reducing the required FLOP. Our development is complemented with a detailed hardware profiling of strategies that will maximize efficiency -- not just on contemporary hardware but on products that will be offered in the near/medium term future. Code is avaiable at \url{https://github.com/mlpen/LookupFFN}.

In Bayesian optimisation, we often seek to minimise the black-box objective functions that arise in real-world physical systems. A primary contributor to the cost of evaluating such black-box objective functions is often the effort required to prepare the system for measurement. We consider a common scenario where preparation costs grow as the distance between successive evaluations increases. In this setting, smooth optimisation trajectories are preferred and the jumpy paths produced by the standard myopic (i.e.\ one-step-optimal) Bayesian optimisation methods are sub-optimal. Our algorithm, MONGOOSE, uses a meta-learnt parametric policy to generate smooth optimisation trajectories, achieving performance gains over existing methods when optimising functions with large movement costs.

In an incredible display of friendship, a meerkat has been captured roping a robot into babysitting duty. In the clip, which will be aired on the BBC One series'Spy in the Wild' this week, a mechanical meerkat is taken in and accepted into a colony of the animals. The incredible footage shows how meerkat mothers band together to raise their young, which can number as many as 18 a year. In an incredible display of friendship, a meerkat has been captured roping a robot into babysitting duty. In the clip, which will be aired on the BBC One series'Spy in the Wild' this week, a mechanical meerkat is taken in and accepted into a colony of the animals At first sight they look exactly like the real thing – cute, cuddly and in some cases terrifying creatures of the wild.

Welcome to the home page of the computer Hex research group. We --- Kenny Young, Kelly Li, Broderick, Phil, Ryan, Jakub (and previously Aja, David, Jack, Mike, Morgan, Nathan Po, Maryia, Martha, Leah, Yngvi, Geoff Ryan, and Robert Budac) --- build Hex players and solvers. The group informally dates from 1999, when Jack, who wrote Queenbee, started an MSc with Jonathan. Current projects include MoHex, and Solver. Previous projects include Wolve, Mongoose and Queenbee.

Hot Chips Samsung has revealed the blueprints to its mystery M1 processor cores at the heart of its S7 and S7 Edge smartphones. International versions of the top-end Android mobiles, which went on sale in March, sport a 14nm FinFET Exynos 8890 system-on-chip that has four standard 1.6GHz ARM Cortex-A53 cores and four M1 cores running at 2.3 to 2.6GHz. Only two M1 cores are allowed to kick it up to the maximum frequency at any one time to avoid draining batteries and overheating pockets. The M1, codenamed Mongoose, was designed from scratch in three years by a team in the US, and it runs 32-bit and 64-bit ARMv8-A code. In benchmarks, the Exynos 8890 SoC is behind Apple's iPhone 6S A9 chip in terms of single-core performance, but pushes ahead in multi-core tests.