We're releasing an analysis showing that since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.5 month-doubling time (by comparison, Moore's Law had an 18-month doubling period). Since 2012, this metric has grown by more than 300,000x (an 18-month doubling period would yield only a 12x increase). Improvements in compute have been a key component of AI progress, so as long as this trend continues, it's worth preparing for the implications of systems far outside today's capabilities. The chart shows the total amount of compute, in petaflop/s-days, that was used to train selected results that are relatively well known, used a lot of compute for their time, and gave enough information to estimate the compute used. A petaflop/s-day (pfs-day) consists of performing 1015 neural net operations per second for one day, or a total of about 1020 operations.
In 1965 Gordon Moore, the founder of Intel, predicted that the number of components that could fit on a microchip would double every year for the next decade. Moore revised his prediction in 1975 to a doubling of components every two years – a prophecy that remained true for another four decades. The ramifications on the world of technology and, by extension, society itself of what is now known as "Moore's Law" have proven immeasurable. The doubling of transistors – semi-conductor devices that switch electronic signals and power – meant that technology would become exponentially more powerful, smaller and cheaper. The fact that the smartphone in your pocket is now many thousands times more...
Long-awaited improvements to rail services in the north of England will get a £589m kick-start, the government has announced. Most of it will go on electrifying the line between Manchester and Leeds, the Department for Transport (DfT) said. The Transpennine route upgrade was first announced in 2011 and a modified £2.9bn scheme was confirmed in 2019. A new body to speed up transport investment across the north is also being set up, the DfT added. The improvements include electrification and a doubling of the number of tracks on the most congested sections around Leeds and Huddersfield, allowing faster trains to overtake slower ones.
Corrections officials have said they were separating inmates with flu-like symptoms from the general prison population, providing soap for cleaning housing areas and practicing good hygiene and waiving a $4 medical co-payment that inmates must pay for receiving treatment for cold and flu symptoms. The officials have said their top priority is the health and safety of staff, inmates and the communities they serve.
My MacBook Pro is three years old, and for the first time in my life, a three-year-old primary computer doesn't feel like a crisis which must be resolved immediately. True, this is partly because I'm waiting for Apple to fix their keyboard debacle, and partly because I still cannot stomach the Touch Bar. But it is also because three years of performance growth ain't what it used to be. It is no exaggeration to say that Moore's Law, the mindbogglingly relentless exponential growth in our world's computing power, has been the most significant force in the world for the last fifty years. So its slow deceleration and/or demise are a big deal, and not just because the repercussions are now making their way into every home and every pocket.