Analyses performed using Spark of brain activity in a larval zebrafish: embedding dynamics of whole-brain activity into lower-dimensional trajectories. The growth of data volumes in industry and research poses tremendous opportunities, as well as tremendous computational challenges. As data sizes have outpaced the capabilities of single machines, users have needed new systems to scale out computations to multiple nodes. As a result, there has been an explosion of new cluster programming models targeting diverse computing workloads.1,4,7,10 At first, these models were relatively specialized, with new models developed for new workloads; for example, MapReduce4 supported batch processing, but Google also developed Dremel13 for interactive SQL queries and Pregel11 for iterative graph algorithms. In the open source Apache Hadoop stack, systems like Storm1 and Impala9 are also specialized. Even in the relational database world, the trend has been to move away from "one-size-fits-all" systems.18 Unfortunately, most big data applications need to combine many different processing types. The very nature of "big data" is that it is diverse and messy; a typical pipeline will need MapReduce-like code for data loading, SQL-like queries, and iterative machine learning. Specialized engines can thus create both complexity and inefficiency; users must stitch together disparate systems, and some applications simply cannot be expressed efficiently in any engine. In 2009, our group at the University of California, Berkeley, started the Apache Spark project to design a unified engine for distributed data processing. Spark has a programming model similar to MapReduce but extends it with a data-sharing abstraction called "Resilient Distributed Datasets," or RDDs.25 Using this simple extension, Spark can capture a wide range of processing workloads that previously needed separate engines, including SQL, streaming, machine learning, and graph processing2,26,6 (see Figure 1).

Video Friday is your weekly selection of awesome robotics videos, collected by your Automaton bloggers. We'll also be posting a weekly calendar of upcoming robotics events for the next two months; here's what we have so far (send us your events!): Let us know if you have suggestions for next week, and enjoy today's videos. I really don't know much about this beyond what's in the video, but we don't see a lot of Russian robots around here, so: It looks to be a project from the Russian equivalent of DARPA, designed to go into space by 2021. This video of Agile Justin feeling up different kinds of rods was a finalist for both "IROS Best Paper on Cognitive Robotics" and "IROS Best Student Paper": This is cool because until this point, building pneumatic robots required making molds and casting custom parts.

The new MacBook Pro was always going to be expensive. But it is very expensive indeed in the UK, after the prices were shifted because of the falling pound. The computers are as much as £500 more expensive than previous models, because of a combination of the currency changes and the upgrades introduced by Apple. Buying the cheapest version of the MacBook Pro with Touch Bar that was released yesterday will cost customers £1,749. Customers can spend as much as £2,699 on stock models of the computer, if they buy the 15-inch version.

The New York businessman with a penchant for celebrity television may suddenly find himself in love with artificial intelligence developed in India. The polls and simulations that involve the skills and insight of human beings suggest Donald Trump could be heading for something of a pasting. But an artificial intelligence (AI) system developed in Mumbai, and which correctly predicted the last three US presidential elections, puts the Republican nominee ahead of his rival Hillary Clinton in the battle to secure the keys to the White House. MogIA was developed by Sanjiv Rai, the founder of Indian start-up Genic.ai. It has taken 20 million data points from public platforms such as Google, Facebook and Twitter and analysed the information to create predictions, CNBC reported.

European privacy experts have sent letters to WhatsApp telling it to stop sharing people's data with Facebook. WhatsApp announced in recent weeks that it would start handing over information about the people who use it to Facebook, so that its parent company could use that data to better target ads. But the company didn't give a very easy way of opting out of it, and the deal has drawn the attention of customers and regulators. Now EU privacy watchdogs have told WhatsApp and Facebook that the deal must be stopped until it can be checked whether it is legal or not. Presumably if the deal is found to be illegal then it will be forced to stop.

Adult website Pornhub says that it can save Vine, the video sharing company, by buying it. The company has sent an official letter to Jack Dorsey, the CEO of Vine's owner Twitter, offering to buy the company. In doing so it hopes to help avoid it being shut down entirely, as Twitter and Vine have announced it will do. The company does make clear that the site would turn into a porn site. The letter sent to Mr Dorsey by Pornhub CEO Corey Price says that Vine would be restored "back to its NSFW glory".

Apple has launched its latest products in its headquarters in Cupertino, California. In the 24 hours since, I've been trying out the new MacBook Pro for size. First of all, I should point out that this is the entry-level MacBook Pro, not the more expensive MacBook Pro with TouchBar. TouchBar is the ground-breaking innovation where the top row of function keys is replaced with a long, touch-sensitive display strip. I tried this briefly and it's frankly pretty amazing. But I'll be reviewing that and the sapphire-crystal power and Touch ID button also found on the MacBook Pro with TouchBar in due course.

George "Geohot" Hotz, PlayStation and iPhone hacker extraordinaire, has canceled production on Comma One, a $1,000 aftermarket add-on that he said would allow some cars to operate semi-autonomously. He claimed the tech was "about on par" with Tesla's Autopilot and it used cars' video feeds to navigate the roads. It was due to start rolling out at the end of this year. But, after receiving a special order from the National Highway Traffic Safety Administration today, Geohot decided Comma One wasn't worth the paperwork. Geohot said on Twitter that the special order opened "with threats" rather than an attempt at dialogue.

Companies have been trying to drum up enthusiasm for them for years, with little success. Pepper, a humanoid machine carrying the hopes of SoftBank Group Corp.'s billionaire founder Masayoshi Son, was supposed to change that. Promoted as the first robot to be endowed with emotions, the company marketed Pepper aggressively after it was unveiled in 2014, promising the gadget was sophisticated enough for tasks usually handled by shop clerks, receptionists and translators. "It's not there to have a conversation," said Junichi Nishi, a municipal official in Fujieda, Shizuoka Prefecture, a city of about 140,000. "We use it primarily as a tablet," he said, referring to the touch screen attached to the robot's chest.

A yet-to-be-released Segway Ninebot personal transportation robot is seen onstage during the Intel keynote address at the Consumer Electronics Show in Las Vegas. Editor's Note: Economist Dean Baker is the author of the new book, "Rigged: How Globalization and the Rules of the Modern Economy Were Structured to Make the Rich Richer." There is a recurring concern in discussions about the economy that technology threatens the livelihood of large segments of the workforce. In one version, the robots will take all the jobs, leading to a massive surge in unemployment. A somewhat different version has the development of technology benefiting people with college and advanced degrees to the detriment of those with less education.