Most developers who grapple with big data are data engineers, data scientists, or machine learning engineers. This book is aimed at those professionals who are looking to use Spark to scale their applications to handle massive amounts of data. In particular, data engineers will learn how to use Spark's Structured APIs to perform complex data exploration and analysis on both batch and streaming data; use Spark SQL for interactive queries; use Spark's built-in and external data sources to read, refine, and write data in different file formats as part of their extract, transform, and load (ETL) tasks; and build reliable data lakes with Spark and the open source Delta Lake table format. For data scientists and machine learning engineers, Spark's MLlib library offers many common algorithms to build distributed machine learning models. We will cover how to build pipelines with MLlib, best practices for distributed machine learning, how to use Spark to scale single-node models, and how to manage and deploy these models using the open source library MLflow.

Devin Wenig, here at eBay's Manhattan offices, joined the company in 2011 and became CEO in 2015. Devin Wenig buys his garbage bags, shampoo and even toothpaste on eBay. That's not typical for your regular eBay customer, who may visit for a car, cardigan or collectible coin. But as eBay's CEO, the 50-year-old Wenig likes to buy as much as he can from the site, now the world's third-largest e-retailer after Amazon and Alibaba. Wenig, who took over at eBay in mid-2015 after the company split with PayPal, met with CNET at his firm's Manhattan office last week during a snowstorm.

Online retailer eBay is attempting to extend its machine language capabilities beyond automatic language translation to e-commerce uses designed to make product searches more relevant. As automation improves, the company said one goal eliminating the search box. Meanwhile, development cycles have been reduced as more machine learning libraries are released to the open source community. "As machines get better at decoding natural language, commerce should become increasingly conversational -- eventually rendering the search box redundant," eBay CEO Devin Wenig noted recently. Wenig added that the pace of machine intelligence development has quickened over the last year.

As one of the internet's pioneers, eBay's seen it all. And now, it has a plan to stay relevant for years to come and even be "one of the winners," the CEO told CNBC's "Squawk on the Street." "We are planting the seeds right now to ensure that eBay is not only relevant, but a leader -- a disruptor -- in artificial intelligence," CEO Devin Wenig said on Thursday. "Some of the acquisitions we've done are world class teams of both data analysts and engineering capability that are going to allow us to use this incredible data set that we have." Wenig said eBay hopes to build a personalized shopping experience based on data they have on how people browse their site.

Can Artificial Intelligence Help Save eBay? By The Street For eBay (EBAY), artificial intelligence could prove to be its salvation. The San Jose, Calif.-based e-commerce giant, which has been working to transform itself after spinning off PayPal Holdings (PYPL) last year, could reap some huge benefits if it can find out a way to effectively use AI, according to analysts. At the Code Conference earlier this month, eBay CEO Devin Wenig revealed that the company started getting very serious about using artificial intelligence last year, noting that e-commerce is essentially a data business and that AI may be more important for eBay than its peers due to the "breadth of its inventory." Wenig further explained that eBay plans to formulate a strategy focused on using data to predict what consumers might want and offering a wide selection of products.

For eBay (EBAY), artificial intelligence could prove to be its salvation. The San Jose, Calif.-based e-commerce giant, which has been working to transform itself after spinning off PayPal Holdings (PYPL) last year, could reap some huge benefits if it can find out a way to effectively use AI, according to analysts. At the Code Conference earlier this month, eBay CEO Devin Wenig revealed that the company started getting very serious about using artificial intelligence last year, noting that e-commerce is essentially a data business and that AI may be more important for eBay than its peers due to the "breadth of [its] inventory." Wenig further explained that eBay plans to formulate a strategy focused on using data to predict what consumers might want and offering a wide selection of products. In fact, eBay aims to use algorithms over the next three to four years to better understand consumer intent and create an ecosystem involving social networks and messaging applications to allow for better targeting, R.W. Baird analyst Colin Sebastian wrote in a note earlier this month.

At the Code Conference earlier this month, eBay CEO Devin Wenig revealed that the company started getting very serious about using AI last year, noting that e-commerce is essentially a data business and that AI may be more important for eBay than its peers due to the "breadth of [its] inventory." Wenig further explained that eBay plans to formulate a strategy focused on using data to predict what consumers might want and offering a wide selection of products. In fact, eBay aims to use algorithms over the next three to four years to better understand consumer intent and create an ecosystem involving social networks and messaging applications to allow for better targeting, R.W. Baird analyst Colin Sebastian wrote in a note earlier this month. "They have the most data. Their data is very actionable," Wedbush Securities analyst Gil Luria said of eBay.

"I absolutely think that we can be a leader in artificial intelligence, and I think its future importance to commerce is significant," Wenig said. "I think horizontal use cases of AI are very far away -- this idea that you'll talk to something and it will do anything for you. Though both Wenig and Bezos have hammered the importance of artificial intelligence, Wenig said eBay will stay away from other Amazon strategies, like ramping up faster delivery. Instead, eBay will focus less on speed and more on providing unique products and breadth of selection, Wenig said. "My entire philosophy is to be less like our competitors, and not more like our competitors," Wenig said.

That obsession you have with '80s-era vintage Transformers? AI and predictive learning will be critical to allowing eBay to set up a truly personalized experience for each consumer, according to CEO Devin Wenig, who spoke Thursday at Recode's Code conference in Rancho Palos Verdes, California. He said the company has been working on taking advantage of demographic data and sophisticated algorithms to offer more relevant products. Wenig is just the latest executive to talk up the promise of AI at the Code conference, with high-profile figures such as Google CEO Sundar Pichai, IBM CEO Ginni Rometty and Microsoft co-founder Bill Gates all talking up its potential. "It's all about customer relevance," he said.