Welcome back to, TIME's new twice-weekly newsletter about AI. If you're reading this in your browser, why not subscribe to have the next one delivered straight to your inbox? The three most valuable private companies in the U.S. have big reputations: OpenAI, SpaceX, and Anthropic. But the fourth, Databricks, flies a little more under the radar. This company, which is currently raising funds at a valuation of $134 billion according to reports this week, is a quiet workhorse of the AI revolution.

Inside a 13th-floor boardroom in downtown San Francisco, the atmosphere was tense. It was November 2015, and Databricks, a two-year-old software company started by a group of seven Berkeley researchers, was long on buzz but short on revenue. The directors awkwardly broached subjects that had been rehashed time and again. The startup had been trying to raise funds for five months, but venture capitalists were keeping it at arm's length, wary of its paltry sales. Seeing no other option, NEA partner Pete Sonsini, an existing investor, raised his hand to save the company with an emergency $30 million injection. Founding CEO Ion Stoica had agreed to step aside and return to his professorship at the University of California, Berkeley. The obvious move was to bring in a seasoned Silicon Valley executive, which is exactly what Databricks' chief competitor Snowflake did twice on its way to a software-record $33 billion IPO in September 2020.

Prescient are the entrepreneurs who predicted data would become the new oil, like Ali Ghodsi, Andy Konwinski, Ion Stoica, Matei Zaharia, Patrick Wendell, Reynold Xin, and Scott Shenker. They're the cofounders of Databricks, a San Francisco-based company that provides a suite of enterprise-focused scalable data science and data engineering tools. Since 2013, the year Databricks opened for business, it's had no trouble attracting customers. But this week kicked into high gear the company's uninterrupted march toward market domination. Databricks this morning announced that it's closed a $400 million series F fundraising round led by Andreessen Horowitz with participation from Microsoft, Alkeon Capital Management, BlackRock, Coatue Management, Dragoneer Investment Group, Geodesic, Green Bay Ventures, New Enterprise Associates, T. Rowe Price, and Tiger Global Management.

At the Spark AI Summit, Europe, Enterprise Times sat with Ali Ghodsi, CEO and co-founder, Databricks to talk AI and big data. Ghodsi started as an AI researcher and took that knowledge and experience into Databricks when it was founded. It gives him an interesting perspective on the state of the, often overhyped, AI market. For example, Ghodsi says that one of the reasons for founding Databricks was to: "democratise artificial intelligence and bring it to the masses." One of the problems that AI faces is that it is not a new discipline, it's been around for literally decades.

In conversations about the future of artificial intelligence (AI), the idea that machines will soon take over our whole lives and even eliminate jobs, increasing the numbers of people unemployed, usually comes into play. One of the biggest names in AI, Hinton is known as the godfather of AI for his pioneering work in neural networks. He is now professor of computer science at the University of Toronto and part of the Google Brain project. In his book, Architects of Intelligence: The Truth About AI from the People Building It, Martin Ford talks with Hinton about the economic and social ramifications of AI, and Hinton says that dramatically increasing productivity should be a good thing. According to Hinton, "People are looking at the technology as if the technological advances are the problem. The problem is in the social systems, and whether we're going to have a social system that shares fairly, or one that focuses all the improvement on the 1% and treats the rest of the people like dirt. That's nothing to do with technology."

We study fair allocation of indivisible goods to agents with unequal entitlements. Fair allocation has been the subject of many studies in both divisible and indivisible settings. Our emphasis is on the case where the goods are indivisible and agents have unequal entitlements. This problem is a generalization of the work by Procaccia and Wang (2014) wherein the agents are assumed to be symmetric with respect to their entitlements. Although Procaccia and Wang show an almost fair (constant approximation) allocation exists in their setting, our main result is in sharp contrast to their observation. We show that, in some cases with n agents, no allocation can guarantee better than 1/n approximation of a fair allocation when the entitlements are not necessarily equal. Furthermore, we devise a simple algorithm that ensures a 1/n approximation guarantee. Our second result is for a restricted version of the problem where the valuation of every agent for each good is bounded by the total value he wishes to receive in a fair allocation. Although this assumption might seem without loss of generality, we show it enables us to find a 1/2 approximation fair allocation via a greedy algorithm. Finally, we run some experiments on real-world data and show that, in practice, a fair allocation is likely to exist. We also support our experiments by showing positive results for two stochastic variants of the problem, namely stochastic agents and stochastic items.

The concept of machine learning, which is a subset of artificial intelligence, has been around for some time. Ali Ghodsi, an adjunct professor at UC Berkeley, describes it as "an advanced statistical technique to make predictions on a massive amount of data." Ghodsi has been influential in areas of Big Data, distributed systems, and in machine learning projects including Apache Spark, Apache Hadoop, and Apache Mesos. Here, he shares insight on these projects, various use-cases, and the future of machine learning. There are some commonalities among these three projects that have been influenced by Ghodsi's research.

Microsoft is bringing its Azure Databricks cloud service out of beta today to help its customers better process massive amounts of data, powered by a partnership unlike anything the tech titan has done before. The company worked with Databricks to produce the service, which is an analytics system based on the popular Apache Spark open source project. Customers use it to ingest and process a large amount of data using machine learning techniques. The Azure Databricks service is supposed to give them an easier experience of running big data jobs than rolling their own Spark deployment, and offers a deeper level of compatibility with Microsoft's first-party offerings than other services offered through the Azure Marketplace. This release is a move to capitalize on enterprises' interest in using more data to power artificial intelligence systems.

We see the power of artificial intelligence every day: When Netflix recommends a movie you love, when your bank detects fraud in your account, or when Google routes you around a traffic jam. But outside of examples from mammoth companies with millions to spend on data science initiatives, there's a decided lack of AI success among the rest of us. That's the conclusion that Ali Ghodsi has come to. As the co-founder and CEO of Databricks and an adjunct professor at UC Berkeley, Ghodsi has a direct view into the types the AI projects that organizations are embarking on. It turns out, those organizations are struggling mightily, and he wonders why more people aren't talking about it.

Microsoft this week unveiled a new service that integrates its Azure platform with a globally distributed streaming analytics solution. Now available in preview, Azure Databricks is designed to help app users and developers take advantage of machine learning, graph processing and AI-based applications. It was one of many announcements made by Microsoft during the opening keynote of its annual Connect() developer conference, which kicked off Wednesday in New York City. Azure Databricks will enable organizations to build modern data warehouses that support self-service analytics and machine learning using all data types in a secure and compliant architecture, according to Scott Guthrie, Microsoft's executive vice president of Cloud and Enterprise, speaking at the the Connect() keynote. Databricks is the creator and steward of Apache Spark.