A long-standing goal of data management systems has been to build systems which can compute quantitative insights over large corpora of unstructured data in a cost-effective manner. Until recently, it was difficult and expensive to extract facts from company documents, data from scientific papers, or metrics from image and video corpora. Today's models can accomplish these tasks with high accuracy. However, a programmer who wants to answer a substantive AI-powered query must orchestrate large numbers of models, prompts, and data operations. For even a single query, the programmer has to make a vast number of decisions such as the choice of model, the right inference method, the most cost-effective inference hardware, the ideal prompt design, and so on. The optimal set of decisions can change as the query changes and as the rapidly-evolving technical landscape shifts. In this paper we present Palimpzest, a system that enables anyone to process AI-powered analytical queries simply by defining them in a declarative language. The system uses its cost optimization framework to implement the query plan with the best trade-offs between runtime, financial cost, and output data quality. We describe the workload of AI-powered analytics tasks, the optimization methods that Palimpzest uses, and the prototype system itself. We evaluate Palimpzest on tasks in Legal Discovery, Real Estate Search, and Medical Schema Matching. We show that even our simple prototype offers a range of appealing plans, including one that is 3.3x faster and 2.9x cheaper than the baseline method, while also offering better data quality. With parallelism enabled, Palimpzest can produce plans with up to a 90.3x speedup at 9.1x lower cost relative to a single-threaded GPT-4 baseline, while obtaining an F1-score within 83.5% of the baseline. These require no additional work by the user.

Welcome to our weekly FiftyOne tips and tricks blog where we recap interesting questions and answers that have recently popped up on Slack, GitHub, Stack Overflow, and Reddit. FiftyOne is an open source machine learning toolset that enables data science teams to improve the performance of their computer vision models by helping them curate high quality datasets, evaluate models, find mistakes, visualize embeddings, and get to production faster. Ok, let's dive into this week's tips and tricks! "Is there a way to just get bounding boxes around the possibly missing and possibly spurious objects in my dataset?" Here, George is asking about how to isolate potential mistakes in ground truth labels on a dataset.

I envisioned an old Cadillac with massive Texas longhorns adorning the hood meandering along a dusty road. This road was in Egypt, and Stringfield was behind the wheel, sweat glistening on his brow as he hauled a load of freshly-baked sesame seed bagels. No, I hadn't been experimenting with some designer hallucinogen. But my conversation with him, as happens with particularly captivating sources, conjured evocative concepts and imagery, the kind of stuff that begs to be illustrated in word pictures. Thing is, although his latest enterprise encompassed many of the issues I aimed to address in my most recent feature story in MIT Technology Review -- such as fair labor in the AI industry, data ethics and the future of work -- his background as a former Halliburton executive who became a bagel-making entrepreneur during his time in Cairo as an HR consultant with the oil giant never made it into the story.