In a supervised classification split conformal/inductive framework with $K$ classes, a calibration sample of $n$ labeled examples is observed for inference on the label of a new unlabeled example. In this work, we explore the case where a "batch" of $m$ independent such unlabeled examples is given, and a multivariate prediction set with $1-\alpha$ coverage should be provided for this batch. Hence, the batch prediction set takes the form of a collection of label vectors of size $m$, while the calibration sample only contains univariate labels. Using the Bonferroni correction consists in concatenating the individual prediction sets at level $1-\alpha/m$ (Vovk 2013). We propose a uniformly more powerful solution, based on specific combinations of conformal $p$-values that exploit the Simes inequality (Simes 1986). Intuitively, the pooled evidence of fairly "easy" examples of the batch can help provide narrower batch prediction sets. We also introduced adaptive versions of the novel procedure that are particularly effective when the batch prediction set is expected to be large. The theoretical guarantees are provided when all examples are iid, as well as more generally when iid is assumed only conditionally within each class. In particular, our results are also valid under a label distribution shift since the distribution of the labels need not be the same in the calibration sample and in the new `batch'. The usefulness of the method is illustrated on synthetic and real data examples.

Large Language Models (LLMs) with billions of parameters are known for their impressive predicting capabilities but require lots of resources to run. With their massive rise in popularity, even a small reduction in required resources could have an impact on environment. On the other hand, smaller models require fewer resources but may sacrifice accuracy. In this work, we are proposing an implementation of ``stairs'' assisted greedy generation. It is a modified assisted generation methodology that makes use of a smaller model's fast generation, large model's batch prediction, and "stairs" validation in order to achieve a speed up in prediction generation. Results show between 9.58 and 17.24 percent inference time reduction compared to a stand-alone large LLM prediction in a text generation task without a loss in accuracy.

An increasing number of Enterprise customers are adopting ML/AI as their core transformational pillars, in order to differentiate, increase revenue, reduce costs and maximize efficiency. For many customers ML/AI adoption can be a challenging endeavor not only because of the broad spectrum of applications ML/AI can support, deciding on which one to prioritize can be a challenge, but because moving these solutions into production require a series of security, access and data assessments and features that some ML/AI platforms might not have. This blog post focuses on how to set up your Cloud foundations to cater specifically to the Vertex AI platform and its configuration to be able to set up proper Vertex AI foundations for your future machine learning operations (MLOps) and ML/AI use cases. Explainability is not covered in this blog post, but as a practitioner it is one of the key components for any production ready ML system to take it into account. You can take a look at Vertex Explainable AI for a more in depth approach on feature based explanations, feature attributions methods (Sampled Shapley, Integrated methods and XRAI) and differentiable and non-differentiable models.

As shown by the example in this post, we collected enough images, uploaded them, and annotated them with regions and labels. Then we created datasets and trained a Deepnet to perform Object Detection. We also evaluated the model and used it to predict new images that detected objects accurately. All of these tasks were done on the Dashboard with a few clicks. This is as accessible as it gets in Machine Learning. And just as our motto suggests, BigML has made Object Detection beautifully simple for everyone. Be sure to visit the release page of BigML Object Detection, where you can find more information and documentation.

So far, to showcase BigML's upcoming Object Detection release, we have demonstrated how you can annotate images on the platform, we have covered an example use case to detect cats and dogs and shared how to execute the newly available features by using the BigML Dashboard, as well as another example to build a plant disease detector. In contrast, this installment demonstrates how to perform Object Detection by calling the BigML REST API. Briefly, Object Detection is a supervised learning technique for images that not only shows where an object is in the image, but it also can show where instances of objects from multiple classes are located in the image. Let's jump in and see how we can put it to use programmatically. Before using the API, you must set up your environment variables.

Amazon SageMaker Autopilot is an automated machine learning (AutoML) solution that performs all the tasks you need to complete an end-to-end machine learning (ML) workflow. It explores and prepares your data, applies different algorithms to generate a model, and transparently provides model insights and explainability reports to help you interpret the results. Autopilot can also create a real-time endpoint for online inference. You can access Autopilot's one-click features in Amazon SageMaker Studio or by using the AWS SDK for Python (Boto3) or the SageMaker Python SDK. In this post, we show how to make batch predictions on an unlabeled dataset using an Autopilot-trained model.

However, a model can have hundreds, if not thousands of features. Most feature statistics changes are benign. The problem is not how to detect these changes, but how to know which change actually requires your attention. Real-time machine learning is largely an infrastructure problem. Solving it will require the data science/ML team and the platform team to work together. Both online inference and continual learning require a mature streaming infrastructure.

Amazon Fraud Detector is a fully managed service that makes it easy to identify potentially fraudulent online activities, such as the creation of fake accounts or online payment fraud. Unlike general-purpose machine learning (ML) packages, Amazon Fraud Detector is designed specifically to detect fraud. Amazon Fraud Detector combines your data, the latest in ML science, and more than 20 years of fraud detection experience from Amazon.com and AWS to build ML models tailor-made to detect fraud in your business. After you train a fraud detection model that is customized to your business, you create rules to interpret the model's outputs and create a detector to contain both the model and rules. You can then evaluate online activities for fraud in real time by calling your detector through the GetEventPrediction API and passing details about a single event in each request.

A while ago, I wrote a post about Productionizing Machine Learning Models, where I mentioned strategies for deploying Machine Learning models into production. This article is about one of the widely used approaches -- Batch Prediction using Spark. Using this approach, you can schedule a job to run the predictions at a specific time and output them to database / file systems / streaming / any persistent layer. With each prediction, there comes an overhead of pickling/unpickling the model. Another solution is to use tools like, Airflow and Perfect for batch predictions. Cloud solutions are also available for batch predictions like -- MLFlow and Amazon Sagemaker.

In this Keras tutorial, you'll see how to Extend a Keras Model Generally, you only need your Keras model to return prediction values, but there are situations where you want your predictions to retain a portion of the input. A common example is forwarding unique'instance keys' while performing batch predictions. In this blog and corresponding notebook code, I'll demonstrate how to modify the signature of a trained Keras model to forward features to the output or pass through instance keys. Sometimes you'll have a unique instance key that is associated with each row and you want that key to be output along with the prediction so you know which row the prediction belongs to. You'll need to add keys when executing distributed batch predictions with a service like Cloud AI Platform batch prediction.