Given the significant advances in machine learning techniques on mobile devices, particularly in the domain of computer vision, in this work we quantitatively study the performance characteristics of 190 real-world vision transformers (ViTs) on mobile devices. Through a comparison with 102 real-world convolutional neural networks (CNNs), we provide insights into the factors that influence the latency of ViT architectures on mobile devices. Based on these insights, we develop a dataset including measured latencies of 1000 synthetic ViTs with representative building blocks and state-of-the-art architectures from two machine learning frameworks and six mobile platforms. Using this dataset, we show that inference latency of new ViTs can be predicted with sufficient accuracy for real-world applications.

AI on the Edge, or Edge AI, is a concept that people often disregard in their projects without an idea of how powerful it can be. In this blog, I'll explain what edge AI is, its advantages and weaknesses (and how to work around them), its role compared to cloud AI, and when you should definitely go for it. Edge AI refers to when an ML model is deployed to run client-side. This can be on a device the clients regularly use, like mobile phones or desktop computers, often referred to as on-device ML. Alternatively, the model could be deployed on smaller specialized GPUs inside the client's infrastructure, such as Nvidia's Jetson Nano or Google's Coral Dev Board.

Originally published on Towards AI the World's Leading AI and Technology News and Media Company. If you are building an AI-related product or service, we invite you to consider becoming an AI sponsor. At Towards AI, we help scale AI and technology startups. Let us help you unleash your technology to the masses. Today, the two most popular Deep Learning frameworks are PyTorch and TensorFlow.

During its PyTorch Developer Day conference, Meta (formerly Facebook) announced PyTorch Live, a set of tools designed to make AI-powered experiences for mobile devices easier. PyTorch Live offers a single programming language -- JavaScript -- to build apps for Android and iOS, as well as a process for preparing custom machine learning models to be used by the broader PyTorch community. "PyTorch's mission is to accelerate the path from research prototyping to production deployment. With the growing mobile machine learning ecosystem, this has never been more important than before," a spokesperson told VentureBeat via email. "With the aim of helping reduce the friction for mobile developers to create novel machine learning-based solutions, we introduce PyTorch Live: a tool to build, test, and (in the future) share on-device AI demos built on PyTorch."

Due to increasing amounts of data and compute resources, deep learning achieves many successes in various domains. The application of deep learning on the mobile and embedded devices is taken more and more attentions, benchmarking and ranking the AI abilities of mobile and embedded devices becomes an urgent problem to be solved. Considering the model diversity and framework diversity, we propose a benchmark suite, AIoTBench, which focuses on the evaluation of the inference abilities of mobile and embedded devices. AIoTBench covers three typical heavy-weight networks: ResNet50, InceptionV3, DenseNet121, as well as three light-weight networks: SqueezeNet, MobileNetV2, MnasNet. Each network is implemented by three frameworks which are designed for mobile and embedded devices: Tensorflow Lite, Caffe2, Pytorch Mobile. To compare and rank the AI capabilities of the devices, we propose two unified metrics as the AI scores: Valid Images Per Second (VIPS) and Valid FLOPs Per Second (VOPS). Currently, we have compared and ranked 5 mobile devices using our benchmark. This list will be extended and updated soon after.

PyTorch 1.4 has been released, and the PyTorch domain libraries have been updated along with it. The popular open source machine learning framework has some experimental features on board, so let's take a closer look. PyTorch Mobile was first introduced in PyTorch 1.3 as an experimental release. It should provide an "end-to-end workflow from Python to deployment on iOS and Android," as the website states. In the latest release, PyTorch Mobile is still experimental but has received additional features.

Facebook has updated its popular open-source deep-learning library PyTorch. The latest version, PyTorch 1.3, includes PyTorch Mobile, quantization, and Google Cloud TPU support. The release was announced today at the PyTorch Developer Conference in San Francisco. PyTorch Mobile enables an end-to-end workflow from Python to deployment on iOS and Android. Facebook believes it is increasingly important to be able to run machine learning models on devices such as today's supercharged smartphones, as this delivers lower latency and can help preserve data privacy for example through federated learning approaches. Currently, PyTorch Mobile is in the experimental mode and remains under examination and development.