The next Windows 10 update opens the way for the integration of artificial intelligence within Windows applications, directly impacting hundreds of millions of devices from Windows PCs and tablets to IoT Edge devices. The new version of the Windows ML platform allows developers to integrate pre-trained deep-learning models within their applications directly in Visual Studio. The models must be converted into the Open Neural Network Exchange (ONNX) format before importing into VS tools. ONNX is an open-source machine-learning framework launched by Microsoft and Facebook in September 2017, later joined by AWS. ONNX enables portability between neural-network frameworks, making it possible for models trained with tools like Pytorch, Apache MxNet, caffe2 or Microsoft Cognitive Toolkit (CNTK) to be translated to ONNX and later implemented in Windows applications.
At WWDC 2017 Apple announced ways it uses machine learning, and ways for developers to add machine learning to their own applications. Their machine learning API, called Core ML, allows developers to integrate machine learning models into apps running on Apple devices with iOS, macOS, watchOS, and tvOS. Models reside on the device itself, so data never leaves the device. Multiple API calls are already available that application developers can use without having to add any additional models to their app. Examples of such computer vision algorithms are face detection and tracking, landmark detection, and event detection.
Google recently introduced ML KIT, a machine-learning module fully integrated in its Firebase mobile development platform and available for both iOS and Android. With this new Firebase module, Google simplifies the creation of machine-learning powered applications on mobile phones and solves some of the challenges of implementing computationally intense features on mobile devices. ML Kit allows mobile developers to create machine-learning features based on some of the models available in its deep-learning Vision API such as image labeling, OCR and face detection. ML Kit is available both for Android and iOS applications directly within the Firebase platform alongside other Google Cloud based modules such as authentication and storage. ML Kit aims at solving several of the challenges specific to mobile devices which are raised by the computationally intensive operations required for artificial intelligence.
Dave Burke, VP of engineering at Google, announced a new version of Tensorflow optimised for mobile phones. This new library, called Tensorflow Lite, would enable developers to run their artificial intelligence applications in real time on the phones of users. According to Burke, the library is designed to be fast and small while still enabling state-of-the-art techniques. It will be released later this year as part of the open source Tensorflow project. At the moment, most artificial intelligence processing happens on servers of software as a service providers.
Sahil Dua, developer at Booking.com, explained how they have been able to scale machine learning (ML) models for recommending destinations and accommodation to their customers using Kubernetes, at this year's QCon London conference. In particular, he stressed how Kubernetes elasticity and resource starvation avoidance on containers helps them run computationally (and data) intensive, hard to parallelize, machine learning models. Kubernetes isolation (processes not having to compete for resources), elasticity (auto-scaling up or down based on resource consumption), flexibility (being able to quickly try out new libraries or frameworks) and GPU support (albeit Kubernetes support for NVIDIA GPUs is still in alpha, it allows 20x to 50x speed improvements) are key for Booking.com to run a large number of ML models at their scale (around 1.5 million room nights booked daily and 400 million monthly visitors). Each model runs as a stateless app inside a container. The container image does not include the model itself, it is retrieved at startup time from Hadoop.