Goto

Collaborating Authors

 model development pipeline


Develop End-to-End Anomaly Detection System

Mengoli, Emanuele, Yao, Zhiyuan, Wei, Wutao

arXiv.org Artificial Intelligence

Anomaly detection plays a crucial role in ensuring network robustness. However, implementing intelligent alerting systems becomes a challenge when considering scenarios in which anomalies can be caused by both malicious and non-malicious events, leading to the difficulty of determining anomaly patterns. The lack of labeled data in the computer networking domain further exacerbates this issue, impeding the development of robust models capable of handling real-world scenarios. To address this challenge, in this paper, we propose an end-to-end anomaly detection model development pipeline. This framework makes it possible to consume user feedback and enable continuous user-centric model performance evaluation and optimization. We demonstrate the efficacy of the framework by way of introducing and bench-marking a new forecasting model -- named \emph{Lachesis} -- on a real-world networking problem. Experiments have demonstrated the robustness and effectiveness of the two proposed versions of \emph{Lachesis} compared with other models proposed in the literature. Our findings underscore the potential for improving the performance of data-driven products over their life cycles through a harmonized integration of user feedback and iterative development.


Paperspace adds machine learning model development pipeline to GPU service – TechCrunch

#artificialintelligence

Paperspace has always had a firm focus on data science teams building machine models, offering them access to GPUs in the cloud, but the company has had broader ambition beyond providing pure infrastructure, and today it announced a new set of tools to help these teams pass the model off to developers and operations in a smoother way in a multi-cloud or hybrid environment. Co-founder and CEO Dillon Erb says this an attempt to provide a full tool set for data scientists and developers, beyond providing pure GPU power to test and build the models. "Machine learning teams do a lot of GPU work -- and as you know, we've been working with GPUs for a number of years now, and that's one of our specialties. Now what we're doing is taking a kind of agile methodology approach or CI/CD (continuous integration/continuous delivery) for machine learning, and using that to solve much larger scale [machine learning] problems," Erb said. As the company describes it, "The new release introduces GradientCI, the industry's first comprehensive CI/CD engine for building, training and deploying deep learning models…" Erb says the goal is to provide a way to take the model built on top of Paperspace and put it to work in the company faster.