Goto

Collaborating Authors

 feature engineering process


Application of Explainable Machine Learning in Detecting and Classifying Ransomware Families Based on API Call Analysis

arXiv.org Artificial Intelligence

Ransomware has appeared as one of the major global threats in recent days. The alarming increasing rate of ransomware attacks and new ransomware variants intrigue the researchers to constantly examine the distinguishing traits of ransomware and refine their detection strategies. Application Programming Interface (API) is a way for one program to collaborate with another; API calls are the medium by which they communicate. Ransomware uses this strategy to interact with the OS and makes a significantly higher number of calls in different sequences to ask for taking action. This research work utilizes the frequencies of different API calls to detect and classify ransomware families. First, a Web-Crawler is developed to automate collecting the Windows Portable Executable (PE) files of 15 different ransomware families. By extracting different frequencies of 68 API calls, we develop our dataset in the first phase of the two-phase feature engineering process. After selecting the most significant features in the second phase of the feature engineering process, we deploy six Supervised Machine Learning models: Na"ive Bayes, Logistic Regression, Random Forest, Stochastic Gradient Descent, K-Nearest Neighbor, and Support Vector Machine. Then, the performances of all the classifiers are compared to select the best model. The results reveal that Logistic Regression can efficiently classify ransomware into their corresponding families securing 99.15% overall accuracy. Finally, instead of relying on the 'Black box' characteristic of the Machine Learning models, we present the post-hoc analysis of our best-performing model using 'SHapley Additive exPlanations' or SHAP values to ascertain the transparency and trustworthiness of the model's prediction.


Time Series Forecasting with Supervised Machine Learning

#artificialintelligence

When I first saw a time series forecasting problem I was very confused. Until that moment, I just did some supervised learning predictions on tabular data so I didn't know how to do the forecastings if I didn't have the target values. I decided to write about the machine learning approach of solving time series problems because I believe that these models are very versatile and powerful and they're much more beginner friendly than other statistical approaches. We are going to use Kaggle's Bike Sharing Demand competition dataset because it suites perfectly for this tutorial. Before using any model, it's important to do some time series analysis to understand the data.


Why data preparation is crucial in artificial intelligence (AI) workflows - EDN

#artificialintelligence

For design engineers, an artificial intelligence (AI) workflow encompasses four steps: data preparation, modeling, simulation and testing, and deployment. While all steps are important, many engineers often overemphasize the modeling stage, presuming that it plays the largest role in producing accurate insights. However, since data flows throughout the entire AI workflow, the initial data preparation step is crucial. It ensures that the most useful data is entered into a model. Figure 1 Data is the driving force in the development of an AI workflow.



How to streamline feature engineering for machine learning

#artificialintelligence

For impactful machine learning, data scientists first need clean, structured data. That's where feature engineering comes in -- to refine data structures that improve the efficiency and accuracy of machine learning models. Ryohei Fujimaki, Ph.D., CEO and founder of dotData, a data science platform, said, "Features are, without question, even more critical than the machine learning algorithm itself." Poor quality features will result in a failure of the machine learning algorithm, he said. On the other hand, high-quality features will allow even simple machine learning algorithms like linear regression to perform well.


The Role of Feature Engineering in a Machine-Learning World

#artificialintelligence

Artificial Intelligence(AI) continues to be the next great topic of debate. In fact, Microsoft, Amazon, IBM, Google and Facebook announced on Thursday,Sept.29 the formation of the Partnership on Artificial Intelligence to Benefit People and Society. Within the predictive analytics discipline, though, we tend to use the term "machine learning" as our reference point for artificial intelligence. Much of our thinking in this area has focused around the role of the practitioner or craftsman versus the machine and the concept of machine learning. Yet, machine learning has now evolved into the usage of higher levels of mathematics and computer science with the most recent level being deep learning.