lstm neural network
Cellular Traffic Prediction via Deep State Space Models with Attention Mechanism
Ma, Hui, Yang, Kai, Pun, Man-On
Cellular traffic prediction is of great importance for operators to manage network resources and make decisions. Traffic is highly dynamic and influenced by many exogenous factors, which would lead to the degradation of traffic prediction accuracy. This paper proposes an end-to-end framework with two variants to explicitly characterize the spatiotemporal patterns of cellular traffic among neighboring cells. It uses convolutional neural networks with an attention mechanism to capture the spatial dynamics and Kalman filter for temporal modelling. Besides, we can fully exploit the auxiliary information such as social activities to improve prediction performance. We conduct extensive experiments on three real-world datasets. The results show that our proposed models outperform the state-of-the-art machine learning techniques in terms of prediction accuracy.
Bi-Directional Transformers vs. word2vec: Discovering Vulnerabilities in Lifted Compiled Code
McCully, Gary A., Hastings, John D., Xu, Shengjie, Fortier, Adam
Detecting vulnerabilities within compiled binaries is challenging due to lost high-level code structures and other factors such as architectural dependencies, compilers, and optimization options. To address these obstacles, this research explores vulnerability detection by using natural language processing (NLP) embedding techniques with word2vec, BERT, and RoBERTa to learn semantics from intermediate representation (LLVM) code. Long short-term memory (LSTM) neural networks were trained on embeddings from encoders created using approximately 118k LLVM functions from the Juliet dataset. This study is pioneering in its comparison of word2vec models with multiple bidirectional transformer (BERT, RoBERTa) embeddings built using LLVM code to train neural networks to detect vulnerabilities in compiled binaries. word2vec Continuous Bag of Words (CBOW) models achieved 92.3% validation accuracy in detecting vulnerabilities, outperforming word2vec Skip-Gram, BERT, and RoBERTa. This suggests that complex contextual NLP embeddings may not provide advantages over simpler word2vec models for this task when a limited number (e.g. 118K) of data samples are used to train the bidirectional transformer-based models. The comparative results provide novel insights into selecting optimal embeddings for learning compiler-independent semantic code representations to advance machine learning detection of vulnerabilities in compiled binaries.
Forecasting Imports in OECD Member Countries and Iran by Using Neural Network Algorithms of LSTM
Khajoui, Soheila, Dehyadegari, Saeid, Jalaee, Sayyed Abdolmajid
Artificial Neural Networks (ANN) which are a branch of artificial intelligence, have shown their high value in lots of applications and are used as a suitable forecasting method. Therefore, this study aims at forecasting imports in OECD member selected countries and Iran for 20 seasons from 2021 to 2025 by means of ANN. Data related to the imports of such countries collected over 50 years from 1970 to 2019 from valid resources including World Bank, WTO, IFM,the data turned into seasonal data to increase the number of collected data for better performance and high accuracy of the network by using Diz formula that there were totally 200 data related to imports. This study has used LSTM to analyse data in Pycharm. 75% of data considered as training data and 25% considered as test data and the results of the analysis were forecasted with 99% accuracy which revealed the validity and reliability of the output. Since the imports is consumption function and since the consumption is influenced during Covid-19 Pandemic, so it is time-consuming to correct and improve it to be influential on the imports, thus the imports in the years after Covid-19 Pandemic has had a fluctuating trend.
Object Location Prediction in Real-time using LSTM Neural Network and Polynomial Regression
Stojkoviฤ, Petar, Tadiฤ, Predrag
This paper details the design and implementation of a system for predicting and interpolating object location coordinates. Our solution is based on processing inertial measurements and global positioning system data through a Long Short-Term Memory (LSTM) neural network and polynomial regression. LSTM is a type of recurrent neural network (RNN) particularly suited for processing data sequences and avoiding the long-term dependency problem. We employed data from real-world vehicles and the global positioning system (GPS) sensors. A critical pre-processing step was developed to address varying sensor frequencies and inconsistent GPS time steps and dropouts. The LSTM-based system's performance was compared with the Kalman Filter. The system was tuned to work in real-time with low latency and high precision. We tested our system on roads under various driving conditions, including acceleration, turns, deceleration, and straight paths. We tested our proposed solution's accuracy and inference time and showed that it could perform in real-time. Our LSTM-based system yielded an average error of 0.11 meters with an inference time of 2 ms. This represents a 76\% reduction in error compared to the traditional Kalman filter method, which has an average error of 0.46 meters with a similar inference time to the LSTM-based system.
Harnessing LSTM for Nonlinear Ship Deck Motion Prediction in UAV Autonomous Landing amidst High Sea States
Yu, Feifan, Cong, Wenyuan, Chen, Xinmin, Lin, Yue, Wang, Jiqiang
Autonomous landing of UAVs in high sea states requires the UAV to land exclusively during the ship deck's "rest period," coinciding with minimal movement. Given this scenario, determining the ship's "rest period" based on its movement patterns becomes a fundamental prerequisite for addressing this challenge. This study employs the Long Short-Term Memory (LSTM) neural network to predict the ship's motion across three dimensions: longi-tudinal, transverse, and vertical waves. In the absence of actual ship data under high sea states, this paper employs a composite sine wave model to simulate ship deck motion. Through this approach, a highly accurate model is established, exhibiting promising outcomes within various stochastic sine wave combination models.
Predicting Ground Reaction Force from Inertial Sensors
Song, Bowen, Paolieri, Marco, Stewart, Harper E., Golubchik, Leana, McNitt-Gray, Jill L., Misra, Vishal, Shah, Devavrat
The study of ground reaction forces (GRF) is used to characterize the mechanical loading experienced by individuals in movements such as running, which is clinically applicable to identify athletes at risk for stress-related injuries. Our aim in this paper is to determine if data collected with inertial measurement units (IMUs), that can be worn by athletes during outdoor runs, can be used to predict GRF with sufficient accuracy to allow the analysis of its derived biomechanical variables (e.g., contact time and loading rate). In this paper, we consider lightweight approaches in contrast to state-of-the-art prediction using LSTM neural networks. Specifically, we compare use of LSTMs to k-Nearest Neighbors (KNN) regression as well as propose a novel solution, SVD Embedding Regression (SER), using linear regression between singular value decomposition embeddings of IMUs data (input) and GRF data (output). We evaluate the accuracy of these techniques when using training data collected from different athletes, from the same athlete, or both, and we explore the use of acceleration and angular velocity data from sensors at different locations (sacrum and shanks). Our results illustrate that simple machine learning methods such as SER and KNN can be similarly accurate or more accurate than LSTM neural networks, with much faster training times and hyperparameter optimization; in particular, SER and KNN are more accurate when personal training data are available, and KNN comes with benefit of providing provenance of prediction. Notably, the use of personal data reduces prediction errors of all methods for most biomechanical variables.
Generating New Carnatic Music Patterns Using LSTM Neural Networks
Carnatic music is one of 2 Indian classical music forms (the other being Hindustani music). Unlike in Western classical music where compositions are set to a particular key (e.g. Beethoven's Symphony 5 in C minor) and features multiple modes; Carnatic music compositions are mostly set to a single distinct mode, known as a Raga. The Raga Sankarabharanam for example corresponds to the Ionian mode (commonly known as the major scale). Because of this focus on a certain mode, Carnatic music features complex recurrent patterns, that can often be thought of as building on basic patterns, almost like the equivalent of riffs.
Time series Forecasting: Using a LSTM Neural Network to predict Bitcoin prices
The cryptocurrency market is an extremely unstable and complex market, due to cryptocurrencies themselves being extremely volatile assets: their value fluctuates immensely in the span of a few hours. As opposed to stocks, cryptocurrencies hold no intrinsic value. The value of a stock is intrinsically correlated with a company's performance & profitability. For example, on the one hand, Amazon ($AMZN) and Netflix ($NFLX) saw their stock prices soar during the pandemic, due to an increase in online shopping and a higher demand for video streaming services. Recently, Netflix dropped significantly because the quarter's objectives were not met, and the platform had lost 200,000 subscribers.