A strong cyber defense system should be able to detect, monitor, and promptly leverage defence mechanisms to the cyber threats including evolving and intelligent attacks Hou et al. [2020], Brundage et al. [2018], Jang-Jaccard and Nepal [2014], Camp et al. [2019]. However, traditional defensive techniques cannot avoid the novel and evolving attacks which can leverage AI technology to plan and launch various attacks. AI-powered attacks can be categorized based on AI-aided and AI-embedded attacks. AI-aided attacks are those that leverage AI to launch the attacks effectively. In this type, the intelligent attackers use AI techniques. However, in AI-embedded attacks, the threats are weaponized by AI themselves such as Deep locker Stoecklin [2018] while in the AI-aided attacks, the attackers could launch various AI-based techniques to detect and recognize the target network, vulnerabilities, and valuable targets Kaloudi and Li [2020]. In fact, they utilize various AI techniques as a tool for various purposes. In Kaloudi and Li [2020], the authors investigated the AI-powered cyber attacks and mapped them onto a proposed framework with new threats including the classification of several aspects of threats that use AI during the cyber-attack life cycle.

The rise of the new generation of cyber threats demands more sophisticated and intelligent cyber defense solutions equipped with autonomous agents capable of learning to make decisions without the knowledge of human experts. Several reinforcement learning methods (e.g., Markov) for automated network intrusion tasks have been proposed in recent years. In this paper, we introduce a new generation of network intrusion detection methods that combines a Q-learning-based reinforcement learning with a deep-feed forward neural network method for network intrusion detection. Our proposed Deep Q-Learning (DQL) model provides an ongoing auto-learning capability for a network environment that can detect different types of network intrusions using an automated trial-error approach and continuously enhance its detection capabilities. We provide the details of fine-tuning different hyperparameters involved in the DQL model for more effective self-learning. According to our extensive experimental results based on the NSL-KDD dataset, we confirm that the lower discount factor which is set as 0.001 under 250 episodes of training yields the best performance results. Our experimental results also show that our proposed DQL is highly effective in detecting different intrusion classes and outperforms other similar machine learning approaches.