Packet routing is one of the fundamental problems in computer networks in which a router determines the next-hop of each packet in the queue to get it as quickly as possible to its destination. Reinforcement learning has been introduced to design the autonomous packet routing policy namely Q-routing only using local information available to each router. However, the curse of dimensionality of Q-routing prohibits the more comprehensive representation of dynamic network states, thus limiting the potential benefit of reinforcement learning. Inspired by recent success of deep reinforcement learning (DRL), we embed deep neural networks in multi-agent Q-routing. Each router possesses an independent neural network that is trained without communicating with its neighbors and makes decision locally. Two multi-agent DRL-enabled routing algorithms are proposed: one simply replaces Q-table of vanilla Q-routing by a deep neural network, and the other further employs extra information including the past actions and the destinations of non-head of line packets. Our simulation manifests that the direct substitution of Q-table by a deep neural network may not yield minimal delivery delays because the neural network does not learn more from the same input. When more information is utilized, adaptive routing policy can converge and significantly reduce the packet delivery time.

Pretraining reinforcement learning methods with demonstrations has been an important concept in the study of reinforcement learning since a large amount of computing power is spent on online simulations with existing reinforcement learning algorithms. Pretraining reinforcement learning remains a significant challenge in exploiting expert demonstrations whilst keeping exploration potentials, especially for value based methods. In this paper, we propose a pretraining method for soft Q-learning. Our work is inspired by pretraining methods for actor-critic algorithms since soft Q-learning is a value based algorithm that is equivalent to policy gradient. The proposed method is based on $\gamma$-discounted biased policy evaluation with entropy regularization, which is also the updating target of soft Q-learning. Our method is evaluated on various tasks from Atari 2600. Experiments show that our method effectively learns from imperfect demonstrations, and outperforms other state-of-the-art methods that learn from expert demonstrations.

"Feint Attack", as a new type of APT attack, has become the focus of attention. It adopts a multi-stage attacks mode which can be concluded as a combination of virtual attacks and real attacks. Under the cover of virtual attacks, real attacks can achieve the real purpose of the attacker, as a result, it often caused huge losses inadvertently. However, to our knowledge, all previous works use common methods such as Causal-Correlation or Cased-based to detect outdated multi-stage attacks. Few attentions have been paid to detect the "Feint Attack", because the difficulty of detection lies in the diversification of the concept of "Feint Attack" and the lack of professional datasets, many detection methods ignore the semantic relationship in the attack. Aiming at the existing challenge, this paper explores a new method to solve the problem. In the attack scenario, the fuzzy clustering method based on attribute similarity is used to mine multi-stage attack chains. Then we use a few-shot deep learning algorithm (SMOTE&CNN-SVM) and bidirectional Recurrent Neural Network model (Bi-RNN) to obtain the "Feint Attack" chains. "Feint Attack" is simulated by the real attack inserted in the normal causal attack chain, and the addition of the real attack destroys the causal relationship of the original attack chain. So we used Bi-RNN coding to obtain the hidden feature of "Feint Attack" chain. In the end, our method achieved the goal to detect the "Feint Attack" accurately by using the LLDoS1.0 and LLDoS2.0 of DARPA2000 and CICIDS2017 of Canadian Institute for Cybersecurity.

Editor's note: This exclusive article from CTech (Meir Orbach) was syndicated with permission. Inc. is acquiring Tel Aviv-based conversational AI startup Bonobo, incorporated as Bonobot Technologies Ltd., the company announced Thursday. The company did not disclose the financial terms of the deal, but one person familiar with the matter who spoke to Calcalist on condition of anonymity put it at $50 million. Founded in 2017 by Barak Goldstein, Efrat Rapoport, Idan Tsitiat, and Ohad Hen, Bonobo develops conversational AI technology designed to extract insights from online customer interactions. The company's technology integrates directly with various sources of customer interaction data, such as voice, chat, video, and email, running AI algorithms to analyze the data.

Three representatives from their respective fields of AI – clinical practice, research and healthcare apps came together for a panel discussion around the current and future developments of AI in healthcare on the second day of the HIMSS Singapore eHealth & Health 2.0 Summit on April 24. The panel consisted of Dr Ali Parsa, Founder and CEO, Babylon Health, Dr Ngiam Kee Yuan, Group Chief Technology Officer, National University Health System, Singapore and Dr Hwang Hee, Chief Information Officer & Associate Professor, Department of Pediatrics, Seoul National University Bundang Hospital, South Korea. Mr Neil Patel, President, Healthbox, Executive Vice President, HIMSS, USA, who was the panel moderator, began the discussion asking the panelists on their thoughts on the current hype cycle of AI broadly and in healthcare. "I think at the general level, we're seeing a much greater update of machine learning and deep learning because of the availability of two things: one is the data that becomes available and secondly, relatively cheaper or cheap computing power that one can get today. That spurred a new revolution and allowed us to use information in ways we never thought possible. But it's also created real challenges – one of the key things I tell every software developer is to ensure that the data is'clean', that's paramount. And I think from that point of view, we always have to think about AI with reference to the data we select," said Dr Ngiam.

Ahead of the much-anticipated IS Auto Asia 2019 this September, we spoke to speak Mahabir Gupta, Solutions & Products Consultant - IoT, Mobility & Data Security at Volvo, to discuss the opportunities and challenges with using artificial intelligence in the automotive industry. Your presentation is on'Artificial intelligence: boon or bane in the automotive industry', what can we expect to learn from your talk? The possibility of creating a thinking machine raises a host of opportunities and ethical issues. These questions relate both to ensuring that such machines do not harm humans and provide value to society. The presentation explores the opportunity of AI and the problem of creating AI more intelligent than humans, and ensuring that they use their advanced technology for good rather than ill.

The fashion industry is forever being accused of using models that have unrealistic standards of beauty -- but in the future, the models themselves could be unreal, too. All of the realistic-looking people in the video below are actually fakes -- dreamed up by a pair of AI developed by researchers from Kyoto University in Japan. The AI were first trained on real-life pictures of humans models. From this, one AI was tasked with repeatedly trying to dream up images of replica models that its counterpart could not distinguish form the real thing. This model-creating technology could one day be used to create fake models for use in advertisements and by the clothing and fashion industries.

For instance, dual arm manipulators have been effectively employed in a diversity of tasks including assembling a car, grasping and transporting an object or nursing the elderly [7]. In those scenarios, the DAR have been expected to behave like a human, which is they should be able to manipulate an object similarly to what a person does [3]. As compared to a single arm robot, the DAR have significant advantages such as more flexible movements, higher precision and greater dexterity for handling large objects [8, 9]. Nevertheless, since the kinematic and dynamic models of the DAR system are much more complicated than those of a single arm robot, it has more challenges to effectively and efficiently control the DAR, where synchronously coordinating the robot arms are highly expected. In order to accurately and stabily track the robot arms along desired trajectories, a number of the control strategies have been proposed. For instance, the traditional methods such as nonlinear feedback control [10] or hybrid force/position control relied on the kinematics and statics [11, 12] have been proposed to simultaneously control both of the arms. In the works [13, 14, 15], the authors have proposed to utilize the impedance control by considering the dynamic interaction between the robot and its surrounding environment while guaranteeing the desired movements. More importantly, robustness of the control performance is also highly prioritized in consideration of designing a controller for a highly uncertain and nonlinear DAR system. In literature of the modern control theory, sliding mode control (SMC) demonstrates a diverse ability to robustly control any system.

Convolutional Neural Network (CNN) is the state-of-the-art for image classification task. Here we have briefly discussed different components of CNN. In this paper, We have explained different CNN architectures for image classification. Through this paper, we have shown advancements in CNN from LeNet-5 to latest SENet model. We have discussed the model description and training details of each model. We have also drawn a comparison among those models.

Naive Bayes estimator is widely used in text classification problems. However, it doesn't perform well with small-size training dataset. We propose a new method based on Naive Bayes estimator to solve this problem. A correlation factor is introduced to incorporate the correlation among different classes. Experimental results show that our estimator achieves a better accuracy compared with traditional Naive Bayes in real world data.