Abstract--Smart services are an important element of the smart cities and the Internet of Things (IoT) ecosystems where the intelligence behind the services is obtained and improved through the sensory data. Providing a large amount of training data is not always feasible; therefore, we need to consider alternative ways that incorporate unlabeled data as well. In recent years, Deep reinforcement learning (DRL) has gained great success in several application domains. It is an applicable method for IoT and smart city scenarios where auto-generated data can be partially labeled by users' feedback for training purposes. In this paper, we propose a semi-supervised deep reinforcement learning model that fits smart city applications as it consumes both labeled and unlabeled data to improve the performance and accuracy of the learning agent. To the best of our knowledge, the proposed model is the first investigation that extends deep reinforcement learning to the semi-supervised paradigm. As a case study of smart city applications, we focus on smart buildings and apply the proposed model to the problem of indoor localization based on BLE signal strength. Indoor localization is the main component of smart city services since people spend significant time in indoor environments. Our model learns the best action policies that lead to a close estimation of the target locations with an improvement of 23% in terms of distance to the target and at least 67% more received rewards compared to the supervised DRL model. The rapid development of Internet of Things (IoT) technologies motivated researchers and developers to think about new kinds of smart services that extract knowledge from IoT generated data. The scarcity of labeled data is a main issue for developing such solutions especially for IoT applications where a large number of sensors participate in generating data without being able to obtain class labels corresponding to the collected data. This publication was made possible by NPRP grant# [71113-1-199] from the Qatar National Research Fund (a member of Qatar Foundation). The statements made herein are solely the responsibility of the authors.

The development of smart cities and their fast-paced deployment is resulting in the generation of large quantities of data at unprecedented rates. Unfortunately, most of the generated data is wasted without extracting potentially useful information and knowledge because of the lack of established mechanisms and standards that benefit from the availability of such data. Moreover, the high dynamical nature of smart cities calls for new generation of machine learning approaches that are flexible and adaptable to cope with the dynamicity of data to perform analytics and learn from real-time data. In this article, we shed the light on the challenge of under utilizing the big data generated by smart cities from a machine learning perspective. Especially, we present the phenomenon of wasting unlabeled data. We argue that semi-supervision is a must for smart city to address this challenge. We also propose a three-level learning framework for smart cities that matches the hierarchical nature of big data generated by smart cities with a goal of providing different levels of knowledge abstractions. The proposed framework is scalable to meet the needs of smart city services. Fundamentally, the framework benefits from semi-supervised deep reinforcement learning where a small amount of data that has users' feedback serves as labeled data while a larger amount is without such users' feedback serves as unlabeled data. This paper also explores how deep reinforcement learning and its shift toward semi-supervision can handle the cognitive side of smart city services and improve their performance by providing several use cases spanning the different domains of smart cities. We also highlight several challenges as well as promising future research directions for incorporating machine learning and high-level intelligence into smart city services.

Rebalancing is a critical service bottleneck for many transportation services, such as Citi Bike. Citi Bike relies on manual orchestrations of rebalancing bikes between dispatchers and field agents. Motivated by such problem and the lack of smart autonomous solutions in this area, this project explored a new RL architecture called Distributed RL (DiRL) with Transfer Learning (TL) capability. The DiRL solution is adaptive to changing traffic dynamics when keeping bike stock under control at the minimum cost. DiRL achieved a 350% improvement in bike rebalancing autonomously and TL offered a 62.4% performance boost in managing an entire bike network. Lastly, a field trip to the dispatch office of Chariot, a ride-sharing service, provided insights to overcome challenges of deploying an RL solution in the real world.

Abstract--With the rapid development of knowledge base, question answering based on knowledge base has been a hot research issue. In this paper, we focus on answering singlerelation factoid questions based on knowledge base. We build a question answering system and study the effect of context information on fact selection, such as entity's notable type, outdegree. Experimental results show that context information can improve the result of simple question answering. Question answering (QA) is a classic natural language processing task, which aims at building systems that automatically answer questions formulated in natural language [1]. In recent years, several large-scale general purpose knowledge bases (KBs) have been constructed, including Freebase [2], YAGO [3], DBpedia [4] and Wikidata [5] .

Intrinsically motivated goal exploration algorithms enable machines to discover repertoires of policies that produce a diversity of effects in complex environments. These exploration algorithms have been shown to allow real world robots to acquire skills such as tool use in high-dimensional continuous state and action spaces. However, they have so far assumed that self-generated goals are sampled in a specifically engineered feature space, limiting their autonomy. In this work, we propose to use deep representation learning algorithms to learn an adequate goal space. This is a developmental 2-stage approach: first, in a perceptual learning stage, deep learning algorithms use passive raw sensor observations of world changes to learn a corresponding latent space; then goal exploration happens in a second stage by sampling goals in this latent space. We present experiments where a simulated robot arm interacts with an object, and we show that exploration algorithms using such learned representations can match the performance obtained using engineered representations.

If you've spent any amount of time on the internet over the past year or so, you've inevitably come across the terms machine learning/artificial intelligence and deep learning. In many instances, these buzzwords are being used interchangeably, which has ultimately caused some misunderstandings – particularly for those who are not familiar with the data science sector. With so many different terminologies being tossed around, it's easy to get confused. To clarify things a bit, let's delve into some of the key differences between these emerging technologies. Ever used the online chat feature of a website?

Events If you want to exploit machine learning and AI, the range of technologies and techniques available can appear dizzying. Luckily, there's just one week to go until we open the doors on MCubed, our highly practical two- to three-day dive into machine learning, AI and data science and what they mean for your organisation. Whether you're just making your initial forays into what the technology can do for your organisation, looking to sharpen up your existing development operations, or want to dive deep in key technologies, such as TensorFlow or reinforcement learning, our speakers are just what you're looking for. You can see the full agenda here. We also have some spaces left in our brace of workshops covering developing and deploying machine learning and using the cloud, containers and DevOps to get your project into production.

Rooted in a research team investigating deep learning at the Chinese University of Hong Kong, SenseTime earned early renown by occasionally beating Google and Facebook in image-recognition competitions. Rapidly expanding on the back of massive rounds of VC, it currently supplies face-recognition tech that the Chinese government plans to use to track citizens through its network of 170 million CCTV cameras, and with which state-owned telecoms behemoth China Mobile will monitor its 300 million users. Banks, prisons, airports, police and retailers are already on the SenseTime client list; it may add autonomous driving and augmented reality to that roster soon.

What is the purpose of a theory? To explain why something works. But to also make predictions–testable predictions. Recently we introduced the theory of Implicit Self-Regularization in Deep Neural Networks. Most notably, we observe that in all pretrained models, the layer weight matrices display near Universal power law behavior.

In this approach, we rely on the hallucinations of pre-trained neural networks to fill large holes in images. Deep neural networks use supervised image classification. In supervised image classification, each image has a specific label, and neural networks are learned to approximate the image-label mapping through a cascade of elementary operations. When trained on huge training datasets (millions of images with thousands of labels), deep networks have remarkable classification performance that can occasionally surpass the human accuracy. A Discriminative pre-trained neural network is implemented to guide the image reconstruction where directly the last layer of the deep network is used in the image inpainting problem.