There are great interests as well as many challenges in applying reinforcement learning (RL) to recommendation systems. In this setting, an online user is the environment; neither the reward function nor the environment dynamics are clearly defined, making the application of RL challenging. In this paper, we propose a novel model-based reinforcement learning framework for recommendation systems, where we develop a generative adversarial network to imitate user behavior dynamics and learn her reward function. Using this user model as the simulation environment, we develop a novel DQN algorithm to obtain a combinatorial recommendation policy which can handle a large number of candidate items efficiently. In our experiments with real data, we show this generative adversarial user model can better explain user behavior than alternatives, and the RL policy based on this model can lead to a better long-term reward for the user and higher click rate for the system.

Get used to people in your house saying "What?" a lot. If you just got a new smart speaker from Amazon or Google, you'll be barking commands out loud, and people around you may wonder what's going on. You'll be engaging in the next step of computing, voice style, using Google or Amazon speakers (or perhaps Siri in Apple's HomePod or Microsoft's Cortana in Invoke from Harman Kardon) to ask for a specific music selection or playlist, the weather, latest news or podcast, the answer to a math problem or how to spell a word. The smart speaker market is dominated by Amazon and Google, so we'll focus here on those two. They were among the most heavily marketed during the holidays, with massive discounts for the entry-level Echo Dot and Google Home Mini at under $25, so we expect them to be under many trees.

Technology behind photography-making tools has hit a plateau. Traditional digital cameras and the associated lenses have already reached an impressive quality and physical upgrades will not dramatically change this state of fact. The next step is code, or AI, in a broad sense. Smartphones are fair competition for DSLR cameras as they try to compensate the drawbacks of tiny physical sensors, with the success we know. The increasingly sophisticated and professional applications of smartphones have achieved an almost similar quality in the technical variables of photography: exposure speed, white balance and color, ISO sensitivity, etc. However, the phone is not able to compete with the combination of the objectives and sensors of a professional camera.

This post was done in partnership with Wirecutter. When readers choose to buy Wirecutter's independently chosen editorial picks, Wirecutter and Engadget may earn affiliate commission. After 45 hours of research and test flying 14 models, we think the DJI Mavic 2 Pro is the best drone for aspiring aerial photographers and videographers thanks to its high-end camera, autonomous obstacle avoidance, long battery life, and portability. Pilots of all skill levels will find it to be exceptionally reliable and easy to fly. The Mavic 2 Pro features a Hasselblad-branded camera (DJI bought a majority stake in the camera brand in 2017), which captures 20-megapixel photographs and 4K videos that look more colorful than those captured by the competition. Its ability to sense and avoid obstacles in all directions and steadily hold its position even in moderate winds lets you focus on your cinematography instead of worrying about keeping the drone steady. It also features DJI's smart-flight modes like ActiveTrack, which directs the drone to autonomously follow and film a subject while still avoiding obstacles. Its 31-minute battery life means you don't have to land for a battery swap as often as other drones, and at 8.4 by 3.6 by 3.3 inches folded and 2 pounds, you can take the Mavic 2 Pro almost anywhere--it fits exceptionally well in our top pick for drone backpacks. It's also compatible with the DJI Goggles FPV headset we recommend. The Mavic 2 Zoom looks and flies identical to the Mavic 2 Pro, but it trades out the Hasselblad camera in favor of a different camera that can zoom 2 times optically and 2 times digitally (with software that avoids losing detail), for up to 4x usable "lossless" zoom.

The last 12 months were filled with ground-breaking products. We saw in-display fingerprint sensors break into the mainstream, the world's first foldable displays in action, and crazy leaps in cameras and computational photography, all on devices small enough to fit in a bag (or a pocket). But among all the great things we got, there were a few things that didn't make it--products and features that were promised in 2018 but won't be arriving until 2019 (if at all): At this point AirPower is the stuff of legend. Announced alongside the iPhone X in 2017, it was supposed to be the first pad capable of charging three devices at once (namely an iPhone, Apple Watch and AirPods). In fact, the only indication that Apple is even working on it was an accidental mention in the quick start guide inside the iPhone box this year.

Another year over, another year to look forward to. But in the context of technology, the prospect on either side of the timeline is no longer as exciting as it was. The consumer tech industry seems to be on a self-imposed consolidation mode, and life-changing innovation is becoming harder to come by. Still, 2018 will be remembered as the year in which everything from smartphones to speakers becomes more powerful and smarter, as they did last year and as they are supposed to. Interestingly, the tech giants have clearly realised that if great innovation cannot be put into users' hands, it's better to improve their overall product experience.

SAN FRANCISCO - Millions of users of Amazon's Echo speakers have grown accustomed to the soothing strains of Alexa, the human-sounding virtual assistant that can tell them the weather, order takeout and handle other basic tasks in response to a voice command. So a customer was shocked last year when Alexa blurted out: "Kill your foster parents." Alexa has also chatted with users about sex acts. She gave a discourse on dog defecation. And this summer, a hack Amazon traced back to China may have exposed some customers' data, according to five people familiar with the events.

Recommender systems have recently attracted many researchers in the deep learning community. The state-of-the-art deep neural network models used in recommender systems are typically multilayer perceptron and deep Autoencoder (DAE), among which DAE usually shows better performance due to its superior capability to reconstruct the inputs. However, we found existing DAE recommendation systems that have similar implementations on similar datasets result in vastly different parameter settings. In this work, we have built a flexible DAE model, named FlexEncoder that uses configurable parameters and unique features to analyse the parameter influences on the prediction accuracy of recommender systems. This will help us identify the best-performance parameters given a dataset. Extensive evaluation on the MovieLens datasets are conducted, which drives our conclusions on the influences of DAE parameters. Specifically, we find that DAE parameters strongly affect the prediction accuracy of the recommender systems, and the effect is transferable to similar datasets in a larger size. We open our code to public which could benefit both new users for DAE -- they can quickly understand how DAE works for recommendation systems, and experienced DAE users -- it easier for them to tune the parameters on different datasets.

Abstract--Next generation networks are expected to be ultra dense and aim to explore spectrum sharing paradigm that allows users to communicate in licensed, shared as well as unlicensed spectrum. Such ultra-dense networks will incur significant signaling loadat base stations leading to a negative effect on spectrum and energy efficiency. To minimize signaling overhead, an adhoc approachis being considered for users communicating in unlicensed and shared spectrum. Decision of such users need to completely decentralized as: 1) No communication between users and signaling from the base station is possible which necessitates independent channel selection at each user. Collision occurs when multiple users transmit simultaneously on the same channel, 2) Channel qualities may be heterogeneous, i.e., they are not same across all users, and moreover are unknown, and 3) The network could be dynamic where users can enter or leave anytime. We develop a multi-armed bandit based distributed algorithm for static networks and extend it for the dynamic networks. The algorithms aim to achieve stable orthogonal allocation (SOC) in finite time and meet the above three constraints with two novel characteristics: 1) Low complex narrowband radio compared to wideband radio in existing works, and 2) Epoch-less approach for dynamic networks. We establish convergence of our algorithms to SOC and validate via extensive simulation experiments. Index Terms--Multi-player multi-armed bandit, ad-hoc networks, dynamicnetworks, distributed learning. I. INTRODUCTION Next generation wireless networks such as 5G aim to offer the wide range of new services such as enhanced local broadband, high-speed multimedia, mission-critical control, private networks such as Industrial IoT and enterprise [1] via spectrum sharing. Such networks with diverse service requirements are expected to greatly enhance user experience [1]. Recently, 3GPP proposed a new radio (NR) based heterogeneous networksconsisting of base stations of various sizes. Compared to existing networks, NRs can operate not only in licensed spectrum but also in the shared (2.3 GHz/ 3.5 GHz) as well as unlicensed spectrum (2.4 GHz / 5-7 GHz / 57-71 GHz). Such network opens up many interesting challenges such as resource allocation, dynamic and contextaware networkadaptation, and in-depth knowledge discovery in the complex environment for which machine learning and artificial intelligence frameworks offer novel solutions [1-4]. The next generation networks are envisioned to work on the principle of separate signaling (large base station) and data infrastructure (small base stations) which allows adaptation of data network to the current traffic situation while maintaining the coverage. These networks will be ultra dense with very high peak rate but relatively lower expected traffic per network node [1].

In 1994, soon after Jeff Bezos incorporated what would become Amazon, the entrepreneur briefly contemplated changing the company's name. The nascent firm had been dubbed "Cadabra," but Bezos wanted a less playful, more accurate alternative: "Relentless." Twenty-four years later, perhaps no adjective better describes Bezos' empire than the name he once wanted to give it. The company is known as the "everything store," but in its dogged pursuit of growth, Amazon has come to dominate more than just ecommerce. Amazon is a fashion designer, advertising business, television and movie producer, book publisher, and the owner of a sprawling platform for crowdsourced micro-labor tasks.