Concept drift - a change, either sudden or gradual, in the underlying properties of data - is one of the most prevalent challenges to maintaining high-performing learned models over time in autonomous systems. In the face of concept drift, one can hope that the old model is sufficiently representative of the new data despite the concept drift, one can discard the old data and retrain a new model with (often limited) new data, or one can use transfer learning methods to combine the old data with the new to create an updated model. Which of these three options is chosen affects not only near-term decisions, but also future needs to transfer or retrain. In this paper, we thus model response to concept drift as a sequential decision making problem and formally frame it as a Markov Decision Process. Our reinforcement learning approach to the problem shows promising results on one synthetic and two real-world datasets.

The project started as the brainchild of Elad Liebman, a Ph.D. student in computer science who also has a degree in music composition. The program that he, Saar-Tsechansky, and Peter Stone, a professor of computer science, designed runs a series of feedback loops. It tries out a song, the listener rates it, and the program heeds that rating in choosing the next song. "Then you alter the model accordingly," says Liebman. The program adapts to the listener's mood, considering not only which songs he or she will enjoy, but also in what order.

My main research motivation is to develop complete autonomous agents that interact with people socially. For an agent to be social with respect to humans, it needs to be able to parse and process the multitude of aspects that comprise the human cultural experience. That in itself gives rise to many fascinating learning problems. I am interested in tackling these fundamental problems from an empirical as well as a theoretical perspective. Music, as a general target domain, serves as an excellent testbed for these research ideas. Musical skills---playing music (alone or in a group), analyzing music or composing it---all involve extremely advanced knowledge representation and problem solving tools. Creating "musical agents"---agents that can interact richly with people in the music domain---is a challenge that holds the potential of advancing social agents research, and contributing important and broadly applicable AI knowledge. This belief is fueled not just by my background in computer science and artificial intelligence, but also by my deep passion for music as well as my extensive musical training. One key aspect of musical intelligence which hasn’t been sufficiently studied is that of sequential decision-making. My thesis strives to answer the following question: How can a sequential decision making perspective guide us in the creation of better music agents, and social agents in general? More specifically, this thesis focuses on two aspects of musical intelligence: music recommendation and multiagent interaction in the context of music.

You don't see self-driving cars taking over American cities yet, but robotic tractors already roar through our corn and soybean farms, helping to plant and spray crops. They also gather huge troves of data, measuring moisture levels in the soil and tracking unruly weeds. Combine that with customized weather forecasts and satellite imagery, and farmers can now make complex decisions like when to harvest--without ever stepping outside. These tools are part of a new trend, known as "precision agriculture," that is transforming how we grow crops. Using everything from sensors on combines to drones equipped with infrared cameras that monitor plant health, service providers--ranging from Monsanto and DuPont to startups--take data from the fields, upload it to the cloud, crunch it, and provide farmers with advice on how to run their operations.