We present the first class of policy-gradient algorithms that work with both state-value and policy function-approximation, and are guaranteed to converge under off-policy training. Our solution targets problems in reinforcement learning where the action representation adds to the-curse-of-dimensionality; that is, with continuous or large action sets, thus making it infeasible to estimate state-action value functions (Q functions). Using state-value functions helps to lift the curse and as a result naturally turn our policy-gradient solution into classical Actor-Critic architecture whose Actor uses state-value function for the update. Our algorithms, Gradient Actor-Critic and Emphatic Actor-Critic, are derived based on the exact gradient of averaged state-value function objective and thus are guaranteed to converge to its optimal solution, while maintaining all the desirable properties of classical Actor-Critic methods with no additional hyper-parameters. To our knowledge, this is the first time that convergent off-policy learning methods have been extended to classical Actor-Critic methods with function approximation.

TD(0) is one of the most commonly used algorithms in reinforcement learning. Despite this, there is no existing finite sample analysis for TD(0) with function approximation, even for the linear case. Our work is the first to provide such results. Existing convergence rates for Temporal Difference (TD) methods apply only to somewhat modified versions, e.g., projected variants or ones where stepsizes depend on unknown problem parameters. Our analyses obviate these artificial alterations by exploiting strong properties of TD(0). We provide convergence rates both in expectation and with high-probability. The two are obtained via different approaches that use relatively unknown, recently developed stochastic approximation techniques.

We propose semi-random features for nonlinear function approximation. The flexibility of semi-random feature lies between the fully adjustable units in deep learning and the random features used in kernel methods. For one hidden layer models with semi-random features, we prove with no unrealistic assumptions that the model classes contain an arbitrarily good function as the width increases (universality), and despite non-convexity, we can find such a good function (optimization theory) that generalizes to unseen new data (generalization bound). For deep models, with no unrealistic assumptions, we prove universal approximation ability, a lower bound on approximation error, a partial optimization guarantee, and a generalization bound. Depending on the problems, the generalization bound of deep semi-random features can be exponentially better than the known bounds of deep ReLU nets; our generalization error bound can be independent of the depth, the number of trainable weights as well as the input dimensionality. In experiments, we show that semi-random features can match the performance of neural networks by using slightly more units, and it outperforms random features by using significantly fewer units. Moreover, we introduce a new implicit ensemble method by using semi-random features.

Artificial Intelligence is a central topic in the computer science curriculum. From the year 2011 a project-based learning methodology based on computer games has been designed and implemented into the intelligence artificial course at the University of the Bio-Bio. The project aims to develop software-controlled agents (bots) which are programmed by using heuristic algorithms seen during the course. This methodology allows us to obtain good learning results, however several challenges have been founded during its implementation. In this paper we show how linguistic descriptions of data can help to provide students and teachers with technical and personalized feedback about the learned algorithms. Algorithm behavior profile and a new Turing test for computer games bots based on linguistic modelling of complex phenomena are also proposed in order to deal with such challenges. In order to show and explore the possibilities of this new technology, a web platform has been designed and implemented by one of authors and its incorporation in the process of assessment allows us to improve the teaching learning process.

An intelligent robot agent based on domain ontology, machine learning mechanism, and Fuzzy Markup Language (FML) for students and robot co-learning is presented in this paper. The machine-human co-learning model is established to help various students learn the mathematical concepts based on their learning ability and performance. Meanwhile, the robot acts as a teacher's assistant to co-learn with children in the class. The FML-based knowledge base and rule base are embedded in the robot so that the teachers can get feedback from the robot on whether students make progress or not. Next, we inferred students' learning performance based on learning content's difficulty and students' ability, concentration level, as well as teamwork sprit in the class. Experimental results show that learning with the robot is helpful for disadvantaged and below-basic children. Moreover, the accuracy of the intelligent FML-based agent for student learning is increased after machine learning mechanism.

The First International Workshop on Rough Sets: State of the Art and Perspectives was held on 2-4 September 1992 in Kiekrz, Poland. To stimulate the discussion, the participation was limited to 40 researchers who are involved in fundamental research in rough set theory and its extensions, logic for approximate reasoning, machine learning, knowledge representation and transfer, and applications of rough set methodology. The workshop focused primarily on applications of the basic idea of the approximate definition of a set and its consequences in other areas of science and engineering. Applications discussed at the workshop included machine learning, medical diagnosis, fault detection, medical image processing, neural net training, database organization, drug research, and digital circuit design. The workshop was the first international meeting of researchers working in this relatively new area. The approximate definition of a set in terms of lower and upper bounds, as introduced in the ...

In fact, we have a knowledge infrastructure already, and it is already immense. AI Mugaztine 7(l): 34- served the most successful work on expert systems: that (today) knowledge comes (mostly) from people. Editor: Mark Stefik Xerox PARC 3333 Coyote Hill Road Palo Alto, California 94304 Workshop on the Foundations of Al: An On-The-Spot Report The NSF and AAAI sponsored Workshop on the Foundations of AI (6-8 February 1986, Las Cruces, New Mexico) is over and, from my perspective at least, it was a very worthwhile event. I am preparing a report that I will send to you in due course. In addition, I noticed that John McCarthy was snapping freely with his camera at the workshop.

A hybrid fuzzy knowledge-based system with crisp and fuzzy rules as well as numerical methods was developed for multiobjective optimization of power distribution system operation. The development process and knowledge-acquisition process for the fuzzy knowledge-based system are described in detail. Fuzzy sets are defined for recent temperature trend, line section loading, transformer aging, voltage-level guidelines, and the degree of desirability of a proposed switching combination. After a heuristic preprocessor proposes a list of switch openings that would seem to reduce system losses, network radiality rules consider whether to open a particular switch and find a corresponding switch that can be closed to maintain radiality. Network parameter rules determine whether the proposed switching combination will violate network integrity.

Foremost Manufacturing Inc. (Union, NJ), a manufacturer of reflectors for lighting fixtures, has adopted a fuzzy logic-based application to produce quotations for customers in less time. The company is using a fuzzy system to produce bids in about 1.5 minutes, compared to an industry average of two weeks. Carnegie Group Inc. (Pittsburgh, PA) has developed a hybrid neural network/expert system for diagnostic situations where signal data and symbolic data must be combined to perform a definitive diagnosis and repair procedure. This technology was developed with funding from the National Science Foundation. Working with experts from Armco Steel (Middletown, OH), Carnegie Group developed a prototype system to diagnose chatter in a coldrolling mill.

Humans have a remarkable capability to perform a wide variety of physical and mental tasks without any measurements and any computations. Familiar examples are parking a car, driving in city traffic, playing golf, cooking a meal, and summarizing a story. In performing such tasks, humans use perceptions of time, direction, speed, shape, possibility, likelihood, truth, and other attributes of physical and mental objects. Reflecting the bounded ability of the human brain to resolve detail, perceptions are intrinsically imprecise. In more concrete terms, perceptions are f-granular, meaning that (1) the boundaries of perceived classes are unsharp and (2) the values of attributes are granulated, with a granule being a clump of values (points, objects) drawn together by indistinguishability, similarity, proximity, and function.