Browsing by Author "Budhraja, Karan Kumar"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Neuroevolution-Based Inverse Reinforcement Learning(2016-01-01) Budhraja, Karan Kumar; Oates, Tim; Computer Science and Electrical Engineering; Computer ScienceMotivated by such learning in nature, the problem of Learning from Demonstration is targeted at learning to perform tasks based on observed examples. One of the approaches to Learning from Demonstration is Inverse Reinforcement Learning, in which actions are observed to infer rewards. This work combines a feature based state evaluation approach to Inverse Reinforcement Learning with neuroevolution, a paradigm for modifying neural networks based on their performance on a given task. Neural networks are used to learn from a demonstrated expert policy and are evolved to generate a policy similar to the demonstration. The algorithm is discussed and evaluated against competitive feature-based Inverse Reinforcement Learning approaches. At the cost of execution time, neural networks allow for non-linear combinations of features in state evaluations. These valuations may correspond to state value or state reward. This results in better correspondence to observed examples as opposed to using linear combinations. This work also extends existing work on Bayesian Non-Parametric Feature construction for Inverse Reinforcement Learning by using non-linear combinations of intermediate data to improve performance. The algorithm is observed to be specifically suitable for a linearly solvable non-deterministic Markov Decision Processes in which multiple rewards are sparsely scattered in state space. Performance of the algorithm is shown to be limited by parameters used, implying adjustable capability. A conclusive performance hierarchy between evaluated algorithms is constructed.Item Programming Agent-Based Models by Demonstration(2019-01-01) Budhraja, Karan Kumar; Oates, Tim; Computer Science and Electrical Engineering; Computer ScienceAgent-based modeling is a paradigm for modeling dynamic systems of interacting agents that are individually governed by specified behavioral rules. Training a model of such agents to produce an emergent behavior by specification of the emergent (as opposed to agent) behavior is easier from a demonstration perspective. While many approaches involve manual behavior specification via code or reliance on a defined taxonomy of possible behaviors, the AMF framework (existing work) generates mapping functions between agent-level parameters and swarm-level parameters which are re-usable once generated. This work builds on that framework by exploring sources of variance in performance, composition of framework output, and the integration of demonstration using images. The demonstrator specifies spatial motion of the agents over time, and retrieves agent-level parameters required to execute that motion. The framework, at its core, uses computationally cheap image processing algorithms. This makes it suitable for time-critical applications. The proposed framework (AMF+) seeks to provide a general solution to the problem of allowing abstract demonstrations to be replicated by agents in a swarm. On solving this problem, the framework has potential usage in a variety of applications such as games, education, surveillance, and search-and-rescue, where the swarm may be controlled remotely. The availability of this software for academic research is therefore also a contribution to the scientific community. The abstraction of demonstration also removes technical requirements for the user. The framework may be used with varied input methodologies, allowing for usage by a wide audience spanning varied demonstration preferences and capabilities. The framework is analyzed in detail for its current and potential capabilities. Our work is tested with a combination of primitive visual feature extraction methods (contour area and shape) and features generated using a pre-trained deep neural network in different stages of image featurization. The framework is also evaluated for its potential using complex visual features for all image featurization stages. Experimental results show significant coherence between demonstrated behavior and predicted behavior based on estimated agent-level parameters specific to the spatial arrangement of agents. The framework is also evaluated using agent-based models or similar systems for comparison.