# Autonomous Extraction of a Hierarchical Structure of Tasks in Reinforcement Learning, A Sequential Associate Rule Mining Approach

@article{Ghazanfari2018AutonomousEO, title={Autonomous Extraction of a Hierarchical Structure of Tasks in Reinforcement Learning, A Sequential Associate Rule Mining Approach}, author={Behzad Ghazanfari and Fatemeh Afghah and Matthew E. Taylor}, journal={ArXiv}, year={2018}, volume={abs/1811.08275} }

Reinforcement learning (RL) techniques, while often powerful, can suffer from slow learning speeds, particularly in high dimensional spaces. Decomposition of tasks into a hierarchical structure holds the potential to significantly speed up learning, generalization, and transfer learning. However, the current task decomposition techniques often rely on high-level knowledge provided by an expert (e.g. using dynamic Bayesian networks) to extract a hierarchical task structure; which is not… Expand

#### 2 Citations

From Reinforcement Learning Towards Artificial General Intelligence

- Computer Science
- WorldCIST
- 2020

It is proposed that this road is breaking through barriers in Reinforcement Learning and making us closer to Artificial General Intelligence, and views about the current challenges to get us further towards this goal are shared. Expand

Wildfire Monitoring in Remote Areas using Autonomous Unmanned Aerial Vehicles

- Computer Science, Engineering
- IEEE INFOCOM 2019 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)
- 2019

Simulation results confirm that the performance of the proposed system- without the need for inter-coalition communications- approaches that of a centrally-optimized system. Expand

#### References

SHOWING 1-10 OF 28 REFERENCES

Discovering hierarchy in reinforcement learning

- Mathematics, Computer Science
- 2003

This thesis addresses the open problem of automatically discovering hierarchical structure in reinforcement learning with an algorithm, called HEXQ, that automatically decomposes and solves a multi-dimensional Markov decision problem (MDP) by constructing aMulti-level hierarchy of interlinked subtasks without being given the model beforehand. Expand

Automatic Discovery and Transfer of Task Hierarchies in Reinforcement Learning

- Computer Science
- AI Mag.
- 2011

It is demonstrated that causally motivated task hierarchies transfer more robustly than other kinds of detailed knowledge that depend on the idiosyncrasies of the source domain and are hence less transferable. Expand

Automatic discovery and transfer of MAXQ hierarchies

- Computer Science
- ICML '08
- 2008

It is demonstrated empirically that HI-MAT constructs compact hierarchies that are comparable to manually-engineered hierarchies and facilitate significant speedup in learning when transferred to a target task. Expand

Hierarchical Reinforcement Learning with the MAXQ Value Function Decomposition

- Computer Science
- J. Artif. Intell. Res.
- 2000

The paper presents an online model-free learning algorithm, MAXQ-Q, and proves that it converges with probability 1 to a kind of locally-optimal policy known as a recursively optimal policy, even in the presence of the five kinds of state abstraction. Expand

Recent Advances in Hierarchical Reinforcement Learning

- Computer Science
- Discret. Event Dyn. Syst.
- 2003

This work reviews several approaches to temporal abstraction and hierarchical organization that machine learning researchers have recently developed and discusses extensions of these ideas to concurrent activities, multiagent coordination, and hierarchical memory for addressing partial observability. Expand

Extracting bottlenecks for reinforcement learning agent by holonic concept clustering and attentional functions

- Computer Science
- Expert Syst. Appl.
- 2016

Holonic concept clustering and attentional functions are proposed to extract bottleneck states by means of a holonic clustering to extract high-level concepts and the proposed mechanism has a better time complexity and fewer requirements to the designer's help. Expand

Learning MDP Action Models Via Discrete Mixture Trees

- Mathematics, Computer Science
- ECML/PKDD
- 2008

A regression tree algorithm in which each leaf node is modeled as a finite mixture of deterministic functions is introduced, which is approximated via a greedy set cover. Expand

Dynamic abstraction in reinforcement learning via clustering

- Computer Science
- ICML
- 2004

This work considers a graph theoretic approach for automatic construction of options in a dynamic environment and considers building a map that includes preliminary indication of the location of "interesting" regions of the state space, where the value gradient is significant and additional exploration might be beneficial. Expand

AUTOMATIC COMPLEXITY REDUCTION IN REINFORCEMENT LEARNING

- Mathematics, Computer Science
- Comput. Intell.
- 2010

This work derives the knowledge of complexity reduction from partial solutions and provides algorithms for automated dimension reduction in RL and proposes the cascading decomposition algorithm based on the spectral analysis on a normalized graph Laplacian to decompose a problem into several subproblems and then conduct parameter relevance analysis on each subproblem to perform dynamic state abstraction. Expand

AUTOMATED DISCOVERY OF OPTIONS IN REINFORCEMENT LEARNING

- 2003

AI planning benefits greatly from the use of temporally-extended or macroactions. Macro-actions allow for faster and more efficient planning as well as the reuse of knowledge from previous solutions.… Expand