Artificial intelligence
Skill chaining
Definition
Skill chaining learns or arranges temporally extended skills so that completing one skill places an agent in a state from which the next skill can succeed. Chains can turn sparse end-of-task feedback into a sequence of reachable intermediate behaviours.
Updated
Make the next skill reachable
A temporally extended skill has a policy, a set of states where it can start, and a condition for termination. In a chain, the termination region of one skill leads into the next skill's initiation region. Executing the chain can therefore reach a distant goal through locally manageable behaviours.
Konidaris and Barto introduced skill chaining as a discovery method for continuous reinforcement-learning domains. Their method builds backwards from an end-of-task reward, learning predecessor skills that can reach the initiation set of a skill already in the chain.
A chain is more than a fixed action script
Each skill is a policy that can respond to a range of states, rather than one recorded action sequence. A manipulation chain might approach an object, establish a grasp, carry it, and place it, with each stage handing the robot to the next stage under stated conditions.
Skill chaining belongs to hierarchical reinforcement learning, where decisions can invoke multi-step behaviours. It differs from a behavior tree, which is an execution structure and need not learn either its nodes or their initiation sets.
Interfaces between skills are failure points
A chain works only where the learned initiation and termination sets are accurate. Small errors can accumulate until a later skill starts outside its competence. A backwards-discovered chain may also cover one route to a reward while missing safer or shorter alternatives.
Robotic use adds state-estimation error, contact variation, and actuator limits. Recovery transitions, uncertainty-aware initiation tests, and online replanning may be needed when execution leaves the states represented during training.
Sources
Related terms
Hierarchical reinforcement learning
Hierarchical reinforcement learning organizes learned decision-making into levels, often with a higher-level policy selecting goals or skills and lower-level policies producing actions. The levels can operate over different time scales.
Reinforcement learning
Reinforcement learning trains an agent to choose actions that maximize expected cumulative reward through experience with an environment. In robotics, the learned policy can select movements or higher-level behaviors from observations.
Task and motion planning
Task and motion planning jointly searches over discrete task decisions and continuous robot motions. It connects choices such as which object to move or which grasp to use with geometrically and kinematically feasible trajectories.
Behavior tree
A behavior tree is a hierarchical control structure that selects and switches between an autonomous system's actions according to the status returned by its child nodes. Robots use behavior trees to compose reactive task logic from reusable actions and conditions.