Implicit Graph Alignment
How it works
Objects are represented as graphs, and the task is cast as finding a trajectory of task-relevant alignments between two objects. The method learns a task-agnostic conditional distribution of alignments using an energy-based model over the graph representation of objects; the model jointly infers which parts of the objects are task-relevant and how to align them consistently with the provided demonstrations. Learning this general distribution requires a large and diverse dataset of consistent alignments between instances of same-category objects; the authors obtain it with a correspondence-preserving shape augmentation method that generates arbitrarily many objects aligned in a consistent (but task-agnostic) way. At run time, after receiving a few demonstrations, the model infers a trajectory of alignments and carries out the task by moving one object relative to the other to recreate the inferred alignments.
Problem solved
How to make a robot perform a task on new, previously unseen objects of a class given only a few demonstrations on other objects — without knowledge of the object class and without further training. The large intra-class variety of objects makes it hard to infer the task-relevant relationship between new objects and those in the demonstrations.
Components
Objects are represented as graphs, allowing the method to model task-relevant relationships and alignments between objects.
Learns a task-agnostic conditional distribution of alignments over the object graph representation; it jointly infers the task-relevant parts of objects and how to align them consistently. This is the 'implicit' element of the method.
A correspondence-preserving shape augmentation method that generates a large, diverse dataset of consistently aligned instances of same-category objects, used to train the model.
Implementation
Evolution
Vosylius and Johns introduce Implicit Graph Alignment — few-shot in-context imitation learning via conditional alignment of object graphs.