1) A vision-language model interprets the instruction and the scene, setting the task goal (WHAT to do). 2) A planner/orchestrator decomposes the task into a sequence of skills and retrieves the appropriate validated skills from the library. 3) Each skill performs a specific, repeatable action (HOW), often pre-tested in simulation and reality. 4) Skill outcomes are monitored (measurability), and the orchestrator corrects/continues the plan until the task is complete.
End-to-end VLA models can be unpredictable, hard to validate and error-prone out of distribution. VLS increases reliability, repeatability and testability by basing execution on pre-validated skills.
Interprets the scene and language instruction, setting the task goal (WHAT to do).
Official
A collection of pre-validated, repeatable skills performing specific actions (HOW).
Official
Decomposes the task into skills, retrieves them from the library, composes them into a sequence and monitors execution.
Official
Tasks needing skills outside the library are infeasible; covering the long tail is hard.
Wrong choice or ordering of skills breaks the task despite correct individual skills.
Each skill requires validation (sim + real) and maintenance, which scales expensively.
Vision-Language-Action models (e.g. RT-2) establish the monolithic, end-to-end approach to robot control that VLS differentiates from.
Holiday Robotics promotes the Vision-Language-Skill architecture for the FRIDAY humanoid as a modular, validated alternative to end-to-end VLA (a $105M Series A).
Time complexity: Zalezna od VLM (planowanie) + wybor/wykonanie umiejetnosci. Space complexity: O(VLM + biblioteka umiejetnosci).
VLS effectiveness depends on task coverage by validated skills; library gaps (the long tail) and orchestration errors limit flexibility relative to end-to-end VLA. Validating skills (sim + real) is costly.
How fine-grained/complex individual skills are.
Number and coverage of validated skills.
How the task is decomposed and composed from skills (planning, LLM, rules).
Skill selection depends on the task and scene.
The orchestrator conditionally selects skills from the library for a given task.
VLM planning and skill selection can be parallelized; in-task execution is sequential.
The vision-language (planning) layer benefits from GPU accelerators; skill execution can run on the robot's controllers/edge.
