The pipeline consists of four layers: (1) input โ text or speech (ASR); (2) intent understanding โ an LLM or NLU classifier maps the utterance to an action schema and parameters; (3) execution โ tool calls (function calling), database queries (NLโSQL), document generation, or agent orchestration; (4) result return โ natural language text, structured data, a document, or a UI state change. Session context and memory (short-term + long-term) enable multi-turn dialogue and anaphoric references.
Traditional GUIs impose high cognitive load: users must know the application structure, navigation paths, and UI terminology. NLI eliminates this overhead by allowing users to express intent directly. It also addresses the problem of lengthy enterprise application onboarding and accessibility barriers for users with disabilities, mobile users, and hands-free scenarios.
Text field or microphone with ASR converting speech to text.
LLM or NLU model mapping the utterance to an action schema with parameters and context.
Function-calling mechanism that executes the recognized intent against APIs, databases, or services.
Session and long-term memory enabling multi-turn dialogue and anaphoric references.
Output layer: text, structured data, document, or UI update; optionally TTS.
Clarifying-question mechanism and graceful fallback to GUI when intent is ambiguous.
Natural language is ambiguous โ without clarifying questions, the system executes incorrect actions.
In a GUI, available functions are visible to the user; in an NLI, the user has no inherent awareness of system capabilities, making onboarding and worked examples essential.
LLM may invoke a nonexistent function or pass incorrect parameters โ schema validation is required.
"Delete all" must include a confirmation step โ in a GUI this is handled by a confirm dialog, but in an NLI it must be added deliberately.
Every interaction involves an LLM call โ slower and more expensive than a button click.
"Select the third row from the top, second column" โ positional operations are easier with a click.
ASR and NLU performance degrades for dialects, accents, and domain-specific jargon.
Text, speech, or multimodal (text + image + speech).
Number and granularity of tools/functions exposed to the model.
How responses are grounded: RAG, schema structure, ontology, documents.
Degradation policy: clarification, GUI suggestion, refusal.
Generation randomness โ critical for executional vs exploratory tasks.
NLI based on LLMs requires GPU for low inference latency โ critical for interactive systems where expected response time is <500ms.
