Natural Language Interface

A Large Language Model (LLM) is a class of machine learning models based on the Transformer architecture, trained on large text datasets via autoregressive language modeling (next-token prediction). These models have billions of parameters and can generate coherent text, answer questions, write code, translate languages, and perform many other language-cognitive tasks without task-specific fine-tuning. The term covers models such as GPT, LLaMA, Gemini, Claude, and Mistral. Most modern LLMs are instruction-tuned (SFT + RLHF) after the pre-training phase.

Tool-augmented LLM is an architectural pattern in which a large language model is equipped with access to one or more external tools that it can invoke during inference by generating structured function-call or API-call outputs. The model learns when and how to call tools by producing special tokens or structured output (e.g., JSON function calls) that are intercepted by a host runtime, executed against the tool, and whose results are returned to the model as new context for continued generation. The canonical formalization appeared in the Toolformer paper (Schick et al., Meta AI, 2023), which demonstrated that LLMs can learn to self-supervise their own tool-use through API call annotation without requiring large labeled datasets. Toolformer showed that models trained this way can decide which tools to call, when, and with which arguments, and that tool use substantially improves performance on tasks requiring fresh information, arithmetic, multilingual lookup, and question answering. The pattern encompasses several mechanisms: (1) in-context tool specification, where tool interfaces are described in the system prompt or context (JSON Schema, OpenAPI, natural language); (2) function calling APIs, where the model produces structured output matched to a defined schema and the host dispatches the call; (3) ReAct-style interleaving, where the model alternates reasoning traces with tool-use observations; and (4) parallel tool calling, where the model emits multiple tool calls simultaneously to be executed concurrently. Key implementations include OpenAI function calling (GPT-4, June 2023), Anthropic tool use (Claude, 2023), Google Gemini function calling, and the Model Context Protocol (MCP, 2024) which standardizes tool server connectivity.

An AI Agent (autonomous agent) is a single, autonomous system based on an AI model — most often an LLM — that dynamically directs its own process and tool usage to accomplish a given goal. In Anthropic's definition (December 2024), an agent is a system in which an LLM independently controls its actions, in contrast to a workflow, where LLMs and tools are orchestrated through predefined code paths. An AI Agent is the concrete executable artifact of the Agentic AI paradigm — analogous to how a microservice is an instance of the microservice paradigm. A single agent has a clearly defined goal, access to a set of tools (web search, code execution, file operations, APIs, MCP), memory (in-context and optionally external), a control loop (perceive → reason → act → observe), and termination conditions (goal achievement, max_steps, escalation). The agent starts work from a command or interactive discussion with a human; once the task is clarified, it operates independently, optionally returning for further information or approval. During execution it obtains "ground truth" from the environment after each step (tool results, code execution) and may pause at checkpoints. In practice, an AI Agent is typically just an LLM using tools in a loop based on environmental feedback — the implementation is often simpler than a framework, but requires care in designing the agent-computer interface (ACI) and tool documentation. AI Agent should be distinguished from related concepts: Agentic AI is the paradigm (class of systems), an AI Agent is an instance (concrete actor); a Multi-Agent System is a collective of multiple cooperating agents; a Workflow is a predefined orchestration of LLMs without decisional autonomy.

Agentic AI denotes an architectural transition from single-turn, stateless generative models toward goal-directed systems capable of autonomous perception, planning, action, and adaptation through iterative control loops. An agentic system wraps a large language model in a runtime that gives the model access to tools (web search, code execution, APIs, file I/O), persistent memory, and feedback from prior steps. The model then decides dynamically which tools to call, in what order, and whether to loop or stop, rather than following a predefined code path. Two primary system types are commonly distinguished: (1) Workflows, in which LLMs and tools are orchestrated through predefined code paths, and (2) Agents, in which the LLM itself directs its process and tool usage dynamically. Both can be composed into multi-agent systems where specialized agents collaborate, with one acting as orchestrator and others as subagents. Key design patterns identified by Anthropic (2024) include prompt chaining, routing, parallelization, orchestrator-workers, and evaluator-optimizer loops. Andrew Ng's 2024 taxonomy describes four foundational patterns: Reflection, Tool Use, Planning, and Multi-Agent Collaboration. Formal frameworks model agentic control loops as Partially Observable Markov Decision Processes (POMDPs). The control loop is: perceive state → reason/plan → select action → execute tool → observe result → update state → repeat. Agentic systems introduce risks not present in single-turn models, including hallucination in action, prompt injection through observed content, infinite loops, reward hacking, and tool misuse.

Agents as a Service (AaaS) is a software delivery model in which a vendor provides the customer with an autonomous AI agent that performs concrete business tasks, instead of a traditional human-operated application. The term was publicly introduced on March 25, 2026, by Sierra co-founders Bret Taylor and Clay Bavor in a blog manifesto announcing their Ghostwriter agent as their own realization of this paradigm. Unlike Software as a Service (SaaS), where customers buy access to an interface (menus, form fields, tables) and perform the work themselves through clicks, in AaaS the customer defines a desired outcome in natural language, and the vendor delivers an agent that builds and improves production agents or performs the work directly. The defining property is full autonomy: the agent has access to data, tools, a sandboxed test environment, and the deployment pipeline, while the human acts as a supervisor approving changes. The key technical enabler is the agent harness — scaffolding of tools, memory, planning, and task context — combined with refactoring the platform into headless infrastructure that an agent can invoke programmatically rather than navigating through a UI. The work cycle includes analyzing interactions, identifying improvement opportunities, validating in a sandbox, and preparing for review — which Sierra calls an "agent assembly line." AaaS is tightly coupled with the Agentic AI paradigm (its technical foundation) and outcome-based billing models (its commercial superstructure).

Speech-to-speech AI (S2S AI) denotes a class of systems and architectures that take spoken audio as input and generate spoken audio as output, spanning conversational agents, real-time spoken language translation, voice conversion, and expressive speech interaction. Two principal architectural paradigms exist: 1. Cascade (pipeline) architecture: The input speech is processed by an Automatic Speech Recognition (ASR/STT) module producing a text transcript, which is then passed to a language model (LLM) or NLP module for understanding and response generation, and finally synthesized into output speech by a Text-to-Speech (TTS) module. This approach offers modularity, interpretability, and ease of debugging, and each component can be independently optimized with abundant unimodal data. Its main limitations are accumulated latency across pipeline stages (typically 2–4 seconds end-to-end), error propagation between stages, and loss of non-textual information (prosody, emotion, speaker identity) at the speech→text transcription step. 2. Direct (end-to-end) architecture: A single model processes audio input representations directly to audio output, bypassing the intermediate text stage. Early examples include Translatotron (Google, 2019), the first sequence-to-sequence model for direct speech-to-speech translation, which took source mel spectrograms as input and produced target language mel spectrograms as output via an attentive encoder-decoder. More recent conversational S2S models (Moshi by Kyutai Labs, 2024; LLaMA-Omni; Ultravox) extend this to real-time spoken dialogue by conditioning large language model backbones on audio tokens or embeddings. The direct approach preserves paralinguistic information and reduces latency (sub-1 second time-to-first-audio in best-case deployments), but requires paired speech data for training and currently has more limited fine-grained control compared to cascade systems. A hybrid class combines LLM-based reasoning with tightly integrated or low-latency STT/TTS, achieving latency in the 250–500 ms range while retaining some interpretability. Key differentiating dimensions include: (a) presence or absence of an intermediate text representation; (b) whether the model is trained end-to-end or composed of independently trained components; (c) half-duplex (turn-based) vs. full-duplex (simultaneous send/receive) operation; (d) the approach to voice activity detection and barge-in handling. Notable end-to-end S2S systems documented in primary technical literature: Translatotron (Jia et al., 2019, speech-to-speech translation); Translatotron 2 (Jia et al., 2022); AudioPaLM (Google, 2023); Moshi (Kyutai Labs, 2024, real-time full-duplex dialogue); LLaMA-Omni (2024); GPT-4o Realtime (OpenAI, 2024); Gemini 2.5 Flash Live (Google, 2025).

Retrieval-Augmented Generation (RAG) was introduced by Lewis et al. (2020) as a general-purpose fine-tuning recipe combining pre-trained parametric memory (a seq2seq language model, specifically BART in the original paper) with non-parametric memory (a dense vector index of Wikipedia, accessed via Dense Passage Retrieval, DPR). In the original formulation, both the retriever and the generator are fine-tuned end-to-end: given an input query x, the retriever retrieves top-k documents z from the corpus, and the generator produces an output y conditioned on x and z. Two formulations were proposed: RAG-Sequence (the same retrieved documents condition the full output sequence) and RAG-Token (different documents may be used per generated token, marginalized during generation). In widespread contemporary usage (post-2022, with the growth of LLM applications), 'RAG' has expanded to describe a broader class of retrieve-then-generate pipelines, typically with a frozen LLM, a vector store containing pre-computed dense embeddings of document chunks, and a retrieval step that fetches top-k relevant chunks based on embedding similarity to the query. The retrieved chunks are appended to the prompt as context before the LLM generates a response. This non-trainable pipeline variant is technically distinct from the original Lewis et al. formulation but is the dominant practical interpretation of RAG as of 2023–2025. The canonical modern RAG pipeline consists of an offline indexing phase (document chunking, embedding computation, storage in a vector database) and an online query phase (query embedding, approximate nearest neighbor search, context-augmented generation). Key design decisions include: chunk size and overlap, embedding model choice, retrieval strategy (dense, sparse/BM25, or hybrid), number of retrieved documents k, and context integration method (prepend to prompt, cross-attention injection, or fusion-in-decoder). RAG addresses two fundamental limitations of parametric-only LLMs: the knowledge cutoff problem (inability to access post-training information) and hallucination (generation of factually incorrect content). However, RAG introduces its own failure modes, including retrieval of irrelevant or misleading context and the LLM's susceptibility to being distracted by retrieved content that contradicts its parametric knowledge.