Robots Atlas>ROBOTS ATLAS
Architecture

GenAI

ActivePublished: 2 September 2026Updated: 2 September 2026Published
Key innovation
Shifted machine learning from recognition and classification toward producing new, realistic data (text, images, audio, video, code) by modeling the full distribution of the training data.
Category
Architecture
Abstraction level
Paradigm
Operation level
ModelTrainingInference
Use cases
Text generation and editingImage synthesis and editingCode generationSpeech and music synthesisVideo generationConversational agents and chatbotsSynthetic data generationDrug discovery and molecule design3D design and prototyping

How it works

Generative models learn (usually via self-supervised training) the data distribution from massive corpora by minimizing a loss specific to the model family. LLMs predict the next token autoregressively; diffusion models learn to reverse a gradual noising process to reconstruct a sample from random noise; GANs train a generator and a discriminator in an adversarial game; VAEs encode data into a latent space and decode new samples from it. After training, generation samples from the learned distribution, often conditioned on an input (prompt, class, image). Modern systems are built on foundation models pre-trained at scale and then fine-tuned or prompt-steered for specific tasks.

Problem solved

Classical machine learning focused on discriminative tasks (classification, regression, detection) and could not create new, coherent content. Generative AI addresses data synthesis — automatically producing realistic text, images or audio, filling in missing data, and generating synthetic training sets where real data is scarce or costly.

Implementation

Reference implementations
Hugging Face Transformers
Python · Hugging Face
Hugging Face Diffusers
Python · Hugging Face
Implementation pitfalls
HallucinationsHigh

Generative models can produce plausible-sounding but factually incorrect content.

Fix:Grounding on external sources (e.g. RAG), fact-checking, calibration and output control.
Training-data biasHigh

Models reproduce and amplify biases present in their training data.

Fix:Data curation and balancing, bias evaluation, filtering and fine-tuning.
Difficult quality evaluationMedium

The lack of a single objective quality metric makes generation hard to compare and control.

Fix:Combine automatic metrics (e.g. FID, perplexity) with human evaluation.
High compute costMedium

Training and serving large generative models demand substantial compute and energy.

Fix:Quantization, distillation, sparsification and efficient serving.

Evolution

2013
Variational Autoencoder (VAE)
Inflection point

Kingma and Welling introduce the VAE, a probabilistic latent-variable model for data generation.

VAE (concept)Auto-Encoding Variational Bayes (paper)
2014
Generative Adversarial Networks (GANs)
Inflection point

Goodfellow et al. propose training two networks in an adversarial game, enabling high-quality image synthesis.

GAN (concept)Generative Adversarial Networks (paper)
2017
Transformer architecture
Inflection point

Self-attention becomes the backbone of scalable generative language models.

Transformer (concept)Attention Is All You Need (paper)
2020
Diffusion models (DDPM) and GPT-3
Inflection point

DDPM makes diffusion competitive with GANs, and GPT-3 (175B parameters) demonstrates large-scale few-shot learning.

Diffusion Model (concept)Denoising Diffusion Probabilistic Models (paper)
2022
Mainstream adoption: ChatGPT, DALL·E 2, Stable Diffusion
Inflection point

Public generative products reach mass audiences, triggering a surge of interest and deployment.

Hardware requirements

Primary

Training and inference of large generative models rely on massive matrix multiplication, ideal for GPU tensor cores.

Good fit

TPUs efficiently execute the large-scale matrix operations of generative models.