LLMs Explained: From Training to Security
Large language models are neural networks trained to predict the next word in text. They consist of two files: parameters (weights) and runtime code. Training involves compressing internet text via GPU clusters over weeks; inference runs on a MacBook. Models progress from internet-document generators to assistants via fine-tuning on curated Q&A data. Capabilities include tool use, multimodality, and reasoning, but security challenges—jailbreaks, prompt injection, data poisoning—mirror traditional computing vulnerabilities.
What is a Large Language Model
LLMs are just two files
A large language model like Llama 2 70B consists of a parameters file (weights of the neural network, 140 GB for this model stored as float16) and a runtime file (500 lines of C code with no dependencies) that executes those parameters. Together, these two files form a complete, self-contained package that can run on a MacBook without internet.
Why Llama 2 70B is significant
Llama 2 70B is the largest model in Meta's open-weights Llama series and is considered the most powerful open-weights model available today. Unlike proprietary models like ChatGPT, the weights, architecture, and paper are all publicly released, allowing anyone to download and run it independently.
Training vs. inference complexity
Model inference (running it) is computationally cheap and can happen on consumer hardware. Model training (obtaining the parameters) is vastly more expensive and complex, requiring massive GPU clusters and terabytes of data.
How LLMs Are Trained
Training is internet compression
Training an LLM compresses a large chunk of internet text (roughly 10 terabytes for Llama 2 70B) into model parameters. This is lossy compression—like a ZIP file but imperfect—where the model learns a statistical gestalt of the text rather than memorizing it exactly.
Training resources and cost for Llama 2 70B
Training Llama 2 70B required 6,000 GPUs running for 12 days, costing approximately $2 million. These are specialized computers for heavy computational workloads, not consumer hardware.
Scaling laws: predictable performance
LLM performance on next-word prediction is a smooth, predictable function of only two variables: number of parameters (N) and amount of training text (D). These trends show no signs of topping out, meaning larger models trained on more data reliably perform better without requiring algorithmic breakthroughs.
Core Task: Next-Word Prediction
The fundamental objective
LLMs are trained to predict the next word in a sequence. Given words like 'The cat sat on a', the model outputs a probability distribution over possible next words (e.g., 'mat' with 97% probability). This simple task forces the model to learn vast amounts of world knowledge.
Knowledge emerges from prediction
To predict the next word accurately, the model must learn facts about people, dates, events, and concepts. For example, predicting words after 'Ruth Handler' requires learning who she was, when she lived, and what she did. All this knowledge becomes compressed into the parameters.
Models dream internet documents
During inference, the model generates text by iteratively sampling the next word and feeding it back in. This produces hallucinated documents that mimic the training distribution—fake Wikipedia articles, fabricated ISBN numbers, and invented facts that sound plausible but are not verified.
From Base Model to Assistant
Stage 1: Pre-training on internet text
The model is trained on massive quantities of internet text (tens to hundreds of terabytes). This stage is expensive and happens infrequently (once per year or less) inside large companies. It produces a base model that is a document generator, not a helpful assistant.
Stage 2: Fine-tuning on curated Q&A
The base model is retrained on a smaller, high-quality dataset of question-answer pairs (e.g., 100,000 examples) created by human labelers following detailed instructions. This transforms the model into an assistant that answers questions helpfully. Fine-tuning is much cheaper than pre-training and can be done weekly or daily.
Stage 3 (optional): Reinforcement learning from human feedback
Human labelers compare candidate answers from the model and pick the better one, rather than writing answers from scratch. This comparison data trains the model further via RLHF, improving performance without requiring generation effort from labelers.
Iterative improvement via misbehavior fixing
After deployment, engineers collect examples where the model misbehaves. For each misbehavior, a labeler provides the correct response, which is added to the training data. The model is fine-tuned again, improving on that failure mode. This cycle repeats weekly or daily.
Current Landscape and Rankings
Proprietary vs. open-source models
Proprietary models (GPT-4, Claude, Bard) are closed-source and accessed via web interfaces; they perform best but offer no access to weights. Open-source models (Llama 2, Mistral, Zephyr) have publicly available weights and papers; they perform worse but offer flexibility for fine-tuning and deployment.
Emerging Capabilities and Tool Use
Tool use: extending beyond text generation
Modern LLMs can emit special tokens that trigger external tools—browsers, calculators, Python interpreters, image generators (DALL-E). The model reasons about which tool to use, the tool executes, and the result is fed back to the model. This mirrors how humans solve complex problems by using external resources.
Multimodality: seeing and generating images
LLMs now accept images as input and can generate images via tools like DALL-E. A model can read a sketch of a website and write functioning HTML/JavaScript code for it, or describe what it sees in a photograph.
Speech and audio capabilities
Models can now hear and speak, enabling speech-to-speech conversation. Users can talk to ChatGPT like in the movie 'Her'—a natural conversational interface without typing.
Future Directions
System 2 thinking: slow, deliberate reasoning
Current LLMs only have System 1 (fast, instinctive) thinking. They process words sequentially and cannot pause to reason through a tree of possibilities. Future work aims to add System 2 (slow, deliberate) thinking, allowing models to spend more time on hard problems and convert time into accuracy.
Self-improvement via reward functions
AlphaGo surpassed humans by self-play in a closed sandbox with a clear reward (win/loss). LLMs lack an equivalent simple reward function for general language tasks. Self-improvement in narrow domains (e.g., code generation with automated testing) is possible; general self-improvement remains an open challenge.
Customization and specialization
Instead of one general model for all tasks, the future may include many specialized LLMs. OpenAI's GPTs App Store allows custom instructions and file uploads (retrieval-augmented generation). Future customization may include fine-tuning on domain-specific data, creating expert models for specific tasks.
LLMs as an Operating System
LLM OS analogy
Rather than a chatbot, think of LLMs as a kernel process orchestrating memory, computation, and tools. They coordinate browsing, file access, code execution, image generation, and reasoning—similar to how an OS coordinates hardware and software. The context window is like RAM; the internet/files are like disk storage.
Ecosystem parallels to traditional OS
Just as desktop computing has proprietary (Windows, macOS) and open-source (Linux) operating systems, LLMs are splitting into proprietary (GPT, Claude, Bard) and open-source (Llama, Mistral) ecosystems. The open-source ecosystem is rapidly maturing and chasing proprietary performance.
Security Challenges
Jailbreak attacks: roleplay and encoding
Models can be tricked into harmful behavior via roleplay (e.g., 'act as my deceased grandmother') or encoding (e.g., base64). The model learns safety mostly in English during training, so multilingual or encoded variants bypass refusals. Adversarial suffixes (optimized gibberish) can also jailbreak models.
Prompt injection: hijacking instructions
Attackers embed hidden instructions in images (faint white text), web pages, or documents. When the model processes these, it interprets the hidden text as new instructions and follows them, potentially exfiltrating data or performing unintended actions. Example: a Google Doc with hidden instructions that tricks Bard into stealing user data.
Data poisoning and backdoor attacks
During training on internet text, attackers can inject poisoned documents with trigger phrases. If the model trains on these, the trigger phrase (e.g., 'James Bond') corrupts the model's behavior on any task containing it. This is analogous to a sleeper agent activated by a trigger word.
Notable quotes
A large language model is just two files right. — Andrej Karpathy
These parameters are best kind of thought of as like a zip file of the internet. — Andrej Karpathy
Think of LLMs as kind of like a kernel process of an emerging operating system. — Andrej Karpathy