LLM interview questions covering pretraining, fine-tuning, sampling, KV-cache, and other modern large language model concepts — with fully explained answers.
Q1. What's the difference between pretraining, supervised fine-tuning, and RLHF/DPO?
Pretraining trains a model on massive amounts of raw text using next-token prediction, producing a model with broad language and world knowledge but no particular ability to follow instructions helpfully. Supervised fine-tuning (SFT) further trains this base model on curated instruction-response examples, teaching it to follow instructions in a helpful format. RLHF/DPO then further aligns the model's behavior with human preferences — RLHF uses reinforcement learning guided by a reward model trained on human preference comparisons; DPO achieves a similar alignment effect more directly, without training a separate reward model, by optimizing directly on preference pairs.
Q2. Explain what a KV-cache is and why it speeds up generation.
During autoregressive generation, each new token's self-attention computation needs the Key and Value vectors of every previous token. Without caching, these would be recomputed from scratch at every single generation step, wastefully repeating identical computation. The KV-cache stores each token's Key and Value vectors once they're computed, so generating each new token only requires computing that one new token's Q/K/V and reusing the cached K/V from all previous tokens — this avoids massive redundant computation and is essential for practical LLM inference speed.
Q3. What's the difference between temperature, top-k, and top-p sampling?
Temperature scales the logits before the softmax — lower temperature makes the distribution sharper (more deterministic, favoring the most likely tokens), higher temperature flattens it (more random/diverse). Top-k restricts sampling to only the \(k\) most probable tokens at each step, discarding the long tail entirely. Top-p (nucleus sampling) instead selects the smallest set of tokens whose cumulative probability exceeds \(p\), which adapts dynamically to how peaked or flat the distribution is at each step — often preferred over a fixed top-k for this reason.
Q4. Why can LLMs hallucinate, and how does RAG help mitigate this?
An LLM generates text by predicting the statistically most plausible next token given context, not by looking up verified facts from a reliable database — it can produce fluent, confident-sounding text that's factually incorrect, especially for specific facts poorly represented or absent in its training data, or that postdate its training cutoff. RAG mitigates this by retrieving relevant, verifiable documents at query time and providing them as context, letting the model ground its answer in actual retrieved evidence rather than relying purely on its parametric (trained-in) knowledge — reducing, though not eliminating, hallucination.
Q5. What is the context window, and why can't it be arbitrarily large?
The context window is the maximum number of tokens (input plus generated output) a model can process in a single sequence. It's limited primarily because self-attention has \(O(n^2)\) computational and memory cost in sequence length — doubling context length quadruples the compute/memory needed for attention alone, making arbitrarily large context windows computationally expensive. Modern long-context techniques (sparse attention, efficient attention implementations like FlashAttention, and specialized positional encoding schemes) push this limit further, but it remains a genuine constraint, not an arbitrary choice.
Q6. What's the difference between fine-tuning and prompt engineering? When would you use each?
Prompt engineering changes only the input given to a fixed, unchanged model — crafting instructions, examples, or context to elicit better behavior, with no training involved. Fine-tuning actually updates the model's weights on task-specific data, changing the model itself. Prompt engineering is faster, cheaper, and reversible — a good first approach for most tasks. Fine-tuning is worth the added cost when a task needs behavior or knowledge that prompting alone consistently can't achieve reliably, or when a very specific, narrow, high-volume use case justifies the investment.
Q7. What is LoRA, and why is it more efficient than full fine-tuning?
LoRA (Low-Rank Adaptation) freezes the original pretrained weights entirely and instead trains a pair of small, low-rank matrices whose product is added to specific weight matrices during the forward pass. Since the low-rank matrices have far fewer parameters than the full weight matrix they modify, this dramatically reduces the number of trainable parameters (and the memory needed for gradients and optimizer state) compared to full fine-tuning, while still achieving comparable task performance for many use cases — making fine-tuning practical on much more modest hardware.
Q8. What does "next-token prediction" actually optimize for during pretraining?
The model is trained to predict the probability distribution over the next token given all previous tokens in a sequence, minimizing cross-entropy loss between its predicted distribution and the actual next token in the training corpus. This simple, self-supervised objective (no manual labeling required — the "label" is just the next word in existing text) is what allows pretraining on massive, naturally-occurring text corpora, and turns out to implicitly force the model to learn grammar, facts, and reasoning patterns needed to predict text well.
Q9. Why does a lower perplexity generally indicate a better language model?
\[ \text{Perplexity} = e^{\text{cross-entropy loss}} \]Perplexity is essentially an exponentiated version of the model's average cross-entropy loss on held-out text, interpretable as roughly "how many equally-likely choices the model is effectively choosing between at each step." A lower perplexity means the model assigns higher probability to the actual, correct next tokens in real text — indicating it has learned the statistical structure of language more accurately. It's a useful, easily computed intrinsic metric, though it doesn't fully capture everything that matters for a model's real-world usefulness (like following instructions or reasoning well).
