AI Glossary
VRAM
What is VRAM?
Video Random Access Memory — the memory on a GPU used to store model weights and intermediate computations during inference. A model's VRAM requirement is the primary hardware constraint that determines what consumer or enterprise hardware can run it. Quantisation and model distillation are the primary techniques for reducing VRAM requirements.
Example in practice
An IT team evaluating whether their existing A100 GPU server can run a 70B-parameter model locally would check VRAM requirements first — a 70B model typically requires 140GB of VRAM in 16-bit precision, far exceeding most single-GPU setups.