ephizen / wiki

HuggingFace Transformers

The library that made pretrained transformers trivially loadable — from BERT to Llama — with a consistent API across tasks.

Category
LLM & Agent Frameworks
Difficulty
Intermediate
When to use
Loading, fine-tuning, or running any pretrained transformer model in Python.
When not to use
You're serving at high throughput — reach for vLLM or TensorRT-LLM for inference, transformers is slower.
Alternatives
vLLM llama.cpp ONNX Runtime
https://huggingface.co/docs/transformers →

At a glance

FieldValue
CategoryPretrained model library
DifficultyIntermediate
When to useLoading, fine-tuning, and experimenting with models
When not to useHigh-throughput production inference
AlternativesvLLM, llama.cpp, ONNX Runtime

What it is

HuggingFace Transformers gives you AutoModel, AutoTokenizer, and task-specific classes (AutoModelForCausalLM, AutoModelForSequenceClassification, etc.) that load any compatible model from the Hub with a single line. Paired with datasets, accelerate, peft, and trl, it covers the full training → inference loop.

When we reach for it at Ephizen

  • Fine-tuning an open model (Llama, Mistral, Qwen) with PEFT/LoRA.
  • Running a local embedding or reranker model for RAG.
  • Quick evaluation and sanity checks on new models as they’re released.
  • Any time we need exact access to logits, hidden states, or attention weights.

Getting started

from transformers import AutoTokenizer, AutoModelForCausalLM
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-1B-Instruct",
    torch_dtype="auto",
    device_map="auto",
)
inputs = tok("What is RAG?", return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**inputs, max_new_tokens=80)[0]))

Gotchas

  • generate() is slow and memory-hungry; for serving, use vLLM or TGI.
  • Pin the transformers version with your model — new architectures land frequently and break on older versions.
  • Downloading gated models (Llama, Gemma) requires an HF token and license acceptance.
  • For quantized runs, use bitsandbytes, AWQ, or GPTQ — don’t try to hand-roll it.

Related tools