LLM Fine-Tuning Guide: Full Fine-Tuning, LoRA, Learning Rate, and VRAM

LLM Fine-Tuning Guide: Full Fine-Tuning, LoRA, Learning Rate, and VRAM

From data preparation and tokenizer selection to pretraining, LoRA, RLHF, evaluation, and production monitoring, this guide covers the major stages involved in training an AI model.

Training an artificial intelligence model is not simply a matter of loading a dataset onto a GPU and running a few commands. A successful model requires a measurable objective, legally usable and carefully cleaned data, an architecture suited to the problem, controlled optimization, independent evaluation, and continuous monitoring after deployment.

In large language model development, a mistake in any one of these stages can waste millions of training examples and a significant amount of compute. This guide explains the model development process primarily through the training of large language models. However, fundamental concepts such as dataset splitting, loss functions, overfitting, and evaluation also apply to computer vision, speech, and predictive models.

The goal is not to provide a single fixed recipe. Instead, it is to explain which training approach is appropriate for which problem and to clarify the cost difference between training a model from scratch and adapting an existing model.

In Brief: How Is an AI Model Trained?

First, the target task and success criteria are defined. Data is collected, reviewed for licensing and privacy, cleaned, and divided into training, validation, and test sets. The model generates predictions from the input data. The difference between the prediction and the correct target is measured using a loss function. Backpropagation calculates how each parameter contributed to the error, and an optimization algorithm updates the parameters. This process is repeated under controlled conditions until the model achieves acceptable results in independent tests and safety evaluations.

What Does Training a Model Actually Mean?

A neural network initially contains a large number of numerical parameters. During training, the model generates a prediction for a given input. This prediction is compared with the expected output, and the difference is measured using a loss function. Backpropagation calculates how much each parameter contributed to the error. An optimization algorithm such as AdamW then updates the parameters in small steps intended to reduce the loss.

A training step can be summarized as follows:

data batch ↓
model prediction ↓
loss calculation ↓
backpropagation ↓
parameter update ↓
validation and logging

For language models, the most common pretraining objective is next-token prediction: given the preceding tokens, the model attempts to predict the next token. In image classification, the target may be a class label. In speech recognition, the target is usually text. In regression, it may be a numerical value.

Training is therefore not a single algorithm. It is a general optimization process that uses different objectives depending on the problem being solved.

Choose the Right Method First: Not Every Project Should Start from Scratch

One of the most expensive mistakes in model development is selecting a training method before clearly defining the requirement. A company that wants an assistant capable of answering questions from internal documents does not need to train a new foundation model. Similarly, training on billions of tokens is usually unnecessary when the goal is simply to create a brand-specific writing style.

LoRA reduces adaptation costs by freezing the original model weights and training smaller low-rank matrices. The original study showed that this method could dramatically reduce the number of trainable parameters at GPT-3 scale without introducing additional inference latency. QLoRA further reduces memory requirements by representing the frozen base model in 4-bit precision.

Define the Objective and Success Criteria Before Training

Statements such as "the model should be good at Turkish" or "it should perform well in customer service" are not measurable objectives. Instead, a project should define specific task groups, such as:

• following instructions in Turkish
• summarizing long documents
• answering questions about product policies
• refusing unsafe requests correctly
• generating valid structured JSON
• using tools reliably
• citing retrieved sources accurately

Each task should have clearly defined measurements. Depending on the application, these may include accuracy, source consistency, format validity, latency, inference cost, user success rate, and safety compliance.

It is also important to establish a baseline using a strong existing model before beginning training. After training, the evaluation should measure not only improvements on the target task but also possible losses in general capabilities, performance differences across user groups, and changes in inference cost. Otherwise, a model may improve on a particular benchmark while becoming worse in real-world use.

Training Data: Rights, Quality, and Representation Come Before Quantity

A robust data pipeline includes:

• source inventory
• licensing and usage-right verification
• personal-data removal
• language detection
• quality filtering
• harmful-content policies
• deduplication
• dataset-mixture design

A text being publicly accessible on the internet does not automatically mean that it can be used without restriction for model training. The source license, terms of service, personal-data status, and applicable laws must be evaluated separately.

Duplicate data is not merely a waste of compute. Repeated examples can increase memorization and distort evaluation results when training data overlaps with benchmark or test data. Research by Lee and colleagues showed that removing near-duplicate content from language-model datasets could achieve similar or better validation loss with fewer training steps.

A reliable data pipeline should include the following controls:

• Source provenance: Record where each document came from, when it was collected, and under which license it may be used.
• PII removal: Detect email addresses, phone numbers, identification numbers, confidential records, and other sensitive information before training.
• Deduplication: Detect exact and near-duplicate content at both document and chunk level.
• Language balance: Do not represent low-resource languages exclusively through translated content.
• Test isolation: Prevent evaluation questions, answers, and close derivatives from entering the training pipeline.

Why Are the Tokenizer and Data Mixture Part of the Model Design?

A language model does not process text directly as words. It processes sequences of tokens. When a tokenizer vocabulary is poorly suited to a particular language, the same meaning may require a much longer token sequence. This increases both training and inference costs while reducing the amount of useful content that fits within the model's context window.

For agglutinative languages such as Turkish, tokenizer selection is therefore not a minor implementation detail. It is part of the data and architecture design. A suitable tokenizer should represent common roots, suffixes, word forms, and domain-specific terminology efficiently.

Subword methods such as Byte Pair Encoding generally provide better coverage than a purely word-level vocabulary. However, some structures should remain atomic and should not be divided into multiple subword tokens. These may include:

• ChatML control tokens such as <|im_start|> and <|im_end|>
• role markers
• tool-call delimiters
• end-of-sequence tokens
• structured output markers
• frequently used code expressions
• important domain-specific terms

Preserving these structures as dedicated tokens can help the model learn message boundaries, conversation roles, tool-call formats, and structured generation more consistently.

The data mixture is just as important as the total number of tokens. The proportions of web text, code, mathematics, academic material, conversational data, and domain-specific documents influence which capabilities the model develops. The Llama 3 technical report presents dataset filtering, data-mixture selection through scaling experiments, and training on approximately 15 trillion multilingual tokens as interconnected parts of a single model-development system.

How Should Model Size, Token Budget, and Compute Be Planned?

A larger model is not always a better investment. Scaling-law research by Kaplan and colleagues showed that model performance changes predictably with model size, dataset size, and compute. The later Chinchilla study demonstrated that many large models had been undertrained relative to their parameter count. Under a fixed compute budget, model size and the number of training tokens must be scaled together.

In practice, teams should first run smaller pilot experiments and study their learning curves. Before beginning a large training run, the following variables should be validated:

• batch size
• learning rate
• warmup ratio
• learning-rate schedule
• sequence length
• optimizer configuration
• data-mixture proportions
• checkpoint frequency
• numerical precision

The project should also estimate:

• total token count
• approximate FLOPs
• GPU hours
• checkpoint size
• storage requirements
• network bandwidth
• evaluation cost
• expected failure and restart overhead

Training cost is not limited to GPU rental. It also includes data preparation, failed experiments, storage, engineering work, evaluation, deployment, and continuous inference after the model enters production.

Which Technical Signals Should Be Monitored During Training?

Training loss alone is not sufficient. When training loss decreases but validation loss does not, the model may be overfitting or memorizing the training data. A sudden increase in gradient norm may indicate instability. Reduced token throughput may indicate an infrastructure bottleneck. Different loss patterns across data sources or languages may reveal a problem in the dataset mixture.

At minimum, the following signals should be recorded during training:

• training and validation loss
• perplexity
• task-specific intermediate evaluations
• learning rate
• gradient norm
• weight norm
• numerical overflow and underflow events
• tokens processed per second
• GPU utilization
• memory consumption
• distributed communication time
• checkpoint duration
• sample and token distributions by language and source
• random seed
• code version
• dataset version
• all hyperparameters

Mixed-precision and distributed training can reduce costs, but they also introduce new failure modes. Checkpoints should be created regularly. Corrupted or incomplete checkpoints should be detected automatically, and the training pipeline should be able to resume safely without accidentally repeating or skipping large portions of the dataset.

From a Pretrained Model to an Assistant: SFT, Preference Training, and Safety

A pretrained language model learns statistical patterns in language, but it does not automatically behave like a safe and reliable assistant that follows user instructions. During supervised fine-tuning, or SFT, the model is trained on carefully prepared instruction-and-response examples.

After SFT, developers may use preference ranking, reward modeling, reinforcement learning from human feedback, or other preference-optimization techniques. The InstructGPT study showed that post-training with human feedback could make a smaller model more aligned with user preferences than a larger raw pretrained model.

Direct Preference Optimization, or DPO, introduced an alternative that directly optimizes the policy model from preference pairs without requiring a separate reward model and a complex reinforcement-learning loop.

Regardless of the selected method, preference data should distinguish between:

• factual accuracy
• helpfulness
• safety
• relevance
• style
• instruction compliance

When these criteria are mixed together carelessly, a model may learn that sounding confident or persuasive is more important than being correct.

Evaluation: A Benchmark Score Is Not the Same as Product Quality

A strong evaluation framework generally has three layers:

1. General-purpose benchmarks
2. Private tests designed for the target task
3. Realistic user scenarios

Automated metrics provide scale and reproducibility. Human evaluation captures nuance. Red-team testing searches for vulnerabilities such as:

• prompt injection
• sensitive-data leakage
• unsafe guidance
• hallucinated citations
• unauthorized tool use
• privilege escalation
• failure to follow access-control rules

When benchmark questions have entered the training data, the model may memorize their answers instead of demonstrating genuine reasoning or generalization. Research on data contamination in modern LLM benchmarks has shown that benchmark scores may overestimate real-world generalization performance.

To reduce this risk, evaluation pipelines may use:

• time-based dataset cutoffs
• canary examples
• similarity searches
• unpublished test sets
• independently created adversarial cases

Common Mistakes in Model Training

Using Fine-Tuning to Solve the Wrong Problem

Do not fine-tune a model to solve a current-information problem that could be handled more effectively through retrieval-augmented generation.