Research and models20:08
RAG or fine-tuning LLM in a week, before you spend 15,000 PLN
From 500 prompt-response pairs to 50,000 examples: thresholds, QLoRA costs, API option, and a one-week test that decides between RAG and fine-tuning.

Listen to the article summary
Synthetic voice.Treat this as a free practical guide: useful to get started and talk to your team, not sufficient on its own for critical decisions. Monday morning: a 180-person law firm with an archive of 900,000 contracts and legal opinions. A cloud vendor pitches an offer: fine-tuning an LLM to your terminology, starting at 15,000 PLN. Before anyone approves that budget, a single week of internal evaluation will show whether changing model weights makes sense at all, instead of simply improving search.

Fine-tuning is not document memorization. Training updates model weights, so the system adapts its tone and output formatting, but it must still retrieve facts externally. If over 90% of tasks involve finding clauses, quoting, and summarizing existing text, RAG is usually enough: the model receives the document in context and cites the source. Personal data processing requirements are defined by the GDPR, while documentation requirements for high-risk systems are set by the AI Act.
What you can do this week
Step one: select one repeatable process where output format matters more than retrieval: a contract addendum draft, a monthly audit report, or a technical proposal description. This takes 2 hours of a department director's time. Step two: collect 80 prompt-response pairs from the last three months, stripped of personal data; 6 hours of work for one analyst. Step three: prepare 20 test cases and a clear rubric: formatting accuracy, source correctness, revision time; 2 hours for a QA specialist. Step four: run a baseline RAG pipeline on these 20 cases, using the existing document base and an open-source vector engine; 8 hours of IT engineering. Step five: calculate token counts from the 80 pairs and project QLoRA and API costs to 500 pairs; 1 hour of the CTO's time. Total: 19 hours of work, 0 PLN in new software spend.
What to avoid doing right away
The initial impulse is to send the entire archive to an API for fine-tuning. This is usually a mistake: personal data and trade secrets enter an external system, and removing individual records from model weights post-training is technically difficult. With personal data, this often requires a data protection impact assessment, as outlined by UODO. Fine-tuning does not create factual memory: a fine-tuned model can still hallucinate facts, just in your company's tone. Renting an A100 GPU before testing RAG is a premature expense, and the ongoing cost of maintaining model versions often exceeds the GPU spend itself. Promising that a model will "know the documents" leads to disappointment, because that is the role of a vector database. Starting with 50,000 examples before 500 pairs pass validation creates data debt, not competitive advantage.

How to verify it works after two weeks
After two weeks of RAG tests on 20 documents, track a single metric: the percentage of documents accepted without factual revisions. If this metric reaches 80% or higher and every response cites a valid source, fine-tuning is currently unjustified: retrieval solved the problem, not a new model. If the rate falls below 60%, the issue lies in the data, not the architecture; clean up the source repository and standardize documents before attempting any model training.
If testing shows that RAG is sufficient, HEXART helps build the vector database and document workflow with source citations. If fine-tuning is required, we prepare the training dataset, metric versioning, and evaluation suite to ensure future base model updates do not disrupt the deployment. See our full scope of services at hexart.pl.
Source materials
Sources consulted during research. The text above is our own; these third parties are not responsible for its content and have not authorised it.
- Kiedy fine-tuning ma sens (a kiedy wystarczy RAG)
- Opodatkowanie przychodów z działalności gospodarczej związanej z rozwojem modeli AI w formie ryczałtu 8,5% - Interpretacja indywidualna z dnia 29 kwietnia 2025 r., Dyrektor Krajowej Informacji Skarbowej, sygn. 0115-KDST2-2.4011.130.2025.2.AP - INFORLEX Freemium
