How to Translate Long Documents with a Local LLM: Chunks and Checks

People translate documents with a local LLM for one reason: the file cannot leave their machine. A local model can do the job, but it gives you plain text, not a translated file, and its worst mistakes are quiet ones. In our test, a single request for a 3,179-word excerpt came back with the right number of paragraphs and a clean finish signal, yet one paragraph had lost more than a third of its content and gained sentences that are not in the source.

Short answer: split the text at paragraph boundaries into chunks of about 600 words, put a glossary of names and recurring terms in the system instruction, turn the model's thinking mode off, and compare every chunk with its source paragraph by paragraph before you trust it. Each of the three ways we ran the test failed somewhere, and no failure produced an error message. Expect a mid-size model on a laptop to take hours for a whole book, and plan to rebuild the formatting yourself.

Everything below comes from one run on October 23, 2026: Qwen 3.6 27B (Q4_K_M GGUF) in LM Studio 0.4.8 on an Apple M4 Max with 64 GB of memory, translating the opening of Kafka's Die Verwandlung from German into English. The test script, results, and raw failure examples are free to download so you can repeat it with your own model.

When a Local LLM Is the Right Tool

Run the translation locally when the document's rules, not convenience, make the decision:

• a client contract, research agreement, or employer policy forbids uploading the file to an outside service;
• the text is personal or confidential and you cannot remove the sensitive parts first; or
• you need to process many files with no per-document cost and you already own suitable hardware.

If none of these applies, a local model is usually the slower path to a worse result. You give up three things a document translation service provides: a translated file in the original format, a model larger than your machine can hold, and a workflow built for consistency across a whole book. Before choosing either route, work through the document translation privacy checklist. It covers authority to process the file, minimal working copies, and what a service does with uploads.

"Local" also needs a precise meaning. The model runs on your computer, and in our test the requests went only to localhost. That does not tell you what the app around the model logs, updates, or reports. If your rules require proof that nothing leaves the device, check the runtime's own settings and policy, or run it on a machine with no network access.

What We Tested

We translated the first 19 paragraphs of Die Verwandlung (3,179 German words) from the public-domain Project Gutenberg edition. Literary German is a fair stress test: the paragraphs are long, the sentences nest, and the same characters and job titles recur.

All three runs used the same model and settings: temperature 0.2, a maximum of 12,000 output tokens, and a 16,384-token context window. The instruction asked for one English paragraph per German paragraph and nothing else.

Every run finished with a normal stop signal, and every run contained a different silent error. Run C dropped and invented text. Run B translated text it was told to skip. Run A, the cleanest, gave one character the wrong job in its last chunk.

Failure 1: One Big Request Dropped Part of a Paragraph

Run C finished normally and returned exactly 19 paragraphs. A paragraph-count check would have passed it. Paragraph 6, however, came back at 214 words for a 260-word source.

In the source, Gregor looks at the clock, realizes he has missed his train, and then considers calling in sick: his boss would arrive with the health-insurance doctor, who regards everyone as healthy but work-shy, and Gregor admits he actually feels fine apart from drowsiness and an unusual appetite. Run C translated the first part, then dropped the entire sick-day passage, 98 of the paragraph's 260 German words. In its place it wrote that Gregor "would have been dismissed on the spot," that the office assistant tends "to see everything reported to him as an opportunity to show himself off," and that the thought "brought a smile to his lips." None of that is in the German.

The chunked run translated the same paragraph completely, at 267 words.

One sample does not prove that long requests always fail. It does show what to check: a long single request can silently compress or invent text while still looking structurally complete. A reader with no German would have no reason to notice.

Failure 2: The Model Translated the "Context" It Was Told to Skip

A common tip is to send the previous paragraph with each chunk so the model keeps the thread. In run B, we added it to the message under the label "Context (already translated, do NOT translate)."

In chunk 4 the model translated it anyway. The chunk returned four paragraphs for three source paragraphs, and the first was a second translation of the last paragraph of chunk 3. Joined together, the book would have contained that paragraph twice, worded differently each time.

The practical rule: if you send context, keep it out of the text the model is asked to translate, keep it short, and check the paragraph count of every chunk. If you cannot check, leave it out.

Failure 3: A Chunk Sent Alone Gave a Character the Wrong Job

Run A sent each chunk with no glossary and no context. For the first five chunks this worked well: Prokurist became "chief clerk" in all 16 places it appeared. The sixth chunk was a single paragraph in which the chief clerk scolds Gregor through the door. Seen alone, the model translated der Prokurist as "the traveling salesman," which is Gregor's own job. The paragraph count matched and the length ratio was 1.00, so neither structural check noticed.

In run B, which had the glossary in the system instruction and the previous paragraph as context, the same paragraph correctly used "chief clerk." This test cannot say which of the two fixed it. Because the context also caused failure 2, the glossary is the safer place to carry names and roles from chunk to chunk: it is short, it sits in the instruction, and you can check it mechanically.

The Workflow That Held Up

1. Get plain text out of the file first

A local LLM translates text, not files. Convert the document before you start:

• DOCX or EPUB: export to plain text or Markdown with one paragraph per block, keeping headings as separate lines.
• PDF with a text layer: extract the text, then repair broken lines and hyphenation. Our guide to fixing OCR line breaks and hyphenation covers the same cleanup. A converter such as Docling can produce Markdown from PDFs and Office files.
• Scanned PDF: run OCR first. If you are unsure whether a PDF has usable text, see PDF text extraction vs OCR.

Keep the paragraph structure intact. Every later check depends on a one-to-one match between source and translated paragraphs.

2. Turn thinking mode off and confirm it

Many current open models reason before they answer. For translation, that wastes time and output tokens. In our setup, a 200-token test request spent 199 tokens on reasoning and returned an empty answer.

We tried three common switches. Adding /no_think to the prompt did nothing, and chat_template_kwargs with enable_thinking: false did nothing. Sending "reasoning_effort": "none" in the request turned reasoning off: the response reported 0 reasoning tokens and returned the translation directly. This parameter is not listed on LM Studio's chat completions page, so treat it as behavior we observed in version 0.4.8 with this model. Check the reasoning_tokens count in your own responses rather than trusting any switch.

3. Size the context window before the chunks

The context window must hold the instruction, the chunk, and the full translation. In our chunked runs, a 500- to 670-word German chunk used about 800 to 1,500 input tokens, depending on the glossary and context, and 640 to 870 output tokens, so a few thousand tokens is enough. The single 3,179-word request needed 4,953 input tokens and 3,930 output tokens.

Ollama sets its default context length by available memory: 4k tokens below 24 GiB of VRAM, 32k from 24 to 48 GiB, and 256k at 48 GiB or more, according to its context length documentation checked on October 23, 2026. You can raise it with OLLAMA_CONTEXT_LENGTH. In LM Studio, set the context length when you load the model. If a chunk plus its translation does not fit, the output can be cut off.

4. Chunk at paragraph boundaries, about 600 words

Never split inside a paragraph or sentence. Our chunks targeted 700 words and ended at the nearest paragraph boundary, which produced six chunks of 243 to 667 words. Smaller chunks cost little: run A took 10.4 minutes against 9.5 minutes for the single request, and a failed chunk is cheap to rerun.

Keep the instruction short and identical for every chunk. Ours was: translate into English, keep exactly one output paragraph per input paragraph separated by a blank line, and output only the translation. Add a glossary to the same instruction: one line per name, title, or recurring term, in the form Prokurist = chief clerk. Our guide to building a translation glossary covers what to include. Short chunks lose the surrounding story, and the glossary is what carries it.

5. Check every chunk before you join them

Run three checks on each chunk:

1. Paragraph count. The output must have the same number of paragraphs as the input. This caught failure 2.
2. Length ratio per paragraph. Compare the word count of each translated paragraph with its source. This caught failure 1, which the paragraph count missed.
3. Glossary terms per paragraph. Where a source term appears, its required translation should appear the same number of times. This caught failure 3, which both other checks missed. Inflected languages will need some judgment here, because a German or Russian term changes form.

For German to English prose, run A's paragraphs stayed between 0.98 and 1.17 words out per word in. Run C's damaged paragraph was 0.82. Set your own range by checking a few paragraphs you trust in your language pair; the defaults in our script are a starting point, not a standard.

A flagged paragraph is where to read first, not proof of an error. Then read the translation itself, especially names, numbers, negations, and anything a reader would act on. The book translation QA checklist covers that review.

6. Rebuild the document

The model returns text. Headings, footnotes, tables, images, page layout, and styles must be put back by hand or by a conversion tool. For a short report this is minor. For a book with footnotes and tables it can take longer than the translation.

How Long a Whole Book Takes

Our chunked run translated 3,179 words in 10.4 minutes, about 300 words per minute on this machine. At that rate a 90,000-word novel would take roughly five hours of generation, before checks and reruns. That is simple arithmetic from one run, not a benchmark. Speed changes with the model size, quantization, hardware, language pair, and how much else the machine is doing.

The model file was 17.5 GB on disk, and the context cache and runtime buffers need memory on top of that, so a machine needs noticeably more free memory than the file size. A smaller model will run faster and on less memory, usually at some cost in quality. Test it on a representative sample before committing a whole book. Our guide to the best LLM for translation explains how to build that sample and compare models fairly.

Local LLM or Document Translation Service

If your rules allow uploading the file, a document service removes most of the work above. BookTranslator translates complete PDF, EPUB, and DOCX files and returns a translated file, and its FAQ says uploaded files and translation output are not used to train AI models. It is a cloud service, so it is not an option when the file must stay on your machine. If you are deciding between running models yourself and paying for a service, what LLM translation actually costs shows how to budget the token side.

Get the Test Files

All files are in the /files/blog/translate-with-local-llm/ folder:

• local_llm_translation_test.py: the full test against LM Studio's local API, with all three runs. Point it at another model or source text to repeat the test.
• check_translation.py: compares a source text and a translation paragraph by paragraph and prints paragraph-count mismatches, out-of-range length ratios, and missing glossary terms (with --term "Prokurist=chief clerk"). It works with output from any tool.
• results.csv: every request with its paragraph counts, word counts, token counts, and time.
• failure-examples.txt: the German source and raw model output for all three failures.

Run check_translation.py with your glossary terms on any long translation before you rely on it. It takes seconds, and it flagged all three failures in this test. It cannot tell you whether a fluent sentence is wrong, so the flagged paragraphs are where your reading starts, not where it ends.