Local LLM translation test: three failures, raw model output Chunks are numbered from 1 here and in the article (chunk 1 = first chunk). Model: qwen/qwen3.6-27b (Q4_K_M GGUF) in LM Studio 0.4.8, temperature 0.2, reasoning_effort none, 2026-10-23. Source: Franz Kafka, Die Verwandlung, Project Gutenberg eBook #22367 (public domain). == Failure 1: whole excerpt in one request (condition C), paragraph 6 == Paragraph count still matched (19 in, 19 out). Word ratio 0.82. The second half of the source paragraph (calling in sick, the insurance doctor, Gregor feeling well and hungry) is missing, and the output adds sentences that are not in the source. -- German source -- Und er sah zur Weckuhr hinüber, die auf dem Kasten tickte. »Himmlischer Vater!« dachte er, Es war halb sieben Uhr, und die Zeiger gingen ruhig vorwärts, es war sogar halb vorüber, es näherte sich schon dreiviertel. Sollte der Wecker nicht geläutet haben? Man sah vom Bett aus, daß er auf vier Uhr richtig eingestellt war; gewiß hatte er auch geläutet. Ja, aber war es möglich, dieses möbelerschütternde Läuten ruhig zu verschlafen? Nun, ruhig hatte er ja nicht geschlafen, aber wahrscheinlich desto fester. Was aber sollte er jetzt tun? Der nächste Zug ging um sieben Uhr; um den einzuholen, hätte er sich unsinnig beeilen müssen, und die Kollektion war noch nicht eingepackt, und er selbst fühlte sich durchaus nicht besonders frisch und beweglich. Und selbst wenn er den Zug einholte, ein Donnerwetter des Chefs war nicht zu vermeiden, denn der Geschäftsdiener hatte beim Fünfuhrzug gewartet und die Meldung von seiner Versäumnis längst erstattet. Es war eine Kreatur des Chefs, ohne Rückgrat und Verstand. Wie nun, wenn er sich krank meldete? Das wäre aber äußerst peinlich und verdächtig, denn Gregor war während seines fünfjährigen Dienstes noch nicht einmal krank gewesen. Gewiß würde der Chef mit dem Krankenkassenarzt kommen, würde den Eltern wegen des faulen Sohnes Vorwürfe machen und alle Einwände durch den Hinweis auf den Krankenkassenarzt abschneiden, für den es ja überhaupt nur ganz gesunde, aber arbeitsscheue Menschen gibt. Und hätte er übrigens in diesem Falle so ganz unrecht? Gregor fühlte sich tatsächlich, abgesehen von einer nach dem langen Schlaf wirklich überflüssigen Schläfrigkeit, ganz wohl und hatte sogar einen besonders kräftigen Hunger. -- One request (condition C) -- And he looked over at the alarm clock, ticking on the chest of drawers. "Good Lord!" he thought. It was half past six and the hands were quietly moving forwards, it was even later than half past, more like quarter to seven. Had the alarm clock not rung? He could see from the bed that it had been set for four o'clock as it should have been; it certainly must have rung. Yes, but was it possible to quietly sleep through that furniture-rattling noise? Indeed, he hadn't slept quietly at all, but had probably slept even more soundly than usual. What should he do now? The next train went at seven; to catch that one he would have to perform miracles because his collection of samples was not packed and he did not feel especially fresh and lively. And even if he did catch the train he would not avoid his boss's anger as the office assistant would have been there at five o'clock to report his absence and he would have been dismissed on the spot. The office assistant was the boss's creature, spineless and stupid with a tendency to see everything reported to him as an opportunity to show himself off. Gregor's thought of him now brought a smile to his lips. -- Chunked (condition A), same paragraph -- And he looked over at the alarm clock ticking on the chest. "Heavenly Father!" he thought. It was half past six, and the hands were moving quietly forward; it was even past half, approaching three-quarters. Had the alarm not gone off? From the bed, one could see that it had been correctly set for four o'clock; surely it had rung. But was it possible to sleep through that furniture-shaking ringing in peace? Well, he had not slept peacefully, but probably all the more soundly. But what was he to do now? The next train left at seven; to catch it, he would have to rush insanely, and the collection was not yet packed, and he himself felt by no means particularly fresh and agile. And even if he did catch the train, a thunderous outburst from his boss was unavoidable, for the office messenger had waited at the five o'clock train and long since reported his absence. He was a creature of the boss, spineless and senseless. What if he called in sick? That would be extremely awkward and suspicious, for during his five years of service, Gregor had not even been ill once. The boss would certainly come with the insurance doctor, reproach the parents for their lazy son, and cut off all objections by referring to the insurance doctor, for whom there are only perfectly healthy but work-shy people. And did he not have a point in this case? Gregor actually felt quite well, apart from a drowsiness that was truly superfluous after such a long sleep, and he even had a particularly strong appetite. == Failure 2: previous paragraph sent as context (condition B), chunk 4 == The prompt said: 'Context (already translated, do NOT translate)'. The model translated it anyway, so chunk 4 returned 4 paragraphs for 3 source paragraphs and repeated the last paragraph of chunk 3. -- First output paragraph of chunk 4 (a second translation of source paragraph 13) -- But then he told himself: "Before it strikes a quarter to eight, I must absolutely have left the bed completely. Besides, someone from the office will come by then to ask after me, since the office opens before seven o'clock." And he now set about rocking his entire body out of the bed as evenly as possible. If he let himself fall out of the bed in this manner, the head, which he intended to raise sharply during the fall, would presumably remain uninjured. The back seemed hard; it would probably suffer no harm from the fall onto the carpet. The greatest concern was the loud crash that would inevitably occur and which would likely cause not only alarm but also anxiety behind all the doors. But this risk had to be taken. -- Source paragraph 13 -- Dann aber sagte er sich: »Ehe es einviertel acht schlägt, muß ich unbedingt das Bett vollständig verlassen haben. Im übrigen wird auch bis dahin jemand aus dem Geschäft kommen, um nach mir zu fragen, denn das Geschäft wird vor sieben Uhr geöffnet.« Und er machte sich nun daran, den Körper in seiner ganzen Länge vollständig gleichmäßig aus dem Bett hinauszuschaukeln. Wenn er sich auf diese Weise aus dem Bett fallen ließ, blieb der Kopf, den er beim Fall scharf heben wollte, voraussichtlich unverletzt. Der Rücken schien hart zu sein; dem würde wohl bei dem Fall auf den Teppich nichts geschehen. Das größte Bedenken machte ihm die Rücksicht auf den lauten Krach, den es geben müßte und der wahrscheinlich hinter allen Türen wenn nicht Schrecken, so doch Besorgnisse erregen würde. Das mußte aber gewagt werden. == Failure 3: last chunk sent alone without a glossary (condition A), chunk 6 == The chunk was a single paragraph. Without the earlier chapters, the model rendered 'der Prokurist' (the chief clerk) as 'the traveling salesman', which is Gregor's own job. Paragraph count and length ratio both looked normal (ratio 1.00). In condition B, which added the glossary to the system instruction and sent the previous source paragraph as context, the same paragraph used 'chief clerk'. This test cannot say which of the two fixed it. -- Source paragraph 19, opening -- »Herr Samsa,« rief nun der Prokurist mit erhobener Stimme, »was ist denn los? Sie verbarrikadieren sich da in Ihrem Zimmer, antworten bloß mit ja und nein, machen Ihren Eltern schwere, unnötige Sorgen und versäumen -- dies nur nebenbei erwähnt -- Ihre geschäftlichen Pflichten in einer eigentlich unerhörten Weise. Ich spreche hier im Namen Ihrer Eltern und Ihres Chefs und bitte Sie ganz ernsthaft u -- Condition A output, opening -- "Mr. Samsa," the traveling salesman now called out in a raised voice, "what is going on? You are barricading yourself in your room, answering only with yes and no, causing your parents heavy, unnecessary worries, and neglecting—this only mentioned in passing—your business duties in an actually unheard-of manner. I am speaking here on behalf of your parents and your boss and I am asking you very se -- Condition B output, opening -- "Mr. Samsa," the chief clerk now called out in a raised voice, "what is going on? You are barricading yourself in your room, answering only with yes and no, causing your parents heavy, unnecessary worries, and neglecting—this only mentioned in passing—your business duties in a truly unheard-of manner. I am speaking here on behalf of your parents and your boss, and I am asking you very seriously fo