Context compression finally works in production: new research cuts LLM input 16x without the accuracy hit

LCLMs compress LLM context before decode — 8.8x faster at 16x compression, beating every KV cache method tested. Open-sourced by NYU and Columbia.

Why Do Chatbots Keep Telling Stories About Someone Named ‘Elias Thorne’?

They looked at about 20,000 stories generated by the models and found a shocking amount of repetition: 11 words—Lighthouse, ...