Infinite-Parameter LLMs: Weights Made From Live Data
A September 16, 2026 preprint proposes a hypernetwork that writes runtime interaction into a model's feed-forward weights instead of rereading it from the prompt.
In short
The Infinite-Parameter LLM replaces a fixed bank of stored feed-forward weights with a compact hypernetwork that generates those weights on the fly from whatever data the model is handed during a live interaction.
At a glance
- Preprint arXiv:2609.18842, submitted September 16, 2026, listed under cs.AI and cs.LG.
- Four authors: Jinli Hu, Ross M. Clarke, Yichuan Zhang, José Miguel Hernández-Lobato.
- Core idea: a hypernetwork generates feed-forward weights at run time instead of storing them.
- Mechanism: low-rank modulation of one shared base network, borrowed from Mixture-of-Experts.
- The abstract page lists no parameter counts, benchmark scores or training costs.
A new preprint asks what happens if a language model stops storing its feed-forward weights and starts producing them instead. In arXiv:2609.18842, submitted September 16, 2026, Jinli Hu, Ross M. Clarke, Yichuan Zhang and José Miguel Hernández-Lobato describe an architecture they call the Infinite-Parameter LLM, in which a small hypernetwork converts live interaction data into the weights the model uses for that request.
Why frozen weights are a limitation
Training ends, and the weights stop moving. Everything a user contributes afterward — a correction, a private fact, a document — has to be squeezed into the prompt, retrieved or instructed in, read once, and thrown away when the request closes. The model never actually learns from the interaction it is in.
How the architecture works
The design borrows its logic from Mixture-of-Experts, where each token activates only a slice of a very large stored weight pool. Here that pool is gone. A compact hypernetwork maps run-time data onto a low-rank modulation of one shared base network, so the feed-forward weights are generated per interaction rather than retrieved from storage.
What the paper does not show
arXiv lists the submission under cs.AI and cs.LG, and it has not been peer reviewed. The abstract page we read gives no parameter counts, no benchmark scores, no training-cost figures and no head-to-head comparison against existing MoE systems. We verified the abstract only; claims inside the full PDF remain unchecked here.
What to watch
If run-time knowledge can be written into weights rather than re-read from a prompt, personalization and correction stop being a context-window problem. They also move out of a place that is easy to inspect. Auditability, isolation between different users' modulations, and the ability to undo a bad update are the questions this abstract leaves open.
FAQ
What is an Infinite-Parameter LLM?
An architecture proposed in preprint arXiv:2609.18842 in which a hypernetwork generates a model's feed-forward weights at run time from the data supplied during the interaction, rather than loading them from a fixed stored bank.
How is this different from RAG or prompt-based context?
Retrieval and instructions place run-time knowledge in the prompt, where it is re-read each request and discarded afterward. This approach instead turns that data into a low-rank modulation of the model's own weights.
Are there benchmark results for the Infinite-Parameter LLM?
The abstract page we checked reports no benchmarks, parameter counts or training costs. The preprint was submitted on September 16, 2026 and has not been peer reviewed.
