Awesome AI Agents › Communication Protocols

thu-nics/C2C

⭐ 446 Python repository created 2025-10-03

C2C, released as the Python package Rosetta, implements Cache-to-Cache communication between large language models, from an ICLR 2026 paper by a group at Tsinghua University. The premise is that when one model passes information to another as generated text, the transfer is lossy and slow: internal representations are collapsed into tokens and then re-encoded by the receiver. C2C instead takes the key-value cache of a sharer model, passes it through a trained projector called a Fuser, and injects the result into the cache of a receiver model, so the receiver conditions on the sharer latent state directly. The authors report accuracy gains of 8.5 to 10.5 percent over the individual models and 3.0 to 5.0 percent over text-based communication between the same models, with roughly a twofold latency reduction because no intermediate text has to be generated. The repository provides the model wrapper and generation path, training and evaluation extras, inference examples that download published Fuser weights from a Hugging Face collection for supported model pairs such as Qwen3 0.6B with Qwen2.5 0.5B, an interactive chat script and a Gradio demo that shows side-by-side comparison against the base models. Preliminary multi-sharer support fuses caches from several sharer models into one receiver. Setup is a conda environment with Python 3.10 and an editable install, with optional training and evaluation dependency groups. A hosted demo runs on Hugging Face Spaces. The work is relevant to multi-agent and multi-model systems because it targets the communication channel that such systems normally implement as text messages between models, and it is aimed at researchers and engineers studying model collaboration, ensembling and inference efficiency rather than at application developers looking for a ready framework.

https://github.com/thu-nics/C2C

llmkv-cachemodel-communicationmulti-modelsemantic-communicationinference-efficiencyresearchiclrpytorchpython