What is it about?

When you use a cloud AI service, the provider’s servers can see everything you type. Providers may promise not to train on your data, but you have to take that on trust. We show another way. We split a large language model across two servers and add a secret random mask to the data before it reaches the server that runs the larger part of the model. Because the heaviest steps there are linear, the mask can be removed exactly afterwards. The answers are identical, bit for bit, to running the model normally. We demonstrate this live at ACM CCS 2026.

Featured Image

Why is it important?

Researchers and companies increasingly rely on large AI models, and much of their work is confidential. They need three things at once: a highly capable model, data privacy, and an affordable price. Today’s options deliver only two. Cloud AI is capable and affordable, but privacy rests on the provider’s promise. Running models on your own hardware is private, but you must choose between a smaller model and a high cost. Our approach is designed to deliver all three: it keeps prompts masked from the server, leaves the model’s output unchanged, and runs on standard cloud GPUs. We demonstrate it on a 119-billion-parameter open-weight model.

Perspectives

I worked as a patent attorney for 15 years, and this year I saw a turning point in both industry and academia. Until last year, security usually came first. Now AI is so productive that people cannot afford not to use it, even while they worry about where their data goes. In R&D the problem is sharper, because the context itself is the intellectual property: hiding a few personal details or filtering parts of a prompt does not solve it. I became convinced that people will need protection built into the system itself, not resting on a promise, and that conviction led me to this work.

Jihun Kim
Sealouette

Read the Original

This page is a summary of: DEMO: End-to-End Privacy-Preserving LLM Inference: Linearity-Based Attention–FFN Split with Additive Masking over Two Cloud GPU Nodes, November 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3830454.3846405.
You can read the full text:

Read

Resources

Contributors

The following have contributed to this page