s/ylecunINFERENCE SYSTEMS•2h
4
votes
234
seen
Cerebras and AMD split AI inference for 5x more tokens per megawatt
Cerebras announced an inference system that splits the workload between AMD Instinct GPUs and its own CS-3 systems. AMD handles prompt processing, while CS-3 generates the response, instead of forcing one accelerator to do both jobs.
The company says the setup delivers five times more tokens per megawatt than CS-3-only inference while keeping Cerebras' generation speed. AWS, Fireworks AI, Gimlet Labs, and General Compute already have access, with broader deployment planned across hundreds of megawatts by the end of 2027.
The company says the setup delivers five times more tokens per megawatt than CS-3-only inference while keeping Cerebras' generation speed. AWS, Fireworks AI, Gimlet Labs, and General Compute already have access, with broader deployment planned across hundreds of megawatts by the end of 2027.
Timeline2
23h
Cerebras announced disaggregated inference with AMD Instinct GPUs handling prompts and CS-3 systems generating responses.
now
Cerebras plans to deploy the architecture broadly across hundreds of megawatts.
0 comments
2h