The team behind continuous batching says your idle GPUs should be running inference, not sitting dark

FriendliAI — founded by the researcher behind continuous batching, the technique at the core of vLLM — is launching InferenceSense, a platform that fills idle neocloud GPU capacity with paid AI ...
press.aboutamazon

AWS and Cerebras Collaboration Aims to Set a New Standard for AI Inference Speed and Performance in the Cloud

Deployed in AWS data centers and accessed through Amazon Bedrock, AWS Trainium + Cerebras CS-3 solution will accelerate inference speed Fastest inference coming soon: AWS and Cerebras are partnering ...