Replymessage unavailable
Фотография
click to show
click to show
Introducing InferBench 📊
Hey Cysor
A tokens-per-second number means very little without the setup behind it. Which GPU? Which model? What quantization? What serving engine? What context length and concurrency?
InferBench is an open benchmarking platform for LLM inference from Cysic AI that shows the full configuration behind every result. No more taking numbers at face value.
🟣See exactly which hardware, model, engine, and runtime settings produced every result
🟣Compare GPUs, models, and serving configurations in one place
🟣Run the InferBench CLI on your own hardware and publish results
🟣Connect benchmark results with costs and estimated serving revenue
The fastest configuration isn't always the best one to run. InferBench helps you evaluate both performance and economics.
A more useful benchmark standard for AI inference starts with showing the full picture 👇