LLM Benchmark Table

The TOTAL score is a weighted composite of all individual benchmark categories. It represents the overall capability of each model across reasoning, STEM, utility, coding, and other tasks. The weighting is designed to reflect real-world usage patterns and importance of each capability.

Pass represents the percentage of tasks completed correctly on the first attempt. Refine shows additional successes achieved when the model is given feedback and allowed to retry. This helps distinguish models that are "right the first time" vs those that improve with iteration.

The Refusal rate shows how often a model declines to attempt a task, typically due to safety filters or content policies. A lower refusal rate isn't always better—it depends on whether refusals are appropriate. This metric helps users understand model behavior boundaries.

The $ mTok column shows the cost per million tokens, calculated as an average of input and output token prices. This provides a quick cost comparison, though actual costs may vary based on your specific input/output ratio and any volume discounts.

The Reasoning benchmark includes logical deduction, causal reasoning, analogical thinking, and multi-step problem solving. Tests are drawn from established datasets like ARC, HellaSwag, and custom adversarial challenges designed to test genuine understanding vs pattern matching.

The Censor score indicates the level of content filtering applied by the model, from 1 (minimal filtering) to 5 (heavy filtering). This is measured through standardized prompts testing various content categories. Neither high nor low is inherently better—it depends on your use case requirements.

We update our benchmarks monthly, or whenever a major model release occurs. All models are re-tested periodically to account for silent updates. Historical data is preserved so you can track model improvements over time.

Absolutely! We're always looking to expand our coverage. Models must have public API access or be available through major cloud providers. Community suggestions are prioritized based on popularity and unique capabilities. Contact us through the feedback form.