WHAT LLM PROVIDER?

Chart Configuration

Filter Models

Current: 0
Current: $30.00

Model Comparison Chart

Color Legend

GPT-4
Llama 3.1 405B
Llama 3.2 90B
Llama 3.1 70B
Llama 3.2 11B
Llama 3.1 8B
Llama 3.2 3B
Llama 3.2 1B
Gemini 1.5 Pro
Gemini 1.5 Flash
Gemma 2 27B
Gemma 2 9B
Claude 3.5 Sonnet
Claude 3 Opus
Mistral Large 2
Mixtral 8x22B
Jamba 1.5 Large
DeepSeek-V2
Qwen2 72B
Yi-Large

Model Comparison

Name
Provider
Creator
Model
Price (per M/Token)
Input Price
Output Price
Output Speed
Context Window
Latency
Quality Index
o1-previewOpenAIOpenAIGPT-4o1-preview$26.25$15.00$60.0030.9 tokens/s128000 tokens32.62 s85
o1-miniOpenAIOpenAIGPT-4o1-mini$5.25$3.00$12.0070.2 tokens/s128000 tokens14.58 s82
GPT-4oOpenAIOpenAIGPT-4o$4.38$2.50$10.00124.2 tokens/s128000 tokens0.42 s77
GPT-4o (May '24)OpenAIOpenAIGPT-4o (May 24)$7.50$5.00$15.00112.6 tokens/s128000 tokens0.42 s77
GPT-4o (May '24)Microsoft AzureOpenAIGPT-4o$7.50$5.00$15.00107.2 tokens/s128000 tokens0.37 s77
GPT-4o miniOpenAIOpenAIGPT-4o mini$0.26$0.15$0.6098.8 tokens/s128000 tokens0.45 s71
Llama 3.1 405BReplicateMetaLlama 3.1 405B$9.50$9.50$9.5018.7 tokens/s128000 tokens1.13 s72
Llama 3.1 405BHyperbolicMetaLlama 3.1 405B$4.00$4.00$4.0016.1 tokens/s128000 tokens0.91 s72
Llama 3.1 405BAmazon BedrockMetaLlama 3.1 405B$7.99$5.32$16.0013.1 tokens/s128000 tokens1.78 s72
Llama 3.1 405BOctoAIMetaLlama 3.1 405B$4.50$3.00$9.0059.6 tokens/s128000 tokens0.3 s72
Llama 3.1 405BLepton AIMetaLlama 3.1 405B$2.80$2.80$2.8020.1 tokens/s128000 tokens1.04 s72
Llama 3.1 405BMicrosoft AzureMetaLlama 3.1 405B$8.00$5.33$16.008.4 tokens/s128000 tokens4.38 s72
Llama 3.1 405BFireworksMetaLlama 3.1 405B$3.00$3.00$3.0069.3 tokens/s128000 tokens0.64 s72
Llama 3.1 405BDeepinfraMetaLlama 3.1 405B$1.79$1.79$1.7922.7 tokens/s33000 tokens0.45 s72
Llama 3.1 405BSambaNovaMetaLlama 3.1 405B$6.25$5.00$10.00127.8 tokens/s8000 tokens1.5 s72
Llama 3.1 405BDatabricksMetaLlama 3.1 405B$7.50$5.00$15.0028.6 tokens/s128000 tokens0.67 s72
Llama 3.1 405B TurboTogether.ai TurboMetaLlama 3.1 405B$3.50$3.50$3.5087.3 tokens/s128000 tokens0.72 s72
Llama 3.2 90B (Vision)HyperbolicMetaLlama 3.2 90B (Vision)$0.40$0.40$0.4040.2 tokens/s128000 tokens0.57 s66
Llama 3.2 90B (Vision)Amazon BedrockMetaLlama 3.2 90B (Vision)$2.00$2.00$2.0018.2 tokens/s128000 tokens0.54 s66
Llama 3.2 90B (Vision)FireworksMetaLlama 3.2 90B (Vision)$0.90$0.90$0.9050.5 tokens/s128000 tokens0.44 s66
Llama 3.2 90B (Vision)DeepinfraMetaLlama 3.2 90B (Vision)$0.36$0.35$0.4024.7 tokens/s128000 tokens0.41 s66
Llama 3.2 90B (Vision) TurboTogether.ai TurboMetaLlama 3.2 90B (Vision)$1.20$1.20$1.2054.6 tokens/s128000 tokens0.38 s66
Llama 3.1 70BCerebrasMetaLlama 3.1 70B$0.60$0.60$0.60569.2 tokens/s8000 tokens0.43 s65
Llama 3.1 70BHyperbolicMetaLlama 3.1 70B$0.40$0.40$0.4029.1 tokens/s128000 tokens0.69 s65
Llama 3.1 70BAmazon BedrockMetaLlama 3.1 70B$0.99$0.99$0.9931.6 tokens/s128000 tokens0.71 s65
Llama 3.1 70BOctoAIMetaLlama 3.1 70B$0.90$0.90$0.9059.6 tokens/s128000 tokens0.36 s65
Llama 3.1 70BLepton AIMetaLlama 3.1 70B$0.80$0.80$0.8052.8 tokens/s128000 tokens0.58 s65
Llama 3.1 70BMicrosoft AzureMetaLlama 3.1 70B$2.90$2.68$3.5420.3 tokens/s128000 tokens0.64 s65
Llama 3.1 70BFireworksMetaLlama 3.1 70B$0.90$0.90$0.9086.2 tokens/s128000 tokens0.39 s65
Llama 3.1 70BDeepinfraMetaLlama 3.1 70B$0.36$0.35$0.4027.6 tokens/s128000 tokens0.31 s65
Llama 3.1 70BGroqMetaLlama 3.1 70B$0.64$0.59$0.79249.7 tokens/s128000 tokens0.44 s65
Llama 3.1 70BSambaNovaMetaLlama 3.1 70B$0.75$0.60$1.20419.7 tokens/s8000 tokens0.81 s65
Llama 3.1 70BDatabricksMetaLlama 3.1 70B$1.50$1.00$3.0054.6 tokens/s128000 tokens0.58 s65
Llama 3.1 70BPerplexityMetaLlama 3.1 70B$1.00$1.00$1.0047.7 tokens/s128000 tokens0.33 s65
Llama 3.1 70B TurboTogether.ai TurboMetaLlama 3.1 70B$0.88$0.88$0.8827 tokens/s128000 tokens0.73 s65
Llama 3.2 11B (Vision)Amazon BedrockMetaLlama 3.2 11B (Vision)$0.35$0.35$0.3541.7 tokens/s128000 tokens0.4 s54
Llama 3.2 11B (Vision)FireworksMetaLlama 3.2 11B (Vision)$0.20$0.20$0.20119.8 tokens/s128000 tokens0.33 s54
Llama 3.2 11B (Vision)DeepinfraMetaLlama 3.2 11B (Vision)$0.06$0.06$0.0677.4 tokens/s128000 tokens0.27 s54
Llama 3.2 11B (Vision) TurboTogether.ai TurboMetaLlama 3.2 11B (Vision)$0.18$0.18$0.18152.2 tokens/s128000 tokens0.32 s54
Llama 3.1 8BCerebrasMetaLlama 3.1 8B$0.10$0.10$0.102023 tokens/s8000 tokens0.46 s53
Llama 3.1 8BHyperbolicMetaLlama 3.1 8B$0.10$0.10$0.1093.4 tokens/s128000 tokens0.52 s53
Llama 3.1 8BAmazon BedrockMetaLlama 3.1 8B$0.22$0.22$0.2289.8 tokens/s128000 tokens0.4 s53
Llama 3.1 8BOctoAIMetaLlama 3.1 8B$0.15$0.15$0.15174.9 tokens/s128000 tokens0.26 s53
Llama 3.1 8BLepton AIMetaLlama 3.1 8B$0.07$0.07$0.07206.7 tokens/s128000 tokens0.38 s53
Llama 3.1 8BMicrosoft AzureMetaLlama 3.1 8B$0.38$0.30$0.6154.3 tokens/s128000 tokens0.42 s53
Llama 3.1 8BFireworksMetaLlama 3.1 8B$0.20$0.20$0.20264.1 tokens/s128000 tokens0.28 s53
Llama 3.1 8BDeepinfraMetaLlama 3.1 8B$0.06$0.06$0.0686.2 tokens/s128000 tokens0.22 s53
Llama 3.1 8BGroqMetaLlama 3.1 8B$0.06$0.05$0.08751.7 tokens/s128000 tokens0.38 s53
Llama 3.1 8BSambaNovaMetaLlama 3.1 8B$0.13$0.10$0.201018.6 tokens/s8000 tokens0.39 s53
Llama 3.1 8BPerplexityMetaLlama 3.1 8B$0.20$0.20$0.20161.8 tokens/s128000 tokens0.2 s53
Llama 3.1 8B TurboTogether.ai TurboMetaLlama 3.1 8B$0.18$0.18$0.18145.6 tokens/s128000 tokens0.58 s53
Llama 3.2 3BHyperbolicMetaLlama 3.2 3B$0.10$0.10$0.10192.3 tokens/s128000 tokens0.5 s47
Llama 3.2 3BAmazon BedrockMetaLlama 3.2 3B$0.15$0.15$0.15142.4 tokens/s128000 tokens0.44 s47
Llama 3.2 3BLepton AIMetaLlama 3.2 3B$0.03$0.03$0.03116.1 tokens/s128000 tokens0.43 s47
Llama 3.2 3BFireworksMetaLlama 3.2 3B$0.10$0.10$0.10257.5 tokens/s128000 tokens0.32 s47
Llama 3.2 3BDeepinfraMetaLlama 3.2 3B$0.04$0.03$0.05104.1 tokens/s128000 tokens0.26 s47
Llama 3.2 3BGroqMetaLlama 3.2 3B$0.06$0.06$0.061406.1 tokens/s8000 tokens0.34 s47
Llama 3.2 3BSambaNovaMetaLlama 3.2 3B$0.10$0.08$0.161565.7 tokens/s4000 tokens0.34 s47
Llama 3.2 3B TurboTogether.ai TurboMetaLlama 3.2 3B$0.06$0.06$0.06124.8 tokens/s128000 tokens0.3 s47
Llama 3.2 1BAmazon BedrockMetaLlama 3.2 1B$0.10$0.10$0.10303 tokens/s128000 tokens0.38 s27
Llama 3.2 1BFireworksMetaLlama 3.2 1B$0.10$0.10$0.10507.1 tokens/s128000 tokens0.29 s27
Llama 3.2 1BDeepinfraMetaLlama 3.2 1B$0.01$0.01$0.02167 tokens/s128000 tokens0.23 s27
Llama 3.2 1BGroqMetaLlama 3.2 1B$0.04$0.04$0.043129.3 tokens/s8000 tokens0.48 s27
Llama 3.2 1BSambaNovaMetaLlama 3.2 1B$0.05$0.04$0.082473.9 tokens/s4000 tokens0.35 s27
Gemini 1.5 Pro (Sep '24) (Vertex)Google (Vertex)GoogleGemini 1.5 Pro$2.19$1.25$5.0059.4 tokens/s2000000 tokens0.44 s80
Gemini 1.5 Pro (Sep '24) (AI Studio)Google (AI Studio)GoogleGemini 1.5 Pro$2.19$1.25$5.0061.3 tokens/s2000000 tokens0.81 s80
Gemini 1.5 Flash-8B AI StudioGoogle AI StudioGoogleGemini 1.5 Flash-8B$0.07$0.04$0.15283.2 tokens/s1000000 tokens0.53 s73
Gemini 1.5 Flash (Sep '24) (Vertex)Google (Vertex)GoogleGemini 1.5 Flash$0.13$0.07$0.30203.8 tokens/s1000000 tokens0.26 s73
Gemini 1.5 Flash (Sep '24) (AI Studio)Google (AI Studio)GoogleGemini 1.5 Flash$0.13$0.07$0.30209.3 tokens/s1000000 tokens0.4 s73
Gemma 2 27BTogether.aiGoogleGemma 2 27B$0.80$0.80$0.8065.5 tokens/s8000 tokens0.48 s61
Gemma 2 9BDeepinfraGoogleGemma 2 9B$0.06$0.06$0.0671.1 tokens/s8000 tokens0.32 s46
Gemma 2 9BGroqGoogleGemma 2 9B$0.20$0.20$0.20665.6 tokens/s8000 tokens0.19 s46
Gemma 2 9BTogether.aiGoogleGemma 2 9B$0.30$0.30$0.30114.9 tokens/s8000 tokens0.44 s46
Gemini 1.5 Flash (May '24) (Vertex)Google (Vertex)GoogleGemini 1.5 Flash$0.13$0.07$0.30303.9 tokens/s1000000 tokens0.27 s73
Gemini 1.5 Flash (May '24) (AI Studio)Google (AI Studio)GoogleGemini 1.5 Flash$0.13$0.07$0.30309.2 tokens/s1000000 tokens0.36 s73
Gemini 1.5 Pro (May '24) (Vertex)Google (Vertex)GoogleGemini 1.5 Pro$5.25$3.50$10.5064.3 tokens/s2000000 tokens0.5 s80
Gemini 1.5 Pro (May '24) (AI Studio)Google (AI Studio)GoogleGemini 1.5 Pro$5.25$3.50$10.5065.4 tokens/s2000000 tokens0.79 s80
Claude 3.5 SonnetAmazon BedrockAnthropicClaude 3.5 Sonnet$6.00$3.00$15.0053.5 tokens/s200000 tokens0.95 s77
Claude 3.5 SonnetAnthropicAnthropicClaude 3.5 Sonnet$6.00$3.00$15.0090.8 tokens/s200000 tokens0.8 s77
Claude 3 OpusAmazon BedrockAnthropicClaude 3 Opus$30.00$15.00$75.0023.6 tokens/s200000 tokens1.67 s70
Claude 3 OpusAnthropicAnthropicClaude 3 Opus$30.00$15.00$75.0028.4 tokens/s200000 tokens2.98 s70
Claude 3 HaikuAmazon BedrockAnthropicClaude 3 Haiku$0.50$0.25$1.25120.8 tokens/s200000 tokens0.48 s54
Claude 3 HaikuAnthropicAnthropicClaude 3 Haiku$0.50$0.25$1.25145.4 tokens/s200000 tokens0.45 s54
Mistral Large 2MistralMistral AIMistral Large 2$3.00$2.00$6.0031.5 tokens/s128000 tokens0.75 s73
Mistral Large 2Amazon BedrockMistral AIMistral Large 2$4.50$3.00$9.0042.8 tokens/s128000 tokens0.42 s73
Mistral Large 2Microsoft AzureMistral AIMistral Large 2$4.50$3.00$9.0053.7 tokens/s128000 tokens0.44 s73
Mixtral 8x22BMistralMistral AIMixtral 8x22B$3.00$2.00$6.0064.6 tokens/s65000 tokens0.59 s61
Mixtral 8x22BOctoAIMistral AIMixtral 8x22B$1.20$1.20$1.2092.4 tokens/s65000 tokens0.33 s61
Mixtral 8x22BFireworksMistral AIMixtral 8x22B$1.20$1.20$1.2079.3 tokens/s65000 tokens0.32 s61
Mixtral 8x22BTogether.aiMistral AIMixtral 8x22B$1.20$1.20$1.2061 tokens/s65000 tokens0.42 s61
Mistral Small (Sep '24)MistralMistral AIMistral Small$0.30$0.20$0.6076.2 tokens/s128000 tokens0.5 s60
Pixtral 12BMistralMistral AIPixtral 12B$0.15$0.15$0.1579.9 tokens/s128000 tokens0.58 s56
Pixtral 12BHyperbolicMistral AIPixtral 12B$0.10$0.10$0.1080.1 tokens/s128000 tokens0.52 s56
Mistral NeMoMistralMistral AIMistral NeMo$0.15$0.15$0.15130.1 tokens/s128000 tokens0.52 s52
Mistral NeMoOctoAIMistral AIMistral NeMo$0.20$0.20$0.20158.5 tokens/s128000 tokens0.31 s52
Mistral NeMoDeepinfraMistral AIMistral NeMo$0.13$0.13$0.1357.7 tokens/s128000 tokens0.25 s52
Mixtral 8x7BMistralMistral AIMixtral 8x7B$0.70$0.70$0.7086.4 tokens/s33000 tokens0.56 s42
Mixtral 8x7BReplicateMistral AIMixtral 8x7B$0.47$0.30$1.0089.5 tokens/s33000 tokens0.55 s42
Mixtral 8x7BAmazon BedrockMistral AIMixtral 8x7B$0.51$0.45$0.7068.7 tokens/s33000 tokens0.36 s42
Mixtral 8x7BOctoAIMistral AIMixtral 8x7B$0.45$0.45$0.4581.5 tokens/s33000 tokens0.33 s42
Mixtral 8x7BLepton AIMistral AIMixtral 8x7B$0.50$0.50$0.50105.4 tokens/s33000 tokens0.48 s42
Mixtral 8x7BFireworksMistral AIMixtral 8x7B$0.50$0.50$0.5098.3 tokens/s33000 tokens0.29 s42
Mixtral 8x7BDeepinfraMistral AIMixtral 8x7B$0.24$0.24$0.2440.5 tokens/s33000 tokens0.28 s42
Mixtral 8x7BGroqMistral AIMixtral 8x7B$0.24$0.24$0.24543.6 tokens/s33000 tokens0.22 s42
Mixtral 8x7BDatabricksMistral AIMixtral 8x7B$0.63$0.50$1.0086.7 tokens/s33000 tokens0.48 s42
Mixtral 8x7BTogether.aiMistral AIMixtral 8x7B$0.60$0.60$0.60104.3 tokens/s33000 tokens0.44 s42
Codestral-MambaMistralMistral AICodestral-Mamba$0.25$0.25$0.2594.1 tokens/s256000 tokens0.68 s36
Command-R+Amazon BedrockCohereCommand-R+$6.00$3.00$15.0044.9 tokens/s128000 tokens0.58 s56
Command-R+CohereCohereCommand-R+$4.38$2.50$10.0065.2 tokens/s128000 tokens0.29 s56
Command-RAmazon BedrockCohereCommand-R$0.75$0.50$1.50102.6 tokens/s128000 tokens0.39 s51
Command-RCohereCohereCommand-R$0.26$0.15$0.60111.5 tokens/s128000 tokens0.22 s51
Command-R+ (Apr '24)Amazon BedrockCohereCommand-R+$6.00$3.00$15.0045.2 tokens/s128000 tokens0.57 s46
Command-R+ (Apr '24)CohereCohereCommand-R+$6.00$3.00$15.0041.4 tokens/s128000 tokens0.32 s46
Command-R+ (Apr '24)Microsoft AzureCohereCommand-R+$6.00$3.00$15.0045.6 tokens/s128000 tokens0.69 s46
Command-R (Mar '24)Amazon BedrockCohereCommand-R$0.75$0.50$1.50102.6 tokens/s128000 tokens0.39 s36
Command-R (Mar '24)CohereCohereCommand-R$0.75$0.50$1.50153 tokens/s128000 tokens0.2 s36
Command-R (Mar '24)Microsoft AzureCohereCommand-R$0.75$0.50$1.50103.9 tokens/s128000 tokens0.52 s36
Sonar 3.1 SmallPerplexityPerplexitySonar 3.1 Small$0.20$0.20$0.20131.8 tokens/s131000 tokens0.19 sN/A
Sonar 3.1 LargePerplexityPerplexitySonar 3.1 Large$1.00$1.00$1.0058.8 tokens/s131000 tokens0.24 sN/A
Phi-3 Medium 14BMicrosoft AzureMicrosoftPhi-3 Medium 14B$0.30$0.17$0.6851.2 tokens/s128000 tokens0.44 sN/A
DBRXDatabricksDatabricksDBRX$1.13$0.75$2.2584.7 tokens/s33000 tokens0.49 s49
DBRXTogether.aiDatabricksDBRX$1.20$1.20$1.20104.8 tokens/s33000 tokens0.36 s49
ALT 11BRekaReka AIALT 11B$0.16$0.16$0.16147.6 tokens/s128000 tokens0.21 s49
ALT 7BRekaReka AIALT 7B$0.10$0.10$0.10152.3 tokens/s128000 tokens0.18 s49
ALT 3BOctoAIReka AIALT 3B$0.05$0.05$0.0591.5 tokens/s128000 tokens0.12 s49
Jamba 1.5 LargeAI21 LabsAI21 LabsJamba 1.5 Large$3.50$2.00$8.0059.2 tokens/s256000 tokens1.01 s64
Jamba 1.5 LargeMicrosoft AzureAI21 LabsJamba 1.5 Large$3.50$2.00$8.0051.1 tokens/s256000 tokens0.69 s64
Jamba 1.5 MiniAI21 LabsAI21 LabsJamba 1.5 Mini$0.25$0.20$0.40161.2 tokens/s256000 tokens0.85 s46
Jamba 1.5 MiniMicrosoft AzureAI21 LabsJamba 1.5 Mini$0.25$0.20$0.4082.5 tokens/s256000 tokens0.5 s46
DeepSeek-Coder-V2DeepSeekDeepSeekDeepSeek-Coder-V2$0.17$0.14$0.2816 tokens/s128000 tokens1.12 s67
DeepSeek-V2DeepSeekDeepSeekDeepSeek-V2$0.17$0.14$0.2816.4 tokens/s128000 tokens1.15 s66
DeepSeek-V2.5DeepSeekDeepSeekDeepSeek-V2.5$0.17$0.14$0.2816 tokens/s128000 tokens1.13 s66
DeepSeek-V2.5HyperbolicDeepSeekDeepSeek-V2.5$2.00$2.00$2.007.6 tokens/s128000 tokens0.84 sN/A
Qwen2.5 72BHyperbolicAlibabaQwen2.5 72B$0.40$0.40$0.4034.4 tokens/s131000 tokens0.63 s75
Qwen2.5 72BDeepinfraAlibabaQwen2.5 72B$0.36$0.35$0.4035.8 tokens/s131000 tokens0.29 s75
Qwen2 72BFireworksAlibabaQwen2 72B$0.90$0.90$0.9058.9 tokens/s128000 tokens0.33 s69
Qwen2 72BDeepinfraAlibabaQwen2 72B$0.36$0.35$0.4031.1 tokens/s33000 tokens0.45 s69
Qwen2 72BTogether.aiAlibabaQwen2 72B$0.90$0.90$0.9065.4 tokens/s33000 tokens0.44 s69
Yi-LargeFireworks01.AIYi-Large$3.00$3.00$3.0063 tokens/s32000 tokens0.48 s58
GPT-4 TurboOpenAIOpenAIGPT-4 Turbo$15.00$10.00$30.0033.1 tokens/s128000 tokens0.67 s74
GPT-4 TurboMicrosoft AzureOpenAIGPT-4 Turbo$15.00$10.00$30.0049.5 tokens/s128000 tokens0.53 s74
GPT-3.5 TurboOpenAIOpenAIGPT-3.5 Turbo$0.75$0.50$1.5086.4 tokens/s16000 tokens0.43 s52
GPT-3.5 TurboMicrosoft AzureOpenAIGPT-3.5 Turbo$0.75$0.50$1.5089.1 tokens/s16000 tokens0.34 s52
GPT-3.5 Turbo InstructOpenAIOpenAIGPT-3.5 Turbo Instruct$1.63$1.50$2.00107.3 tokens/s4000 tokens0.41 sN/A
GPT-3.5 Turbo InstructMicrosoft AzureOpenAIGPT-3.5 Turbo Instruct$1.63$1.50$2.00127.3 tokens/s4000 tokens0.61 sN/A
Llama 3 70BReplicateMetaLlama 3 70B$1.18$0.65$2.7547.9 tokens/s8000 tokens0.55 s62
Llama 3 70BHyperbolicMetaLlama 3 70B$0.40$0.40$0.4029.2 tokens/s8000 tokens1.6 s62
Llama 3 70BAmazon BedrockMetaLlama 3 70B$2.86$2.65$3.5052 tokens/s8000 tokens0.46 s62
Llama 3 70BOctoAIMetaLlama 3 70B$0.90$0.90$0.9069.4 tokens/s8000 tokens0.34 s62
Llama 3 70BLepton AIMetaLlama 3 70B$0.80$0.80$0.8030.1 tokens/s8000 tokens0.82 s62
Llama 3 70BMicrosoft AzureMetaLlama 3 70B$2.90$2.68$3.5418.3 tokens/s8000 tokens0.81 s62
Llama 3 70BFireworksMetaLlama 3 70B$0.90$0.90$0.90107 tokens/s8000 tokens0.33 s62
Llama 3 70BDeepinfraMetaLlama 3 70B$0.36$0.35$0.4021.6 tokens/s8000 tokens0.32 s62
Llama 3 70BGroqMetaLlama 3 70B$0.64$0.59$0.79316.4 tokens/s8000 tokens0.23 s62
Llama 3 70B (Reference, FP16)Together.ai (Reference, FP16)MetaLlama 3 70B$0.90$0.90$0.90108.2 tokens/s8000 tokens0.65 sN/A
Llama 3 70B (Turbo, FP8)Together.ai (Turbo, FP8)MetaLlama 3 70B$0.88$0.88$0.8885 tokens/s8000 tokens0.51 sN/A
Llama 3 8BReplicateMetaLlama 3 8B$0.10$0.05$0.2558.7 tokens/s8000 tokens0.57 s53
Llama 3 8BAmazon BedrockMetaLlama 3 8B$0.38$0.30$0.6078.4 tokens/s8000 tokens0.35 s53
Llama 3 8BLepton AIMetaLlama 3 8B$0.07$0.07$0.0784.1 tokens/s8000 tokens1.08 s53
Llama 3 8BMicrosoft AzureMetaLlama 3 8B$0.38$0.30$0.6173.4 tokens/s8000 tokens0.48 s53
Llama 3 8BFireworksMetaLlama 3 8B$0.20$0.20$0.20118.3 tokens/s8000 tokens0.36 s53
Llama 3 8BDeepinfraMetaLlama 3 8B$0.06$0.06$0.06105.1 tokens/s8000 tokens0.21 s53
Llama 3 8BGroqMetaLlama 3 8B$0.06$0.05$0.081202.2 tokens/s8000 tokens0.3 s53
Llama 3 8BTogether.aiMetaLlama 3 8B$0.20$0.20$0.20291.8 tokens/s8000 tokens0.47 s53
Llama 2 Chat 70BReplicateMetaLlama 2 Chat 70B$1.18$0.65$2.7548.1 tokens/s4000 tokens0.71 s39
Llama 2 Chat 70BAmazon BedrockMetaLlama 2 Chat 70B$2.10$1.95$2.5636.7 tokens/s4000 tokens0.47 s39
Llama 2 Chat 70BOctoAIMetaLlama 2 Chat 70B$0.90$0.90$0.90174.1 tokens/s4000 tokens0.25 s39
Llama 2 Chat 70BMicrosoft AzureMetaLlama 2 Chat 70B$1.60$1.54$1.77 tokens/s4000 tokens s39
Llama 2 Chat 13BAmazon BedrockMetaLlama 2 Chat 13B$0.81$0.75$1.0053.1 tokens/s4000 tokens0.41 s36
Llama 2 Chat 13BOctoAIMetaLlama 2 Chat 13B$0.20$0.20$0.20170.8 tokens/s4000 tokens0.24 s36
Llama 2 Chat 13BTogether.aiMetaLlama 2 Chat 13B$0.30$0.30$0.3052.1 tokens/s4000 tokens0.5 s36
Llama 2 Chat 7BReplicateMetaLlama 2 Chat 7B$0.10$0.05$0.25124.3 tokens/s4000 tokens0.55 s29
Llama 2 Chat 7BMicrosoft AzureMetaLlama 2 Chat 7B$0.56$0.52$0.67 tokens/s4000 tokens s29
Gemini 1.0 Pro (AI Studio)Google (AI Studio)GoogleGemini 1.0 Pro$0.75$0.50$1.5097.7 tokens/s33000 tokens1.21 sN/A
Claude 3 SonnetAmazon BedrockAnthropicClaude 3 Sonnet$6.00$3.00$15.0047.1 tokens/s200000 tokens0.81 s77
Claude 3 SonnetAnthropicAnthropicClaude 3 Sonnet$6.00$3.00$15.0064.9 tokens/s200000 tokens0.79 s77
Mistral LargeMistralMistral AIMistral Large$6.00$4.00$12.0031.6 tokens/s33000 tokens0.76 s56
Mistral LargeAmazon BedrockMistral AIMistral Large$6.00$4.00$12.0037.3 tokens/s33000 tokens0.39 s56
Mistral LargeMicrosoft AzureMistral AIMistral Large$6.00$4.00$12.0040.1 tokens/s33000 tokens0.57 s56
Mistral Small (Feb '24)MistralMistral AIMistral Small$1.50$1.00$3.0076.6 tokens/s33000 tokens0.58 s50
Mistral Small (Feb '24)Microsoft AzureMistral AIMistral Small$1.50$1.00$3.0052.8 tokens/s33000 tokens0.45 s50
Mistral 7BMistralMistral AIMistral 7B$0.25$0.25$0.2596.6 tokens/s33000 tokens0.49 s24
Mistral 7BReplicateMistral AIMistral 7B$0.10$0.05$0.2561.7 tokens/s33000 tokens30.44 s24
Mistral 7BAmazon BedrockMistral AIMistral 7B$0.16$0.15$0.2079.1 tokens/s33000 tokens0.35 s24
Mistral 7BOctoAIMistral AIMistral 7B$0.15$0.15$0.15179.1 tokens/s33000 tokens0.21 s24
Mistral 7BLepton AIMistral AIMistral 7B$0.07$0.07$0.07102.9 tokens/s33000 tokens0.97 s24
Mistral 7BDeepinfraMistral AIMistral 7B$0.06$0.06$0.06111.1 tokens/s33000 tokens0.2 s24
Mistral 7BPerplexityMistral AIMistral 7B$0.20$0.20$0.20123.4 tokens/s16000 tokens0.22 s24
Mistral 7BTogether.aiMistral AIMistral 7B$0.20$0.20$0.20126.2 tokens/s8000 tokens0.32 s24
CodestralMistralMistral AICodestral$0.30$0.20$0.6047.8 tokens/s33000 tokens0.57 sN/A
Mistral MediumMistralMistral AIMistral Medium$4.09$2.75$8.1038 tokens/s33000 tokens0.86 s70
Llama 3.1 8BNebius AIMetaLlama 3.1 8B$0.03$0.02$0.0630 tokens/s128000 tokens s73
Llama 3.1 8BNebius AI FastMetaLlama 3.1 8B$0.04$0.03$0.09155 tokens/s128000 tokens s73
Llama 3.1 70BNebius AIMetaLlama 3.1 70B$0.20$0.13$0.4025 tokens/s128000 tokens s65
Llama 3.1 70BNebius AI FastMetaLlama 3.1 70B$0.36$0.25$0.70140 tokens/s128000 tokens s65
Llama 3.1 405BNebius AIMetaLlama 3.1 405B$1.50$1.00$3.0020 tokens/s128000 tokens s72
Mistral-Nemo-Instruct-2407Nebius AIMistralMistral Nemo$0.06$0.04$0.1230 tokens/s128000 tokens s52
Mistral-Nemo-Instruct-2407Nebius AI FastMistralMistral Nemo$0.12$0.08$0.24100 tokens/s128000 tokens s52
Mixtral-8x7B-Instruct-v0.1Nebius AIMistralMixtral 8x7B$0.12$0.08$0.2423 tokens/s33000 tokens s42
Mixtral-8x7B-Instruct-v0.1Nebius AI FastMistralMixtral 8x7B$0.23$0.15$0.45143 tokens/s33000 tokens s42
Mixtral-8x22B-Instruct-v0.1Nebius AIMistralMixtral 8x22B$0.60$0.40$1.2023 tokens/s65000 tokens s61
Mixtral-8x22B-Instruct-v0.1Nebius AI FastMistralMixtral 8x22B$1.05$0.70$2.10135 tokens/s65000 tokens s61
Qwen2.5-Coder-7BNebius AIQwenQwen2.5-Coder-7B$0.04$0.03$0.0970 tokens/s32000 tokens s74
Qwen2.5-Coder-7BNebius AI FastQwenQwen2.5-Coder-7B$0.31$0.31$0.30132 tokens/s32000 tokens s74
Qwen2.5-Coder-7B-InstructNebius AIQwenQwen2.5-Coder-7B$0.04$0.03$0.0970 tokens/s32000 tokens s74
Qwen2.5-Coder-7B-InstructNebius AI FastQwenQwen2.5-Coder-7B$0.31$0.31$0.30132 tokens/s32000 tokens s74
DeepSeek-Coder-V2-Lite-InstructNebius AIDeepSeekDeepSeek-Coder-V2-Lite$0.06$0.04$0.1230 tokens/s128000 tokens sN/A
DeepSeek-Coder-V2-Lite-InstructNebius AI FastDeepSeekDeepSeek-Coder-V2-Lite$0.12$0.08$0.2450 tokens/s128000 tokens sN/A
Phi-3-mini-4k-instructNebius AIMicrosoftPhi-3-mini-4k$0.06$0.04$0.1313 tokens/s4000 tokens sN/A
Phi-3-mini-4k-instructNebius AI FastMicrosoftPhi-3-mini-4k$0.20$0.13$0.4040 tokens/s4000 tokens sN/A
OLMo-7B-InstructNebius AIAllenAIOLMo-7B$0.12$0.08$0.2425 tokens/s2000 tokens sN/A