ipex-llm/python/llm/dev/benchmark/all-in-one/results.csv
2023-09-01 10:48:00 +08:00

146 B

1model1st token avg latency (ms/token)2+ avg latency (ms/token)input/output tokens
20llama2232.4256.1932/32
31llama29465.5768.671024/128