ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Fix: Correct num_requests in benchmark for Pipeline Parallel Serving (#11462)

Browse source
This commit is contained in:
Xiangyu Tian 2024-06-28 16:10:51 +08:00 committed by GitHub
parent b7bc1023fb
commit fd933c92d8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
python/llm/example/GPU/Pipeline-Parallel-FastAPI/benchmark.py
View file

@ -165,6 +165,7 @@ def benchmark(
total_time = time.perf_counter() - start_time total_time = time.perf_counter() - start_time
log_file = f"{max_concurrent_requests}.log" log_file = f"{max_concurrent_requests}.log"
num_requests = num_requests - num_warmup_requests
with open(log_file, "w") as file: with open(log_file, "w") as file:
print( print(