ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Fix vLLM CPU /chat endpoint (#11748)

Browse source
This commit is contained in:
Xiangyu Tian 2024-08-09 10:33:52 +08:00 committed by GitHub
parent 27b4b104ed
commit 044e486480
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 0 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
python/llm/src/ipex_llm/vllm/cpu/entrypoints/openai/api_server.py
View file

@ -114,7 +114,6 @@ async def create_chat_completion(request: ChatCompletionRequest,
return StreamingResponse(content=generator, return StreamingResponse(content=generator,
media_type="text/event-stream") media_type="text/event-stream")
else: else:
invalidInputError(isinstance(generator, ChatCompletionResponse))
return JSONResponse(content=generator.model_dump()) return JSONResponse(content=generator.model_dump())