ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

LLM: fix llama 4.36 forward(#10047)

Browse source
This commit is contained in:
Ruonan Wang 2024-01-31 10:31:10 +08:00 committed by GitHub
parent 53a5140eff
commit 3685622f29
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/src/bigdl/llm/transformers/models/llama.py
View file

@ -638,7 +638,7 @@ def llama_attention_forward_4_36(
"Please make sure use `attention_mask` instead.`" "Please make sure use `attention_mask` instead.`"
) )
bsz, q_len, _ = hidden_states.size() bsz, q_len, hidden_size = hidden_states.size()
device = hidden_states.device device = hidden_states.device
# for flash attention # for flash attention
original_dtype = hidden_states.dtype original_dtype = hidden_states.dtype