ayo/ipex-llm
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

fix llama on cpu (#12018)

Browse source
This commit is contained in:
Guoqiong Song 2024-09-04 19:17:54 -07:00 committed by GitHub
parent b3b2cd64b4
commit 8803242f5c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/llm/src/ipex_llm/transformers/models/llama.py
View file

@ -1579,7 +1579,7 @@ def llama_attention_forward_4_41_original(
past_key_value.key_cache[self.layer_idx] = key_states past_key_value.key_cache[self.layer_idx] = key_states
past_key_value.value_cache[self.layer_idx] = value_states past_key_value.value_cache[self.layer_idx] = value_states
if cache_position is not None: if attention_mask is not None:
new_attention_mask = attention_mask[:, :, :, 0:kv_seq_len] new_attention_mask = attention_mask[:, :, :, 0:kv_seq_len]
else: else:
new_attention_mask = attention_mask new_attention_mask = attention_mask