Ruonan Wang 64b38e1dc8 llm: benchmark tool for transformers int4 (separate 1st token and rest) (#8460) ... * add benchmark utils * fix * fix bug and add readme * hidden latency data		2023-07-06 09:49:52 +08:00
..
llm	llm: benchmark tool for transformers int4 (separate 1st token and rest) (#8460)	2023-07-06 09:49:52 +08:00