Small LLM on Intel i5: 36 Tokens/Second Reality
Testing five models on an Intel i5 reveals 36 tokens per second is the ceiling for small LLMs due to 20 GB/s memory bandwidth limits.
Priya Nair
15 min read