LLM Performance And Acceleration: Part 1

At Imagination, we are working to accelerate LLMs on everyday devices. In the first blog of this new two-part series on LLM performance and acceleration, we introduce the key performance metrics: Time to First Token (TTFT) and Inter-Token Latency (ITL). In the next installment, we will share the work we’re doing on bringing efficient Llama.cpp inference to Imagination GPUs. If you’ve seen Google’s “AI Overview” or Word predicting your next word, that’s LLMs at work. They’re built on transformer