AI on Your Gaming PC

If you want to experiment with LLMs, you typically have a choice of sending your requests to someone else’s computer or fielding a very large GPU and CPU setup to run models locally. However, a recent crop of projects aims to bring bigger models to much more modest hardware. One example is Strata , a project from [Niko1221], which lets you run a 125-billion-parameter LLM on hardware you might already have for gaming.