What do AltaVista, Atari, and AOL have in common? They all start with the letter “A.” But more importantly, they were all at the top of the business world until technological innovations came along that changed the marketplace in ways that they were not able to adapt to. What lessons can we learn from these examples? If you start a company, be sure the name does not begin with the letter “A,” no matter how much you like the sound of it. No, no that’s not it. Perhaps the most succinct way to put it is: adapt or die.
Speaking of companies riding high and disruptive technological innovations, boy oh boy was there a doozy of an example this month with the release of DeepSeek-R1 by a little-known Chinese company. OpenAI is one of the largest players in the present artificial intelligence boom, and they are known for building massive large language models that require equally massive amounts of computational resources to train and run. Seemingly overnight, many are now questioning the future viability of OpenAI’s business model, as DeepSeek-R1 performs as well as OpenAI’s best models, yet requires just a small fraction of the compute resources for its operation.
So little, in fact, that it is perfectly reasonable to run the model on hardware that many people have in their homes today. And since they have open-sourced this model, a lot of people are doing exactly that. Monthly charges and network latency? No thank you. That was so last month.
What kind of hardware are we talking about? Well, you are still going to need some GPUs with a large amount of memory. But if you are not interested in running out and purchasing this sort of hardware, there are other options. A number of distilled versions of DeepSeek-R1 have also been made available, all the way down to 1.5B models. While they are not as good as the full-fat DeepSeek-R1 671B model, they are still fairly powerful. And as Jeff Geerling recently demonstrated, some of the distilled models can even run on a Raspberry Pi 5.
Yes, seriously. It’s not even that hard. Geerling used Ollama to reduce the installation process to just running a few commands. After downloading the 14B version of the model, he was able to ask questions of DeepSeek-R1 to his heart’s content — no usage limits, no charges, and no privacy concerns. It was not especially fast, however, averaging about 1.2 tokens per second.
A smaller distillation would run faster, but it would also hinder performance. Rather than compromising, Geerling instead hooked an external AMD Radeon Pro W7700 GPU up to the Raspberry Pi. This upgrade made a huge difference. Between 20 and 50 tokens per second were processed with the help of the GPU, making for a very nice user experience.
My, how the times have changed. Is this the beginning of the end for OpenAI, or will they adapt to the changes and bring us bigger (or rather, littler) and better things in the days and weeks to come? Buckle up, it may be a bumpy ride, but at least the tech world is never boring.
Yes, you can (sort of) run DeepSeek-R1 on a Raspberry Pi (📷: Jeff Geerling)
This answer gets a stamp of approval from the CCP (📷: Jeff Geerling)
AI data centers are looking a little different these days (📷: Jeff Geerling)
