If you’re having Claude problems, I feel bad for you, son / I got 99 problems, but AI ain’t one.

Local AI models can’t replace cloud AI models, much like a Peugeot 207 that gets outcompeted by a Panamera on the highway. Nevertheless, an old Peugeot does a great job taking me from A to B. That’s how I see local models: they already do great at specific tasks, especially for SMEs with limited resources.

I’m currently working on a project for an interactive knowledge base that updates itself iteratively, using a single Jetson Orin Nano 8GB to transcribe audio. Running locally, the Jetson has so far transcribed more than 600 hours of support agent talks, filled with technical lingo, a mix of Swedish and English words, and numerous dialects. Not bad for a tiny device, huh? Even at moderate cloud transcription prices, a $350 Jetson will pay for itself several times over within the first month.