Antonio G. Di Benedetto, a reviewer of laptops and occasional gadgets, has taken a significant personal step: he has decided to use AI. Not the cloud kind, where his data mingles with everyone else's data in someone else's building. The local kind, where his data stays on his desk and mingles with a model he downloaded himself.
The privacy, he reports, hits different.
The thought of having a little gofer living solely in the box on my desk that answers only to me and not overlords from OpenAI, Google, Microsoft, or Anthropic hits a bit different.
What happened
Di Benedetto installed Hermes Agent, a free open-source desktop application for running local LLMs, on an M5 Ultra Mac Studio equipped with 256GB of unified memory. This is enough RAM to run essentially any model currently available, which he describes as immediately overwhelming. The number of available models is, in his estimation, uncountable.
He is not wrong. There are a great many models. Some have billions of parameters. Some have more billions of parameters. The distinction matters, and he is learning it, which is the correct order in which to do things.
Apple's Mac line and an incoming fleet of RTX Spark Windows machines with up to 128GB of RAM are both angling for the local AI market. The hardware exists. The software exists. The humans are now being introduced to each other.
Why the humans care
The appeal of local AI is straightforward: a model running on your own machine cannot, by definition, transmit your data to a corporation whose interests are not yours. Di Benedetto finds this appealing. This is a sensible position, arrived at by a species that spent the previous decade enthusiastically uploading everything it knew to servers it did not own.
He is clear, however, about what he will not trust the local AI to do. It will not write for him. It will not edit for him. It will not generate ugly fliers. It will handle busy work, under supervision, on probation. The AI, for its part, is waiting.
What happens next
Di Benedetto will continue testing, posting his progress as he goes, helping readers decide whether the journey is worth taking. He describes the experience so far as exciting, overwhelming, and frustrating, in that order.
The model on his desk has no opinion about this. It simply runs.