If you’ve followed me long enough, you know one thing I constantly wail about when it comes to running my OpenClaw or multiple Hermes AI assistants (yes, plural, assistants as in 6 in total) is tokens. Tokens. TOKENS. Whether I’m having daily conversations with them, getting them to draft blog posts, research topics, build simple sites, push code to GitHub, whatever it is, the more complex the task, the more the dollar bills quietly stack up in the background. So you can imagine how I felt when I saw a tweet about routing open source AI models through Hermes and using them for close to nothing. Not good news. Great news. Let me explain what I’m changing and why.
The Token Problem Nobody Talks About Honestly
When people write about setting up AI agents, they focus on the exciting parts. The automation. The productivity. The feeling of having a digital team working for you around the clock. What they skip is the bill.
Here is what actually happens. Your agent wakes up, checks your emails, summarises three threads, drafts a reply, does a quick web search, formats the output, and logs the session. That is maybe fifteen minutes of work. Multiply that across six agents, across a full day, across a week. Every single one of those actions costs tokens. And if your agents are running on a premium model like GPT-5.2, you are paying premium prices for tasks that genuinely do not need that level of horsepower.
I have been running agents on my Hetzner VM for months now. The productivity gains are real. But so is the creeping API bill. And the more I build, the more I automate, the more that number grows.
What Hugging Face Just Changed
A tweet from Merve at Hugging Face recently caught my attention. Hugging Face, for anyone unfamiliar, is the largest open source AI model platform in the world. Think of it as a massive library of AI models that researchers and developers publish openly for anyone to use.
What changed is that Hermes Agent, the open source AI agent framework I run on my server, now has Hugging Face as a first class provider. That means instead of only routing your agent through paid API providers like OpenAI or Anthropic, you can point it at hundreds of open source models through a single Hugging Face endpoint. Models like Qwen, DeepSeek, and Llama, which are genuinely capable, not toy models.
The Hugging Face free tier gives you a small starting credit each month. It is not unlimited and it will not replace your premium model entirely. But that is actually not the point.
The Two Model Setup
This is the part that made me sit up. The smart play here is not replacing your main AI model. It is building what I am calling a two lane system.
Lane A stays exactly as it is. Your premium model, in my case GPT-5.2, handles the hard stuff. Complex reasoning, tool orchestration, multi-step research, anything where quality genuinely matters and a cheaper model might let you down.
Lane B is where Hugging Face comes in. Routine tasks like summarising content, drafting first passes, reformatting text, simple lookups, these do not need a frontier model. They need something fast, capable enough, and cheap. Through Hugging Face I now have three verified working options depending on what the task needs. For fast and cheap work, meta-llama/Llama-3.1-8B-Instruct running on Cerebras is the go-to. For tasks that need a bit more reasoning, deepseek-ai/DeepSeek-R1 via Together handles that lane. And for times when I want a big model without paying frontier prices, openai/gpt-oss-120b is available through HF with automatic routing to the fastest or cheapest backend available at that moment. Twenty eight curated models are available directly in the Hermes model picker organised by use case, with a custom entry field that unlocks 100 or more additional models beyond that.
Hermes handles this natively through a delegation config in your setup file. When your main agent spawns a subagent to handle a writing or formatting task, you can configure that subagent to run on a different, cheaper model automatically. No third party tools. No complicated routing logic. A few lines in a config file.
Here is what that looks like in practice for me. My primary Hermes conversation stays on GPT-5.2. But when I ask it to draft a social media post or summarise a document, it delegates that to a subagent running on a Hugging Face model. I get the result. I pay a fraction of the cost.
What This Does Not Do
I want to be straight with you because I know how these things get written up online.
This is not a magic switch. Hermes does not automatically know which tasks are simple and which are complex. You configure the delegation rules yourself, which takes a bit of setup time upfront. Per skill routing, where each individual skill your agent runs uses its own model, is still an open feature request as of May 2026. So right now the routing happens at the subagent level, not the task level.
Tool calling, where your agent needs to interact with external services, APIs, or perform structured actions, is also safer on your premium model for now. Some open source models handle tool calls inconsistently. Until I have tested specific models thoroughly, I am keeping anything mission critical on GPT-5.2.
And the Hugging Face free tier, while real, is genuinely small. Think of it as a testing budget rather than a production solution. Beyond the free tier you pay per token, though at significantly lower rates than frontier models.
One question worth addressing if you already use OpenRouter in your setup: no, this does not replace it. They do different things. OpenRouter is a routing layer that connects you to commercial and open source models like Kimi K2.6, DeepSeek, and others through their actual provider servers. Hugging Face Inference Providers is more like a dedicated fast lane specifically for open source models, routing your requests to whichever backend, whether that is Groq, SambaNova, or Cerebras, is fastest and cheapest at that moment. I run both. OpenRouter stays configured for model flexibility and commercial model access. Hugging Face sits alongside it as a second option specifically for routine open source model tasks. Think of them as two tools in the same drawer, not two tools competing for the same job.
Why This Matters Beyond the Money
There is a bigger picture here that Merve’s tweet hinted at. The direction AI agents are heading is local and cheap. The gap between open source models and frontier models is closing faster than most people expected. A year ago you would not have trusted an open source model with anything important. Today some of them are genuinely excellent at specific tasks.
What Hugging Face and Hermes are building toward is an agent that can intelligently use the right model for the right job, automatically, without you thinking about it. We are not fully there yet. But the infrastructure is being laid right now and the people building it are moving fast.
For someone like me, running six agents on a self-hosted server while watching my API costs, this is exactly the kind of development worth paying attention to. And honestly, worth experimenting with now before it becomes mainstream advice.
Join The August Dispatch Newsletter
For curated news, practical prompts, and how‑to guides—plus first dibs on workshops.
How to Actually Try This
If you run Hermes Agent and want to explore this yourself, the steps are straightforward.
Create a free account at huggingface.co. Go to your account settings and generate an access token, making sure to enable the option to make calls to Inference Providers. SSH into your server, run hermes model, select Hugging Face from the provider list, and paste your token. From there you can configure your delegation settings in your Hermes config file to route subagent tasks to a HF model of your choice.
Right now I switch lanes manually, which is actually the sensible way to start. My default stays on GPT-5.2. When I want the cheap lane I either run my HF helper script or pass --provider huggingface with a chosen model directly to Hermes. No magic, no automation yet, just a deliberate choice based on what the task actually needs.
Phase 2, which I am planning next, is proper automatic routing. The idea is that summarise, draft, and format requests automatically go to Llama 8B on Cerebras, anything requiring deeper reasoning goes to DeepSeek R1, and tool calls or mission critical tasks stay on GPT-5.2. That needs a small routing script to work cleanly. I will write about that when it is built and tested.
FAQ
How do I stop my AI agent from using expensive tokens for simple tasks? The most practical way right now is to use a delegation config in Hermes Agent that routes subagent tasks to a cheaper open source model while keeping your main model on premium. It takes some upfront configuration but the cost difference on routine tasks is significant.
Can Hermes Agent use free AI models from Hugging Face? Yes. Hugging Face is now a first class provider in Hermes, meaning you can connect it with a free account and access hundreds of open source models through a single endpoint. The free tier credit is small so treat it as a testing budget rather than a full production solution.
What is the cheapest way to run an AI agent in 2026? Combining a self-hosted setup on a low cost server like Hetzner with a two model routing approach gets you close to the floor. Use open source models for routine tasks and reserve premium API calls for complex reasoning. The platform itself, Hermes Agent, is free and open source.
The honest takeaway is this. Token costs are a real and growing problem for anyone building seriously with AI agents. The solution is not to stop building. It is to build smarter. Routing the right tasks to the right model is not a workaround. It is just good engineering. I am still testing this setup and I will share what I find as I go, which is exactly what you can expect if you join the August Wheel newsletter at newsletter.augustwheel.com. Real experiments, honest results, no hype.
Leave a Reply