Choosing When Hermes Agent Automations Need an LLM

The music did not play, and I could not immediately tell why.

Hermes Agent had accepted my Spotify request and issued a command. From the conversation, the automation looked successful. The missing output was the only obvious sign that something had gone wrong.

That is tolerable for an occasional request. It is a bad property for an automation I expect to run every morning.

I still want Hermes involved because describing an automation is much easier than building one through a sequence of forms. I just do not want an LLM interpreting the same fixed instruction every time it runs.

My choice is to use Hermes for intent and configuration, then give repeatable execution to Home Assistant or a script. The model helps create the rule. The system with the clearer logs and more predictable failure modes owns the schedule.

A successful command is not the same as a successful outcome

Hermes has a native Spotify plugin with structured tools for search, playback, devices, queues, playlists, albums, and the library. The plugin can successfully send a playback command while the room remains silent because the selected device is unavailable, Spotify has no active playback target, or another service fails afterward.

If every scheduled run begins with natural language, another set of failures appears before Spotify is even involved:

• the model can interpret the wording differently;
• the provider can be unavailable or rate-limited;
• the run consumes model usage;
• the agent can select the wrong tool, device, or sequence.

Hermes supports provider fallbacks and credential rotation, which help agent jobs recover. They do not change the underlying trade-off: a routine action now depends on a model provider before it can reach the speaker.

Home Assistant does not guarantee that Spotify will work either. What it gives me is a stable execution path and a trace. Its automation traces show what triggered, which conditions passed, which actions ran, and where an error occurred. Spotify uses standard media_player actions and states, so I can inspect the target rather than ask a model to reconstruct the run from a conversation.

The distinction is observability, not magic reliability. When something fails, I want to know whether the schedule fired, which command ran, and what the device reported.

Interpretation and execution are different jobs

“Play something calm downstairs” needs interpretation. “Play this playlist on the kitchen speaker at 7:00 every weekday” does not, once the playlist, speaker, and schedule are known.

That gives me a practical boundary:

Anthropic makes a similar distinction in Building Effective Agents: predefined workflows provide consistency for well-defined tasks, while agents earn their extra cost when the work requires flexibility and model-directed decisions.

An automation can move from one side of the table to the other. The setup conversation may be ambiguous. Once I have resolved the playlist, speaker, and schedule, future runs are not.

The model is useful at setup time

Home Assistant is easier to debug after an automation exists. Creating that automation through its interface can take more effort than the rule deserves. I have to choose triggers, entities, conditions, actions, modes, and service data, even when the desired outcome fits in one sentence.

This is the kind of setup I give Hermes:

Every weekday at 7:00, play my morning playlist on the kitchen speaker at 35% volume. If the speaker is unavailable, notify me instead of failing silently.

Hermes can interpret that request, identify the missing entity names, and produce structured configuration. I review the rule once. Home Assistant runs it afterward.

That division also means I do not need an LLM every time the clock reaches 7:00. I avoid repeated token cost, provider limits, and another chance for the same intent to be interpreted differently.

For fixed non-device checks, Hermes has a similar escape hatch. Its no_agent cron mode runs Bash or Python directly, without a model call or agent loop.

A threshold check should have a simple contract:

Silence should mean that nothing needs attention. It should not also mean that the job failed before checking.

The failure path decides the architecture

Before I trust a recurring automation, I now want five answers:

1. What starts it?
2. Does the run still require interpretation?
3. Which system performs the action?
4. Where can I inspect the run?
5. How does failure become visible?

If I cannot answer the second question with a reason, I remove the model from the runtime. If I cannot answer the fourth and fifth, the automation is not finished.

This is also a useful security boundary. Playing music needs access to a media player, not the filesystem and an unrestricted shell. A stock threshold needs one data source and one notification target, not every tool available to Hermes. A narrower execution path is easier to debug and gives unattended work fewer capabilities.

The composition I am keeping

My preferred path now looks like this:

Describe the outcome to Hermes
        ↓
Resolve ambiguity and create the rule
        ↓
Save it in Home Assistant or as a script
        ↓
Run without an LLM
        ↓
Keep a trace, job result, or alert

I still use agent-driven jobs for research, synthesis, and writing because there is a new interpretation to perform on every run. I use ordinary automations for music, reminders, device control, and known thresholds because repeatability is the product feature.

Hermes stays on the setup side. Home Assistant or a script owns the recurring run. That leaves me somewhere useful to look the next morning when the music does not play.