One Telegram contact to rule my team of AI agents
Siri doesn’t understand Argentinian Spanish. I have to pronounce some words in a neutral accent, which sounds weird here, and my toddlers learn it as local, or write them out, navigating the awkward iOS cursor, or tap each letter and then correct typos. I use calendar, timers, notes, and reminders as anyone else, and I’ve been a “tech person” since my early years.
So I was surprised by my own surprise when Fede, my brother-in-law, repeated my ask to his Telegram agent: “remind me to pick up the kids at 11:30”. Well that was simple, and it wouldn’t rely on our memory. He didn’t write or pronounce how Siri expects, he just said my words to his phone. I’ve long wanted to schedule calendar events via voice while on the go. Siri promised this for long too, but it never quite delivered.
Three weeks later I had five specialized AI agents running on my iMac under an orchestrator.
Need zero, pick an agent architecture
I’ve been meaning to implement agents but struggled to pick a solution.
- RubyLLM looked too low-level (last month): I was looking for a simple system but with predefined structure. It’s quickly evolving though
- OpenClaw is powerful but ginormous, a big bundle of prompts, and I would never fully control the details of its functioning
- NanoClaw seemed closer to what I’d like, but if it’s so simple, why not roll my own from scratch?
- My own from scratch: where do I even start?
- Mastra, a TypeScript framework for building AI agents, initially felt like much more than I needed. I eagerly watched Damian Galarza’s videos, but I couldn’t predict what exactly each dashboard item does
My ex coworker Anthony Costanzo provided an onramp I needed: he built his Batcomputer from the ground up, and shared his notes. He used familiar tools for programmers: a long-running Claude Code session within tmux, reachable via Telegram with Tailscale, launching on boot using macOS’s LaunchAgents, and a directory of markdown files for memory with auto-commit. This was right up my alley, it would help me delineate what “my own agent” even means.
I added voice note transcription via OpenAI’s whisper, registered a Google account for my agent (Dino) with API access, and a new macOS non-admin user on my iMac for isolation.
Dino also sends me a daily morning brief with my schedule and unread emails. It can also send emails, but with this implementation I worried it may spam people even though its prompt said “don’t without my consent”. A prompt does not suffice to guarantee such rules.
I gave him readonly access to my email and calendar: I don’t want an agent to impersonate me, nor do I want it to exfiltrate or commit mistakes with my data.
It worked, but showed rough edges:
- Screen-scraping tmux couldn’t distinguish a tool working from a finished response
- Adding a second agent meant a second tmux session, a second polling loop, and hand-rolled connections between them
- Multi-turn context lived only in the pane’s scrollback buffer
- Memory wasn’t efficient
- Iterating on prompts and testing them with real usage felt like when I updated PHP websites before learning about automated tests
It proved the concept: a personal agent I can configure with precision, reachable from my phone, understanding both my English and Argentinian Spanish. But I wouldn’t grow it on that early foundation.
I took another look at the implementation options. I rewatched Damian’s videos on Mastra and now it clicked. “Framework” means decisions taken for me, in this case proposing a sensible structure. Mastra has a declarative API that makes for readable code. I rewrote Dino in Mastra. And now I could understand its features.
Like, add a “Human In The Loop” approval requirement before sending emails. Now I can comfortably ask Dino to try sending an email, inspect what it plans to do, and only then approve the operation or request changes.
First need: Maybe a second agent
As Dino proved its worth, I started asking more of it. It started remembering how I use calendar, email, Telegram, plus the communication styles I prefer. And it began mixing preferences up.
Another problem: every request was handled by the smart (and expensive) LLM model, but some simpler operations can be done by cheaper models, in turn faster.
On May 10th I split an orchestrator and a worker. Manuel became the smart orchestrator handling Telegram conversations, memories, and deciding delegations. Dino kept its calendar and email fu, becoming a simpler, more focused utility agent who knew one thing well. Dino doesn’t have much of a personality or agency, but boy does it fill event details and invitees from my natural language instructions properly. No forms, double verification of whether I’m picking the right week, or timezone musings needed anymore.
This pattern unlocked everything that follows.
You may think: how do you keep track of names that don’t even have faces? I don’t. I have a single Telegram contact to access everything: Manuel. I say what I need in either language I’m using, and Manuel routes it. I couldn’t build a simpler UX for this functionality, and that’s why it sticks unlike other “intelligences” and apps I’ve tried before.
Second need: self-improving agents
Sometimes I’d ask Manuel to make changes to the project itself:
“Fix all-day events time boundaries.”
“Add support for recurring events”
Manuel would counter he has no tools to write code. I’d sit on my computer, copy the failed request into Cursor, and redeploy.
On May 14th I added Topo, a sub-agent with Mastra’s Workspace API, allowing access to the project’s directory and to relevant shell commands.
I can now tell Manuel to add a capability to an agent, and it’ll delegate to Topo. The sub-agent implements it, and reports back the generated commit SHA as verifiable evidence of work (without it, LLMs can and will report back enthusiastic success with no changes done).
The friction between “I want this feature” and “the feature exists” dropped to near zero. And in my agents team I optimistically let it deploy its changes: most often it works, and when not it’s easy for me to roll back or fix. I still review all commits to tweak anything needed and to learn, but not before letting it update itself.
Third need: invoicing for my company with Tito agent
For my company each month-end I have to:
- Review each client’s GitHub repos and task managers to gather links to work done
- Write a monthly report summarizing all
- Generate and email an invoice
- Commit compiled notes to my
railspilot-documentsrepository
It’s not a terribly laborious or time-consuming task, but it did distract me the last days of each month with something that didn’t seem to deserve my time. That feeling again, this mustn’t be hard to automate well.
On May 29th, via Telegram request to Manuel, I created Tito, a RailsPilot bookkeeper agent with a RailsPilot-documents workspace. Tito reads
clients.yml to list clients and their relevant URLs, fetches merged PRs via git log and GitHub CLI, tasks to my name from their boards, writes
the report, generates the invoice, and pushes its changes.
I say “May reports” and it does the work, I review the diff, apply small adjustments, and send.
Each client also has a status file that both Tito and I keep up to date, so it can tell me what’s next for each client, and any deadline to keep top of mind.
I speak with clients and interpret their needs, which I try to fulfill as well as I can. I love that work. Tedious tasks that are easy to delegate shouldn’t eat into it.
Fourth need: lights on when I pull into the driveway
The underlying need was implementing yet another agent. Now I got greedy. Lower value than the previous one, yet still satisfying.
I have SmartLife (Tuya) smart switches for some lights. Some evenings I arrive home and enter through the back door, and it’s too dark to find the right key. I would pull my phone’s SmartLife app, tap the very small button next to “Kitchen Lights”, and enter.
On June 3rd I added Jaime, an agent who talks to the Tuya Cloud API to control light switches. Now I tell Siri via my car “phone” button to Telegram Manuel:
“Encender las luces de la cocina”
And it does, while I park. It’s fun, simple, helpful, and has that “wow factor” that lets people who never thought of agents (and who will never read this blog post) see what they can do. Like my brother-in-law when he offloaded a reminder to his agent while waiting for a traffic light.
Jaime may be a shallow and simple agent. He’s still as smart as his Haiku model: he interprets “kitchen”, “all”, “back” or “exterior lights” without me having to predefine groups in an app. People could have done this with Siri, Alexa, and others for years, but while I know many who own them, I know that one weird person who actually uses them in daily life. She also sells them. But almost everyone I know uses a chat app like Telegram every waking hour. The user experience is so much better.
Fifth need: writing this article (Fede)
I called this one Fede, honoring my brother-in-law’s blog. I’m glad he started it, I’m quite enjoying it and you may too.
When I thought of writing this piece, I first built a subagent to draft it in my voice. It has:
- Read-only access to this Mastra code so he could inspect and describe it
- Email tool via Dino to email drafts when done (Manuel’s approval button goes through Fede to Dino. After a bug fixed in Mastra, it works surprisingly well)
- Instructions about my voice, audience, and topics I cover
I told Manuel (typo included):
“Plese write a blog post describing this project”
He delegated to Fede, who read the git history and agent definitions, and wrote two drafts of this post.
Lessons I took, and next steps
Specialized agents beat one general agent. When Manuel started delegating, every interaction got more fruitful. Each agent has a focused set of tools, specific instructions, and a model tuned to its task’s cost and complexity. Focused agents are easier to debug and simpler to extend, like classes in Ruby.
Evidence over claims. Dino has to pass a proof of work before reporting an event created. If it says “done, I scheduled X” but the response doesn’t include the event ID, execution fails and the task is retried. Same principle for Tito and git commit SHAs.
Memory is a sharp tool. Subagents have memory disabled except for the current thread. Memories add to their context, and when not relevant it can lead them astray. The relevant context is in the tools, not in past conversations. Manuel does have memory, because he orchestrates: he may need to edit an event just created, or retry a task that didn’t properly finish.
A friend asked me: who should use agents like these? Pretty much everyone I know. Anyone with a job that has routines will benefit. “This is tedious, could an agent help me do it?” The answer is most often, yes.
The pattern works: identify a repetitive task, design and implement an agent and its tools, and assign it tasks. If you want to build something similar, start with one agent and one real problem. Try Mastra. Tell me how it goes!
Some technical notes if you care for more detail
Summary of my AI team so far, their tools and models:
| Agent | Role | Model | Key tools |
|---|---|---|---|
| Manuel | Orchestrator | Opus | Telegram, memory |
| Dino | Assistant | Haiku | Calendar and email |
| Topo | Self-improvement | Opus | Workspace, shell |
| Tito | Invoicing, progress | Haiku | GitHub, file system |
| Jaime | Smart lights | Haiku | Tuya Cloud API |
| Fede | Writing drafts | Sonnet | Email via Dino |
The system runs on my iMac as a macOS LaunchAgent under a dedicated user account.
Secrets live in 1Password, loaded as ENV variables at runtime.
Telegram webhooks reach Manuel via a Tailscale Funnel.
I’m using about $60 a month in API usage with it, it feels very high value for low cost to me.
I expect to grow this system with more responsibilities, and learn about evolution, and maintainability.
I’m particularly curious about evals: how do I know an agent is doing its job well beyond “it seems to work”? That’s my next path.