Track: building your own tools
This is the last track and the one the others lead to. At some point you notice you are retyping the same request, pasting the same context, and doing the same three steps in the same order. That is a tool trying to exist.
You do not need to be a programmer for the first half of this. You do for the second, and the gap between them is smaller than it was.
---
The signal
You are ready for this track when one of these is true:
- You have retyped roughly the same request more than five times.
- You paste the same background material in before every question.
- You do a sequence — export this, paste it there, ask for that, put the answer somewhere — more than weekly.
- You have explained your setup to a chat box so many times that you have the paragraph memorised.
Each of those is a small tax paid repeatedly. The point of this track is to pay it once.
---
Stage 1 — the saved prompt
The smallest possible tool, and the one most people skip.
Write the request down properly — the role, the audience, the format, the constraints — and put it somewhere you will find it. That is it. That is a tool.
What makes a good one. It survives being used on a slightly different input. If it only works on the exact thing you wrote it for, it is a one-off; if it works on this week's version too, it is worth keeping.
Improve it when it disappoints you rather than rewriting it. A prompt with six months of small corrections in it is dramatically better than anything you would compose fresh, and the corrections are all things you learned the hard way.
What goes wrong. Keeping them in a chat history. Chat histories are not searchable in any way that helps, and a tool you cannot find is not a tool.
---
Stage 2 — the prompt with a slot
One prompt, many inputs.
The next step is a prompt where the variable part is marked: the standing instructions stay fixed and you drop today's material in.
Most chat tools now support this directly — saved prompts, custom instructions, projects that carry context, custom GPTs and their equivalents. Any of them will do. The mechanism matters far less than the discipline of separating what is always true from what is true today.
What goes wrong. Putting today's material in the standing part. Then it goes stale silently and you get last month's answer to this month's question — which is the worst failure in this whole track, because nothing about it looks wrong.
---
Stage 3 — the assembled context
Something else gathers the material.
At this point the thing costing you time is not writing the prompt, it is collecting what goes with it: this week's notes, the current state of the project, the last three conversations.
Two ways forward.
Use a tool that already does it. Anything that keeps your material and sends the relevant parts with your question. This is what AIOS is — you write in a daily note the way you would on paper, and the plan, the calendar and the context that goes to the model are read out of what you wrote. It exists because I got to this stage and wanted the assembly done for me.
Or write twenty lines of script. Read the files, paste them into a prompt, call an API, print the answer. This is genuinely small — see stage 5 of the coding track — and it is the step where "using AI" becomes "building with AI".
What goes wrong. Sending everything. More context is better only up to a point; past it the important part gets diluted and the cost goes up. Send what bears on the question.
---
Stage 4 — the tool that acts
It does something, not just says something.
The jump from answering to doing. The model decides, your code acts: file the receipt, send the draft, update the row, open the ticket.
Start read-only. A version that only ever tells you what it would do, for a week. You will be surprised at least twice, and finding that out on a report costs nothing.
Then let it act, with an undo. Version control, a trash folder, a draft state instead of a sent state. The coding track makes the same point about git and it is the same principle: an action you can reverse is one you can automate, an action you cannot is one you have to supervise — which removes the point.
Never let it act on something irreversible without a person in the loop. Sending, publishing, paying, deleting. This is not caution, it is where the whole approach falls down if you get it wrong once.
What goes wrong. Building the interesting part and skipping the boring part: what happens when the model is slow, returns something that will not parse, or is confidently wrong. Those are not edge cases, they are Tuesday.
---
Stage 5 — the thing you rely on
It runs whether or not you are watching.
The two questions that decide whether you are actually here.
How do you know it is working? Not "did it run" — did it produce the right answer. You need something that checks: a test, a rule, an arithmetic cross-check, a sample you review weekly. Without it, a tool that has quietly started producing rubbish looks exactly like one that is working.
How do you know a change made it better? You will edit the prompt. Everybody does. Unless you have a handful of saved cases you re-run afterwards, you are steering by whether the last answer happened to look nice — and that is how people end up fiddling for months with something that has not improved since week one.
That is the whole of what "evaluation" means and it is unglamorous, which is why it is the thing that separates a demo from something you use.
---
The rules that keep this safe
- Reversible before irreversible. Every time.
- A person on anything that leaves the building. Sending, publishing, paying.
- Never put secrets in a prompt. Keys, credentials, other people's personal data. A prompt goes somewhere.
- Know what your tool does with your input. Trains on it, retains it, or neither. The terms say, and they differ enormously between the free and paid tiers of the same product.
- Have a way of knowing it is still right. Anything running unattended needs it, and the day you need it is the day you did not build it.
---
Where to stop
You do not have to reach stage 5. Most people get more value from a dozen good saved prompts than from one fragile automation, and stage 1 costs nothing and never breaks.
Go up a rung when the current one feels slow. That is the only reason worth doing it for.
---
Already written up here
Two worked examples, both with what went wrong left in:
- An AI App MVP in Ten Days with FlutterFlow — stage 4 and 5 on a real deadline.
- Enabling the Google Cloud Natural Language API — the unglamorous half of "just call an API".
---
That is the last track. Back to all the tracks, or start the foundations at What AI actually is.