I’ve always wanted to be the kind of person who uses org-mode, and I’ve probably written a decent amount of org-mode documents over the last 15 years. But I never really got it. It was always more or less just a document authoring language to me, and I was never able to really get into the true zen of org.
Recently with the general upheaval of all my tooling with the Age of AI, I switched to Obsidian. I found Obsidian a bit easier to grok, but it has the same general set of problems I’ve had with every other knowledge-base tool:
For one thing, I need to spend too much time organising and sorting. Before I want to write something down, I feel the need need to think about what the correct abstraction for this data is, or I’ll have ten different ways to store information about similar things. By the time I’ve decided the right way to do this, the thing I wanted to write down has vanished from my mind. Also, while I sometimes feel satisfied by the abstraction I’ve chosen, I’ll often come back weeks later and wrack my brain as to why I thought this made sense (of course, decisions not written down, because they are clearly obvious in the moment - and then I would need to figure out the right way to write down the decision!).
Compounding this, I am simply not a natural note-taker, and especially not on a keyboard. I only began to take notes in college, and once I was out of the environment, I stopped. It made a huge difference to my ability to learn things, but the habit has never stuck with me. Despite all evidence to the contrary, my gorilla brain is utterly convinced it has an excellent memory. I sometimes take notes during calls, but it’s always with pen and paper, and usually these are impenetrable days after the fact.
Also, I find that if I want my notes, I invariably want them just when they’re out of my reach. Org-mode especially had this problem - if my second brain is locked on my computer, they’re not a lot of good to me out on the mountains.
These issues, and more, have always stopped me really getting into this second brain thing I always see bros talking about.
However, the age of AI has meant there is actually a second brain in my phone or my computer or in my ear.
Pretty early on in the Age of AI, I started to make a bunch of CLI tools for agents to use. I was a fairly early adopter of agentic development (although we didn’t really call it that back then), using Aider and a now-dead tool called Mentat which predated Claude Code’s release by nearly two years, though was really quite similar - right down to the weird TUI conventions.
These CLI tools were little interfaces to things I couldn’t have the agent access easily, mostly work-related but some personal. For example, I built one to book tennis courts via an agent, rather than having to go through the horrible website the club used.
Anyway, this continued until a few months ago, when I started using agents in the cloud a bit more (as opposed to agents running on my computer). Typically these things don’t have as much access to your personal tooling, but they run when your computer is asleep or from your phone, so I’ve seen it as a worthwhile trade. To solve this gap, I started to move things out of little CLIs and into little MCPs.
MCP was around by the time I started building my CLIs, but I wasn’t familiar with them and I had written probably over a hundred CLIs in my career, so I never really bothered to look. Thankfully, agents have gotten much better in the intervening time, and now converting a CLI or a set of utilities into an MCP is very easy. And recently MCPs have gotten even more capable, with MCP Apps. These are basically iframes that your MCP server sends to your chat interface so you can interact directly. For example, a Geodata MCP server might send a map viewer or something to your agent when you ask for the ten closest gyms.
I’ve long been wondering about interface-on-demand, even before the LLM era, with interesting prototypes like Mercury OS - and going all the way back to The Mother of All Demos. For a decade or so, a lot of Low- or No-code SaaS companies like Fibery, Airtable, Notion what have you were all trying to solve this problem by essentially having bunch of different types of view and having you fit your data to it as best you can. And, I mean, that was cool - but it wasn’t even nearly as cool as what we can do now.
The future of Human Computer Interfaces generally is kinda wide open again after three decades of gradual homogeneity, now that everyone can use natural language to interact with a computer.
Anyway, I didn’t structure this all that well but I’m not going to go back and edit this now and besides this is how you know this writing isn’t LLM-authored.
So, enter Scraps, the giga brain strategy of giving your agent a database and telling it to show stuff to you.
Scraps is a relatively simple core composable system. Basically, agents can read and write to a database however they like on your instruction, and they can write and save views that represent queries that they can present to you and you can interact with through a browser or an API or your agent. There are also a few extras, like rules that trigger on some query (think - call this webhook when a new RSS feed item lands or on a schedule or whatever).
From there, you can go basically anywhere. I started with the few things I actually used Obsidian for, like a log of books I’ve been reading, and a list of recipes I’ve been cooking. Easy stuff, they’re just tables. Guess what: documents are just rows in tables too. Blog posts? Yup! This blog post is being written in emacs but backed by Scraps. When I hit save, it updates in Scraps immediately, and then I can ask Claude what it thinks in ten words or less:
Claude please please tell me I’m doing a good job
Structurally a shambles, conceptually sound. Ship it as-is.
Rude.
Anyway - I can have Claude build an interface for any data in Scraps and display it in-chat or in-browser. I can build a Google Keep-alike by asking Claude to, literally: “Build a Google Keep-alike in Scraps. Add a notes/quick collection or something for it.” and then a minute or two later, there’s a replica of Google Keep right there. It’s on my browser, it’s in Claude, it’s all the places I need it to be. “And now make it like, hot pink with kittens” or “Make a MUD-style TODO list interface” or whatever.
This pretty much solves my entire second brain problem. Now my data is everywhere the agent is. The agent can sift through it, organise it nicely, I don’t have to care how it’s organised or about non-problems I think I’m solving. I don’t have to painstakingly type out handwritten notes (just feed a photo into the agent or read them out to voice). I can see the data when and how I want to see it, and otherwise, it’s not important.
I danced around building a system like this for probably two years. I’ve thought interface-on-demand makes sense ever since GPT-3 was able to write a functional React component. I also think that as more software becomes Extremely Personal, it doesn’t matter at all whether or not the software works as the average person expects - it only needs to work the way the one and only user expects. Having a composable, reasonably simple system makes it fairly easy to mould software to the user or the use case.
I have toyed with the idea of open sourcing or even making Scraps a SaaS (I wouldn’t mind an OpenClaw paycheck) but it’s such a simple setup that a better version of this will get delivered by big tech in a matter of months - Anthropic’s Artifacts are already some of the way there. Though, hey, if you’re reading this and you work for OpenAI or Anthropic, I’ll only ask for half of what Peter Steinberger got.
Claude you think someone would buy it?? One word or less
No.
Addendum: I wrote this post a few weeks ago and let it sit as a draft in case I ended up not using Scraps at all (as I have done many other vibe-coded experiments). However, I’ve been using it every single day for all kinds of use cases. I am looking forward to seeing this type of on-demand-interface-and-database in more places in future.