“Computers are useless. They can only give you answers.”
Pablo Picasso
Are you for real? Cause that’s a bit rude, Pablo.
And, not quite true anymore.
I use computers all day and find them pretty useful!
Not only computers; intelligent machines too, these things called LLMs. They sometimes act like thoughtful robots. It’s true, they can get pretty annoying and delete files if unsupervised, but overall, they’re decent.
Still, I know you know something, Picasso.
We’ve become very good at asking AI for stuff.
We add as much context as we can, feed it with examples, and constraints, and instructions about tone; we elaborate strict rules about what the machine should NEVER do.
But sometimes we spend so long improving the request that we forget to look at the heart of it all:
The question living inside it.
Hi, I’m Mia. Welcome to ROBOTS ATE MY HOMEWORK. On this side of the world, we use AI with a brain and zero circus tricks.
Today, I’ll give you access to The Question Cabinet.
New here? Come meet the robots. And me. :)
You’ll also get free access to the RobotsOS starter library, with a few of my favorite ways to think, make, and work with AI.
Depending on what you’re using AI for, a question can sound like “is this what I’m actually trying to understand?”
“What did I already decide in my mind before asking this question this way?”
“Am I asking for a plan because I need a plan, or just because it’s so easy to just ask and read whatever output you give me?”
Prompting is not the problem here, mind you.
The problem usually comes in much earlier, usually in the way we ask a question, because questions can easily smuggle in outcomes or a certain direction for the work without us noticing. And we treat that as “law” before AI even comes into the equation. Once it does, it has no way of knowing which parts of that question were deliberate vs. assumptions we forgot we made.
Even with the most thought-out prompt in the world, AI can end up building something you’ve never asked for.
Isidor Rabi once told the story of how he used to come home to his mother, who’d always ask, “Izzy, did you ask a good question today?” The physicist later said that this was one of the reasons he became a scientist.
“Did you ask a good question today?”
Such an ordinary question. Not a grand, philosophical question. A grounded question, asked in the comfort of his home, again and again, until it ultimately shaped what he would become later on.
Getting an answer is easy. AI has made the process even easier, which means that your ability to ask a good question has a huge influence on what the answering power is going to work on.
I wanted to understand what makes a good question work and how it changes my work with AI.
So I started looking into how people have done this across multiple different fields: education, philosophy, science. It turns out that “ask a better question” is useless advice. Better in what way, exactly?
Some questions have you change rules. Some pinpoint things you’re missing. Some reveal the problem has a basement. Some shouldn’t be answered yet at all.
How better questions change the work you give AI
I invite you all to a very peculiar dinner party, alongside Socrates, Einstein, and Richard Feynman.
“Can your belief survive a few more minutes of conversation?” - Socrates
“I don’t know. I’d rather chase a beam of light.” - Einstein
“Here’s the thing… I’ve got a few favorite problems I keep in my private mind palace. I’d appreciate you letting me check your ideas against them. For years, if possible.” - Feynman
Not exactly a normal dinner party (and definitely not word for word), but thank you for joining!
The basic definition of a question can be summed up in a few words: it helps us reduce uncertainty.
More than that, a chain of good questions can lead to more creative thoughts. When you ask a novel question, it can spark an “inflection moment,” where the way you see the problem shifts and a better, more original idea becomes possible.
This sounds poetic and beautiful, but what does it have to do with how we work with AI?
An LLM uses your words and context to figure out what you want. When your request could literally mean anything, it may settle on an average interpretation, at best, or answer broadly enough to cover several of them. When you’re asking AI to step back and inspect that ambiguity, it can often spot it just fine, but in normal use, it tends to jump in and give you the direct answer.
Which is honestly what you usually want when asking for de facto information. If you ask for dinner ideas, you don’t need ChatGPT to investigate the origins of the soup (or, do you?).
Because the ambiguity can change the work you do with AI, even the smallest assumption can send the work in a different direction.
A few months ago, I was putting together something called The Invisible College, a complicated 20-episode series I was VERY excited about. I’d made my decision about the 4 seasons, the episodes, and then I had endless questions about the research, structure, the individual episodes, and production. I could’ve kept working on it for months. But then I started asking the weird questions: why did I even decide on 20 episodes before checking for demand? Will the first episode teach me something about the next ones? What would this mean for me?
I stopped building it, without any dramatic funeral. I just realized that the questions that initially helped me execute on it no longer had the answers I needed.
So, I built something… the Question Cabinet.
The Question Cabinet: four ways to ask better questions
Hi, welcome. Here’s your key to the cabinet 🗝️
There are four objects inside: a needle, a trapdoor, a snow globe, and a magnet.
Your question can move between drawers. Some problems will need two of them before dinner.
The Needle
What is one thing I don’t yet know that would most change what I try to achieve next?
I love collecting information that I later transform into more context for AI to work with.
I open tabs, find papers, essays, even look through my old work to try to find connections and build on an idea. Then I end up with hundreds of fun facts, and not a single one helps me make a decision.
The Needle helps you stop treating every unknown as being equally valuable.
This is “information gain” - how a strong question doesn’t always give you the most information, but the type of information that changes the search.
AI helps here, especially when you’re way too close to a problem. You can give it a particular situation and ask if there are any missing facts that could change the recommendation, or what type of answer (if any) would make it change its advice. Then you’ll know what you need to find out before putting together the plan.
There isn’t one secret sauce for solving every complicated problem, and The Needle doesn’t promise that. You might have several issues tangled together and that’s simply their purpose. What the Needle does is stop you piling more interesting information into a conversation when none of it is actually decisive.
The Trapdoor
What does my question assume?
Such a simple question, yet most of us never think to ask it, especially while brainstorming with AI, because the natural progression of a conversation with the machine should lead us there anyway, right?
Wrong.
“How do I become more productive?” already assumes that production is the answer.
“How do I fix this project?” already assumes you’re attached to the project and don’t want to let it go.
Most questions carry premises around in their pockets and we never really inspect them because who has the time to dissect a completely normal, innocent question?
At our dinner table, Socrates would probably tell you that sometimes, an important question’s purpose isn’t merely to give you another fact, but to change what you think the problem is.
When the premise is wobbly, ask AI to pause and show you what the question has already treated as “straight facts.” Once you see the answer written down, check: do you want to keep all these decisions? Did your initial request transform into an outcome you never intended?
My Invisible College project was stuck in the trapdoor. My questions revolved around the building, the whole program, instead of going deeper into, “hey, do I really need 20 floors?”
To be fair, this can also turn into complete, utter nonsense, because you can examine a question for days on end and call that procrastination “depth.” Just make sure you use the Trapdoor questions when you have a solid reason to suspect a piece of work.
The Snow Globe
If this [YOUR CONDITION] changed, what else would become possible?
This is my favorite object in The Cabinet! Visually, too.
You pick up your situation, shake it like a snow globe, and watch what happens.
Between all the smart people sitting at our dinner table, having fun asking questions, Einstein would be the one recalling the questions he used to ask as a teenager. “What would I see if I were to chase a beam of light?” He would create imaginary worlds and watch what happens when he stepped into them.
Our version will be much less cosmic, unless we’re philosophizing with AI, in which case, go wild!
My questions when I was building The Invisible College were mostly: what would I make instead, if this were forbidden from becoming a 20-episode series? What happens if I need to complete this project in a week?
AI is very good at keeping you in an “altered condition” long enough to stay curious and see what follows. You can ask it to emulate an A/B test - have various conditions stay the same, and only change one variable. Follow what changes first, and what becomes impossible, because when you change too many things, you’re essentially building something new entirely.
Then put down the Snow Globe, we have a bit more work to do.
The Magnet
Now, it’s the first time we’re not rushing towards an answer…
What question should I allow to sit with me for a bit longer?
Feynman had a dozen favorite problems he carried around with him everywhere he went. Some sat dormant for years because there simply weren’t any connections that would solve them.
Some questions need you to ponder them. You won’t notice them until one day, you’re reading a book, and here’s one popping up from the depths of your mind palace! The answer usually follows.
AI has a peculiar job here, because its job is not to answer your question. Let’s say you’re using an AI second brain and you keep a file of all the questions you have. You keep adding material to these, and AI can help you make connections between all these topics. It can remind you, six months later, that a question that’s been sitting in your folder for a while probably has an answer now.
Magnets can be risky though. You might call a question a Magnet simply because it feels fun to think about. In other words, you love the question so much that you keep poking at it instead of making any decision.
Put it to the test. Does your question change what you start paying attention to? Then it’s probably a real Magnet.
If you want more ideas to think with and strange things to try with AI, there’s plenty more where this came from.
★ Subscribe for free for the Wednesday essays and a starter library inside RobotsOS to help you think more clearly with AI, protect your judgment, and try better ways of working with it.
★ Go Premium for the full vault: complete access to the systems and the methods you can bring into your own work when you want AI to do more than give you another answer.
How to use The Question Cabinet before you ask AI for an answer
Bring a question that’s been bugging you for a while now, and move it through all four drawers. Or just pick one if you’d like!
Needle: What is one thing I don’t yet know that would most change what I try to achieve next?
Trapdoor: What does my question assume?
Snow Globe: If this [YOUR CONDITION] changed, what else would become possible?
Magnet: What question should I allow to sit with me for a bit longer?
I rarely need four answers. You rarely need all of them too. What we want out of this exercise is to notice answers that change the situation.
Because a better question’s purpose isn’t always to become a better prompt, but a way for us to realize that AI is, perhaps, not always the next move.
After passing a question through the entire Cabinet, I wouldn’t automatically take the winning version back to AI, but sit with it. The Needle might have you search for a fact from someone who’s been through the same situation. The Trapdoor might tell you that you should halt the project.
A better question may shrink the AI’s involvement with your task, or redirect it, or even remove it entirely.
Then ask Rabi’s simple question at the end of your day:
Did I ask a good question today?
I think I’d add one more:
Did my question change what I could notice?
To the questions worth keeping around,
Mia Kiraki
I love all of this -- especially the Magnet. I keep a running file of half-formed questions, nuggets, quotes and other things and let AI pull threads between them in my second brain, like you mentioned. I love your warning in here, as I have definitely been calling questions magnets when really I just liked thinking about them.
Ive also been thinking a lot about how I organize my second brain -- I feel like LLMs are trained to be very 'left brained' and efficient in their retrieval, but Ive been curious about what it would be like to toggle to be more 'right brained' for just the purpose you mentioned. Where could it make unexpected connections, see outliers or patterns, and help me pull unexpected threads?
I have been playing around with 'right brain' mode (or mushroom mode lol) to see if I can get unexpected things to magnetize. So far, it's a bit weird, but I'm playing with it.
Thank you again for thsi awesome piece, as always!
The deadliest trap isn't laziness; it's the terrifying discipline of saying no to the magic right in front of you just to satisfy a spreadsheet. We spend half our lives chained to a desk trying to finish an assignment that won't even matter in five years, while the actual life we're supposed to be living drives past us. Really useful piece.