Take it apart

Your paper, with its data. The agent finds the biggest problem and tries to fix it, while we do something else.

It worked for me

  • I did this with a recent paper in my own field.
  • It found a real problem, then a new estimator: valid under weaker assumptions, and better than the one in the paper.
  • My only input was pushing.
  • At Strategy in October, this was the exercise that landed. The catch was waiting for it. So today, we don’t wait.

Where your time goes

The usual way Reading the paper and the literature: weeks judging With the agent judging: the problem, the fix, the field it reads: 1 min · it tries a fix: 10 min

You know the field. The agent does the reading; you do the judging.

Before you leave it alone

It will want to run code. If it stops to ask, it waits for you while you’re away.

  • Switch the mode under the prompt box to Accept edits (it still asks before other commands) or Auto (it asks only when something looks risky).
  • It works in this folder, and you have a save point. Anything it does, you can undo.
  • Asked to install Python or R? Say yes before you go.

Start it

Take this paper apart, like a tough methods referee who also has the data. Run the analysis yourself and check that the numbers in the draft match the code. If there is a preregistration, compare the paper with it. Find the single biggest problem, then propose a fix and show what changes. Write everything to take-apart.md. Work on your own; I’ll be back in 20 minutes.

Then leave it. Next: It wants to please you.

When you’re back

Read take-apart.md. Then push:

  • “Show me where in the paper, and where in the code.”
  • “Rerun it the way the preregistration says.”
  • “Is that really the biggest problem?”
  • “Show that the fix works.”

Interviews, not data? “Which claims rest on the fewest interviews?” “What else would explain the same quotes?”

Round-up

  • Best problem found?
  • Did it find something good for the paper?
  • Worst false alarm?
  • Keep the folder. The referees get the paper next, but not the folder.

Next: the blind referee panel.