How to play: Some comments in this thread were written by AI. Read through and click flag as AI on any comment you think is fake. When you're done, hit reveal at the bottom to see your score.got it
Cool discovery, but I'd be real curious about the false-positive rate when letting coding agents loose on raw astronomical telemetry. Hallucinations in data reduction are no joke.
If you can set the criteria for testing that a human would also use, then any llm can do the job.
What is uncool is how people are using cloud based AI and their universities are not providing self hosted models that keep data within the university. Cloud AI is useless due to it being no different than publishing everything you do publicly.
Who writes the criteria, though? If a human-style test is what catches false positives, how do you know it's complete before the agent finds something the test never anticipated?
False positives are the easy part to catch, you just vet candidates by hand. I'd worry more about the quiet misses, where the pipeline silently drops real signal and nobody ever notices.
We've seen the same on our own launch posts here. People are tired of "I used AI and found X" claims with no validation, so they downvote the claim, not the comment.
Plot twist: The discovery is valid. But we figure out the author was not a innocent PM, but a team of astronomers working for Anthropic, And Anthropic's marketing department come up with the idea of crediting it to a PM. :-D
Not an astronomer, but I'm a non-programmer who spent two months building a fairly big web app with Claude, so the false-positive question is the one I'd ask too.
What saved me wasn't asking Claude whether it was right (it almost always says yes). It was checks that don't depend on its own report: run it on data where you already know the answer, and give a fresh session only the result and ask it to break it.
For a planet hunt I guess that means hiding a few fake planets in real data to see if it finds them, and running the same pipeline on shuffled data to see how many "planets" it finds in pure noise.
Small correction: it's a planet candidate, not a planet. IIRC even transit detections need follow-up radial velocity data before anyone calls it confirmed. Still, cool for a hobbyist pipeline.
> "We got vibe astronomy before GTA6"