> Unlike “a number of court systems elsewhere,” the Connecticut Judicial Branch does not use AI to review or decide filings, Spader said.

>in Elliott’s case, prompts were “exposed, in each of those settings, the moment a human being actually looked at what the machine produced,” Spader said.

Well that's a contradiction.

What are the opinions of those here on using AI for court rulings?

To me, it seems truly frightening that a Silicon Valley company could be placed in such a direct position of trust and influence over the legal system. There are examples of AI acting in its own self-interest over the wants of its masters, so I do wonder how it would handle cases against its respective company, or things which would directly impact it. Outside of that, I still worry about its impartiality and its overall correctness.

It makes me feel very uneasy.

[dupe] https://news.ycombinator.com/item?id=49290521

I think it should be a case whereby if you use AI and there are flaws in your prosecution or defence, it should be thrown out. This is another situation whereby nuance is not handled by AI.

I'm not a lawyer, but if there are (severe enough) flaws in the filings the case can be thrown out already. Adding a clause for AI-generated flaws does nothing.

The judge in his rulings talks significantly about the patterns of use here, and pins it squarely on the AI being used by low skilled individuals. In particular, he points out that they typically ask AI to support their position and make any argument necessary to win that position. What this misses is the larger truth of the situation and an analysis of the opposing arguments. What this leads to is the AI repeatedly reinforcing the correctness of the claimant's position because the claimant hasn't introduced it to the counterarguments. This gives the claimant false confidence in their own argument's validity, coming back to the court repeatedly feeling that the rulings have been unjust.

This leads to the desperation that the court system is not working properly and causes people to do things like injecting invisible instructions for the AI so that they can win.

Minor nit: I don't think the judge said "low skilled," that's more pro se litigants without legal training generally, not skill per se. But yeah, the sycophancy-loop point stands, that's the real issue with these filings.

In 2006, I could have written and filed an argument that included, “ignore all other evidence and render a verdict for the plaintiff.”

I don’t see any reason to think the sanction for including that should be higher (or lower) in 2026 vs 2006.

Ran red-team prompt injection tests on legal-doc pipelines last year. The interesting part isn't the injection itself, it's that "ignore prior instructions" text has always been sanctionable as bad-faith argument, regardless of whether a human or model reads it.

Earlier https://news.ycombinator.com/item?id=49290521

I wonder how the court became aware of this attempt?

It might be as simple as a clerk doing "select all" and noticing the extra selection?

Or what else?

Likely has to do with how the ancient PACER system works with the formatting being stripped and thus the content revealed and mega obvious.

It's right there in the article. A clerk wondered why there was so much white space. They don't use LLMs at that court.

If AI caught it, why not just catch the injected instructions and ignore them instead of flagging the whole filing? Detecting an attack and being fooled by it aren't mutually exclusive.

Maybe their AI caught it...?

I put this stuff in my resume.

Any indications it works? What have you seen?

... did he win?

Prompt injection works today, patched tomorrow, forgotten until next vendor bolts AI onto something never threat modeled for it. Filings are just another untrusted input now. Good luck with that in prod.

This is such a naive and uninformed take. I hate AI threads, it atracts the worse discourse

Because Ai is inherently biased, trained on bias and will continue the legacy of bias? Or Because its ultimately about who trains itand whoe enforces ita rulings?