

The case recently resolved in the plaintiff’s favour. though Google intends to appeal, so it’s up in the air how things go.
And if it’s this easy to poison the AIs, imagine how easy it is for someone with an actual agenda to mislead people in ways that aren’t as fantastical and quickly spotted.
Equally concerning is that these systems are now seeing use in a range of things. There are lawyers who use it to file suits when they shouldn’t be, and a US lawmaker was recently found to be using AI to draft laws. What happens when things like that make it into the models training data, rather than just being pulled in by RAG/web tools? They’d become part of the base knowledge of all the models of that line going forward.
It’s funny when it’s outlandish. The question becomes what happens when it isn’t? Even without an agenda, what happens when it cites an outdated/incorrect source, or assumes that someone making a joke was correct, and ends up drawing from that when filling a lawsuit/drafting a law?
It seems more like a bad editor/insufficient proofreading.
That’s the kind of comment I sometimes make if I rephrase it in the draft, but don’t check to make sure I remove all of the original sentence before posting.