Personally, I think AI is far better at coding than at writing. I have my free Gemini account plugged into vscode via Gemini Code Assist, and I use it all the time. If you haven't tried it, I'd recommend checking it out. There's a learning curve for handling prompting and context effectively, but once you get the feel of it and describe the goal correctly, it's decently effective. No copy/paste. The AI just edits the files directly.

I've also experimented with Cline and replit. They both look great at paid-tier, but free tier doesn't seem to give enough resources to be very productive. I can only imagine what people are doing with the $200 per month subscriptions... From what I understand, Anthropic (Claude) has AIs writing almost all of their code now.

As far as writing, though, yeah lots of hallucinations still. In addition to @chriddi's fictional song list, one of the AIs recently told @cmp2020 that Beethoven had made a statement on CBS (an American broadcasting company) to denounce Napoleon.

OTOH, I use it for identifying the birds in my backyard photography, and it's pretty good at that. If I don't agree, I usually fact-check Gemini with Claude and vice versa. I actually think that ChatGPT is the worst of the major LLMs right now (at free tier), so maybe another LLM might have done better on a song list - or maybe not.

This is also something that worries me about Thoth. What happens if/when Thoth's summary is wildly different from the actual article. So far, the only times I've seen that are in testing where I asked it to evaluate articles that had very few words... if it doesn't have anything to work with, it just makes stuff up. I haven't noticed any flubs for articles that actually had content, though. So hopefully this will not become a big problem.