As AI-generated text continues to evolve, distinguishing it from human-authored content has become increasingly difficult. This study examined whether non-expert readers could reliably differentiate between AI-generated poems and those written by well-known human poets. We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ2(1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ2(2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
I’ve noticed in recent times
Poetry doesn’t rhyme
And even when it can
It doesn’t scan
It’s shit, it’s true
I blame haiku
This is true art
Poetry doesn’t need to rhyme. Rhyming is a mnemonic device, so a poem can be memorized and performed.
There are many other devices.
Also, nice poem. Did you write it or chatGPT?
I never thought I’d see the day
When someone writes a poem
The first thing that we say to them
Is “Did you use an LLM?” :(
If a poem neither rhymes nor scans,
Sorry for my spite
It’s no longer poetry
It’s someone talking shite
Rhyming has other purposes: creation of additional sonic rhythm and restricting of words usage - for making matter more distinct and interesting (as rules do for any game).