I Tested Pangram to See If It Could Actually Detect AI-Generated Content As It Claims.

We know that humans can struggle to recognize AI-generated content, but can detection software do a better job? Pangram is a leader in this field, promising “nearly zero false positives” by analyzing “subtle nuances” in text, based on a learning and feedback model that compares human-generated content with its AI equivalent.
The debate over the validity of Pangram’s claims has both pros and cons. After Pangram detected a short story that was later deemed AI-free , the detection system was accused of bias against texts written by minorities.
Pangram’s supporters point to an ongoing reward offer —currently $3,950—available to anyone who finds an article written before 2019 (and before the completion of an LLM) that Pangram incorrectly labels as being written by artificial intelligence. The reward remains unclaimed.
Since the AI-powered detector recently added image and video recognition capabilities, I thought now was the perfect time to test Pangram’s analytics capabilities for myself. Here’s how it performed on three different types of content.
Is Pangram really capable of recognizing text written by artificial intelligence?
From a human perspective, I chose my last 10 Lifehacker articles that are known to be human-written because I wrote them personally—the traditional way, worrying about word choices, going back and editing sentences, and accidentally making a lot of typos that, fortunately, the editors fixed.
To create AI-powered content, I worked with ChatGPT, Gemini, and Claude, assigning graduate students to write 10 articles with identical headlines and roughly the same word count for a tech-savvy audience. I even asked the AI applications to vary the language used so the results would “sound less like something written by a graduate student.”
Pangram’s AI-powered text recognition feature allows you to upload files, but to make things a little more challenging, I copied and pasted the entire text into an online article so it couldn’t rely on the file’s metadata. The scan always produced the correct result: human-written text was treated as 100% human-written, and AI-written text was treated as 100% AI-written.
Moreover, Pangram was confident in its results. All human-written texts were labeled as highly likely to be human, while the vast majority of AI-written texts were labeled as highly likely to be written by law students. Only a few phrases or sentences among the thousands of AI-written words received a “medium” rating rather than a “high” rating.
Of course, there were a few exceptions where Pangram got it wrong —for example, a poem that repeated the same six words thousands of times—but in this case, it’s perfectly fine. Clearly, my writing style is too idiosyncratic and awkward to be confused with the sleek, polished prose generated by artificial intelligence.
To be fair, I have another advantage over AI: I can test devices and apps in a way that LLM bots can’t. ChatGPT can’t experiment with One UI 9 (at least not yet), although it can collect data from the internet about various experiments and tests conducted by real people.
However, I managed to trick Pangram to some extent by alternating AI-written paragraphs with human-written ones. The text was marked as AI-written, but the percentage score was usually lower (20-25%) than the real one (around 50%). If you feed the AI some text but not all of it, some of it may slip through as genuine.
Is Pangram really capable of recognizing AI-generated images and videos?
Image and video recognition is a new feature of Pangram, so I scoured my phone’s photo and video library for content to test the AI. Again, I used copy and paste to upload files to Pangram, rather than downloading them, to limit the amount of information the AI detector could glean from the metadata.
Again, all human-generated content was recognized as such with a very high degree of reliability—whether complex crowd shots or simpler images of natural scenes. Presumably, there are even more cues here than with text, including camera movement, color, style, consistency, and composition.
Next, I generated several images and videos in Google Gemini and ChatGPT, requesting results that looked as photorealistic and “not AI-generated” as possible. The resulting files included people, animals, and buildings. As before, I used copy and paste and a few screenshots to try to limit Pangram’s access to metadata.
Here, too, the AI recognition was impeccable: all artificial content was identified as such with a high degree of confidence. This is perhaps unsurprising: I think AI-generated images, and especially AI-generated videos, still have more “clues” than AI-generated text, especially if you’ve seen a lot of it, so I didn’t expect Pangram to fail here.
I tried experimenting with real photos with AI added, given how well tools like Gemini and ChatGPT now handle image editing , as well as creating photos from scratch. Pangram labeled them as “undefined,” though to mitigate the situation somewhat, only a small number of images were altered.
As with text, the best way to evade Pangram’s detection systems appears to be to inject AI into human-created content. It likely won’t pass the “100% human factor” test, but my tests have shown that Pangram is less confident in such cases or underestimates the actual amount of AI.
Increase the level of AI, and detection becomes more reliable, but if you only use AI for a small portion of your creations, using AI tools is less practical anyway. As for these tests, Pangram’s accuracy claims are borne out, making it a useful tool if you need to verify the absence of AI in a product.