Convenient, intuitive, and increasingly present in many aspects of our lives, generative AI is reshaping the content industry in remarkable ways. When used wisely, it can help create better content and more efficient processes, but overuse in content writing can lead to bland, inaccurate, or even misleading articles.
This is where AI content checkers come in. But are these detectors reliable?
Yes and no. A single scan is ultimately a roll of the dice, with results varying drastically among tools. And each new AI model further reduces accuracy as generative text becomes more human-like. But with larger sets of articles to analyze, the accuracy gets a bit better.
In the future, popular AI detection apps will likely pivot toward fact-checking and ensuring artificially generated content offers tangible value to readers. Until then, current technology does have a role in your workflow as long as you understand its limitations and how it works.
What Are AI Content Checkers, and How Do They Work?
At a basic level, AI content detection tools examine word probability and sentence structure. However, it’s easier to understand what they do by first looking at how AI writes content.
Large language models, such as Open AI’s ChatGPT, Google’s BERT, and Anthropic’s Claude, function similarly to a librarian. When you ask these librarians a question, they synthesize an answer based on all the knowledge available from the library — the AI’s training data.
Of course, it’s a lot more complicated than that, as large language models use parameters to adjust how they use the information. More parameters mean more ways to work with the training data, and ChatGPT-4 has nearly eight times as many parameters as GPT-3, showing just how fast this technology is evolving.
ChatGPT leverages its enormous training data and over a trillion parameters to predict what it expects to come next in a sentence. It answers questions using probability like a highly sophisticated version of your phone’s autocorrect — albeit with fewer mistakes.
However, it never truly understands the information.
AI content detectors also use probability
Like ChatGPT, AI detectors use machine learning and probability, except they attempt to reverse the process that generates content. They look for text with low randomness, predicting the words generative AI will use in any particular sentence.
However, most tools also check for highly uniform sentences and paragraphs — a characteristic called burstiness. Human writing generally has high burstiness.
For example, human writers may even use single-sentence paragraphs for emphasis.
AI, on the other hand, writes more methodically. The text has a predictable flow and a beautiful conformity that makes it structured, easy to read, and well-organized. However, AI text detectors — and even perceptive humans — can spot this lack of burstiness.
If you scan text with a detector, the tool scores it, typically with a percentage rating. It may also highlight specific sections it believes may be artificially generated.
The percentage rating is usually a confidence score. It’s how certain the tool is that AI text is present, not a measure of how much of the content is artificial, so even a high rating of 70% shows a lot of uncertainty.
Review of Top AI Content Detection Tools
So, where should you start if you need to check the credibility of AI detectors? We think by testing them. We compared 100% human content, 100% AI content, and a mix of human and AI content across multiple AI detectors to evaluate how each one reacts — then we took a closer look at five of the most popular tools.
As you can see, no tool was perfect, but some tools performed better than others.
5 Top AI Detection Tools
Winston AI
Winston AI touts itself as the most accurate detection tool, and it certainly seemed sure of itself when we tested it. It had a low tolerance for AI-written text, even when edited. However, it correctly identified human-written text — albeit with a lower confidence level than other tools.
Overall, Winston AI has many valuable features. For example, its project and document storage lets you check previous scans and organize content. Uniquely, the tool utilizes OCR technology, which lets you check whether text in images is AI-written.
Pros:
- Image scanning
- Downloadable PDF reports
- Team management functions
- API access for integration with other marketing tools
Cons:
- Limited free access
- False positives due to the strict algorithm
Price: $12 per month for 80,000 words
Originality.ai
Originality.ai is another robust AI detection platform with many features to unpack. Besides the scanning tool, which lets you check whether content might be AI-written, you get plagiarism checks, a readability analysis, and fact-checking.
Compared to other tools, Originality has a low tolerance for AI. It rated each test article, including human-written articles, as AI-written with 100% confidence. The tool seems to place additional weight on burstiness, leading to false positives from highly uniform text.
Pros:
- AI, plagiarism, fact-check, and readability scans in one tool
- Team management features to help scale content production
- API for bulk scans
Cons:
- No free functionality
Price: $14.95 per month or $30 pay-as-you-go
GPTZero
GPTZero is one of the most straightforward tools we tested — especially if you only need to check a few pieces of content. This is because you don’t need a subscription or account for a basic scan. Simply paste your content into the web interface. GPTZero also has a novel feature that recreates typing behavior in Word or Google docs, which is a nifty way to be certain that a human typed rather than pasted content.
During our scans, GPTZero identified each type of content relatively accurately. The tool was quite confident when it saw full AI content, with predictable decreases in its confidence rating when presented with mixed content. Compared to Winston AI, GPTZero was also less uncertain about the purely human content.
Pros:
- Easy to use
- Chrome extension available
- Writing reports to certify human writing
- API access
Cons:
- No readability scan
- Account required for more advanced features
- Character count for individual scans limited on free version
Price: Free (50,000 characters/10,000 words per month scan limit)
Sapling
Sapling works differently from other tools on this list. While you can perform a content scan through its web app, the tool also integrates directly into browsers and Google Docs.
Another unique feature of Sapling is its spelling and grammar check, which works similarly to Grammarly. You also get AI-powered writing assistance, recommending ways to complete and enhance content.
When we tested Sapling, it didn’t differentiate well between AI and partially human content. However, it did identify the fully human content.
Pros:
- AI assistant and grammar check included
- Multiple integration options
- User-friendly interface
- API for batch processing
Cons:
- Low non-English accuracy
- Limited free AI detection
Price: $25 per month
Crossplag
Crossplag is primarily a plagiarism detection tool, but the platform also offers free AI detection through a web app. However, you need to sign up for an account to use it.
When we tested Crossplag, the tool correctly identified pure AI content, but it also gave the mixed-origin text a 100% confidence score. Crossplag shows this confidence level as a handy color-coded scale on the dashboard but doesn’t highlight individual sentences like GPTZero and Winston AI do, so there’s no way to know how much potential AI the mixed article had.
Pros:
- AI and plagiarism detection in one tool
- Free AI detection
Cons:
- Limited features
- No detailed scan stats
Price: Free AI detection but $9.95 for a plagiarism check of 5,000 words.
Real-World Application and Case Studies
Given the discrepancies among AI content detection tools, adoption has been somewhat inconsistent. After all, when one app says AI wrote an article but another says the opposite, how can you base decisions on the results?
Universities have had to ask the same question. While many adopted Turnitin’s AI detector early to address academic cheating, this overzealous uptake led to false-positive scans. That’s why prominent institutions such as Vanderbilt and Michigan State inevitably turned the technology off.
Surfer conducted a case study using Originality.ai that also revealed insight into the accuracy of AI content detection. The company ran 100 human and AI-written articles through the tool. Results showed:
- The detector was only 50% confident it found generative text across 78% of AI-written content.
- About 10% of human content received a confidence score lower than 50%.
- Only 28% of the human-written articles received confidence scores of 90% or higher.
What does this tell you about the capabilities and accuracy of AI detection tools? Unfortunately, they’re not reliable at all.
It’s best to take what content checkers tell you with a grain of salt. Use them, absolutely, but only as part of a broader content audit to confirm existing suspicions. Also, don’t just use one tool; use several and cross-reference the scan results.
If content flags across most tools, a false positive is less likely. However, you’ll never completely eliminate false positives.
Alternative Evaluation Methods: Identifying AI Content Without Tools
While AI detection apps can be helpful, it shouldn’t be your only method of determining the authenticity of human-written content. If you review content regularly, it’s a good idea to become familiar with AI-generated content. Read enough of it, and you’ll start to see that even AI has habits.
There are several things to look out for.
- Lack of depth: Granted, writers can’t cover everything about a specific topic. However, unedited AI content skims the surface of topics. More importantly, it rarely shows the firsthand experience or expertise that E-E-A-T compliant content has.
- Unusual phrasing: AI uses specific buzzwords like “meticulous” and too often tacks “-ing” phrases onto the end of sentences that don’t need them, looking a lot like the phrase you’re reading right now. It also sometimes uses phrases that sound odd, such as advising you to delve into the world of toothpaste flavors or embark on a journey to discover dishwasher settings. These phrases lend too much grandeur to mundane topics — something a human would typically avoid.
- Repetition: A talented human writer will avoid repeating ideas unless necessary, but AI often repeats itself in a single article.
- Overly clean structure: AI adheres to a highly predictable flow and uniform sentences. It lacks spontaneity and reads more like academic text structure-wise. Of course, sometimes a project calls for this style from humans, so it’s not definite.
- Way too much voice: If instructed to inject some form of personality into the writing, it often goes over the top, dumping the full salt shaker of voice when just a sprinkle was needed.
When you audit content to confirm suspicions of AI use, look at multiple articles, including those written pre-2023, when generative AI was more primitive. Do you see any dramatic changes in a writer’s style or grammar? Human writers work hard at their craft, and false positives are a common occurrence. A reasonably confident decision requires ample data.
The Limitations of AI Detection Tools
AI detectors play a role in content production but have limitations. Their accuracy is fairly high when detecting verbatim use of AI text. However, the number of false positives from human text should cause you to pause before relying on them.
There’s also a widespread misconception that if human text scans as AI through detectors, they’re probably not valuable. However, this ignores the tools’ emphasis on sentence variation and Google’s overwhelming desire for content matching the E-E-A-T guidelines.
We cover this misbelief — and several others — within our webinar on the myths and realities of AI detection. Watch the webinar, and you’ll discover key insights into how AI detection technology works and how generative AI will continue to evolve in the future.
We also take a deep dive into how we audit content, what we look for during a manual review, and how to address client concerns over AI usage.
