Pangram 4 review: what the 1-in-24,000 claim really means

What Pangram 4 actually is

Pangram Labs makes one thing: a classifier that reads a document and tells you which parts of it were probably written by a language model. Pangram 4 is the fourth generation, and the launch post calls it "our most powerful and accurate AI detector to date."

The company shipped it alongside a $9M round led by Menlo Ventures, announced the same day. It was co-founded by Max Spero, who is CEO, and Bradley Emi, the CTO. They met in a Stanford freshman dorm. Spero's X bio reads "slop janitor @pangram," which tells you roughly how the company sees its own job.

The product surface is wider than the name suggests. There is the web app above, a browser extension, a Google Docs integration, a plagiarism checker bundled into paid plans, an API, and as of the same launch a research-preview image detector claiming 99.8% accuracy. Named customers on the homepage include Substack, Quora, Google Classroom, and Wiki Education. The Substack rollout matters later, because it is why most of 2026's complaints come from newsletter writers rather than students.

If you are new to this category, the short version of the mechanism is in my guide to AI content detectors. If you are comparing tools rather than reading a single review, I keep a running list of detection tools too.

Two others come up constantly in the same conversations. Writer's content detector is the free one people try first, and my notes on picking a detection tool cover what to look for before you pay for any of them.

How Pangram 4 reads a document

This is the part I found genuinely interesting, because Pangram 4 is not one classifier with one output. The model card describes a sparse mixture-of-experts backbone with four separate heads bolted on: a 15-class head estimating overall AI involvement in a window, a three-class head labelling every single token as Human, AI-Assisted, or AI-Generated, a binary head asking whether the segment is mixed, and a four-class humanizer head.

Long documents get chopped into overlapping 512-token windows. Every window gets scored, a conditional random field stitches the token-level calls into one label sequence across the whole document, and a postprocessing step snaps the result to sentence boundaries with a minimum contiguous run of about two sentences. That is a real upgrade over Pangram 3, which chunked text during preprocessing and produced coarser blocks.

It also explains a behaviour people report constantly and read as a bug. Because scoring is windowed and then smoothed, the same paragraph can come back a different colour depending on what surrounds it. Pangram lists this in its own limitations: "Occasionally, a subset of a longer text will receive different predictions depending on whether it is isolated or embedded in the surrounding document." That is candid, and it is also the mechanism behind half the confused forum posts about Pangram.

Two API details are worth knowing before you build anything on this, because they are easy to misread. The ai_assistance_score is computed as P(AI-Generated) + 0.5 × P(AI-Assisted), and the model card says plainly it is "not the probability of the displayed discrete label." The confidence field returns High, Medium, or Low, and the card says it "is not a calibrated probability estimate." So you cannot treat confidence: High as a 95% certainty. If you are wiring detection into a workflow, that distinction is the whole ballgame. My notes on building an AI content pipeline cover where a gate like this actually belongs.

The numbers Pangram publishes

Credit where it is due: Pangram publishes more of its own evaluation detail than any detector I have looked at, including a technical write-up and an arXiv report. Here is the headline set.

The false negative table by generator is the most quotable thing on the model card. Grok 4.3 is the hardest model for Pangram 4 to catch at 0.605%, and Claude Haiku 4.5 is the easiest at 0.130%. Gemini improved the most between versions, from a 5.138% family miss rate on Pangram 3.3.2 down to 0.498%. Pangram also reports no meaningful correlation between a model's release date and how detectable it is, which is a bolder claim than it sounds and cuts against the usual assumption that newer models are harder to spot.

One inconsistency to know about if you cite this: the technical post's key-results block gives the AI-polish false-flag rate as 0.009%, while a section further down the same page gives 0.01%. Small, but it is the kind of thing that makes you check the rest.

Where the headline number stops holding

Now the part I think a fair review has to lead with rather than bury.

Every number above is an English number, measured on long-form prose in complete sentences. Pangram publishes the per-language table too, and the spread is large.

Across 18 languages the multilingual false negative rate is 1.24% against 0.3396% for English. Per language it runs from Czech at 0.2760% up to Urdu at 5.3169%, roughly 16 times the English figure. The false positive side holds up much better, with most languages at 0.0000% and Ukrainian worst at 0.0361%, so the multilingual weakness is misses rather than false accusations. Pangram lists 24 supported languages and does not hide any of this, which I respect. But "one false positive in 24,000" and "we support 24 languages" are two true statements that people will read as one claim, and they are not.

Then there is the scope list, quoted straight from the model card:

"Short conversational replies, answers to questions with a single factual answer, source code, tables of contents, reference sections, templated or automated writing, instructions and technical manuals, and text dominated by mathematical notation are outside the model's primary scope or may be more susceptible to errors."

Pangram 4 model card

Read that again with a real workflow in mind. Support macros are templated writing. Release notes are close to technical manuals. A product FAQ is answers to questions with single factual answers. Anything under 50 words does not get scored at all. Pangram also recommends stripping headers and footers before you scan, and prefers .docx or raw text over PDF because PDF parsing introduces artifacts.

So before you act on any Pangram verdict, it is worth walking the same four checks I do:

What independent researchers actually found

I went looking for someone outside Pangram Labs who had measured this. The answer is more interesting than either "it is verified" or "it is marketing."

Three independent teams have tested Pangram, and none of them measured a higher false positive rate for it than for any other commercial detector. In two of the three it was lowest outright; in the VUB study it tied at 0% with Turnitin and Copyleaks. That is a consistent result across different corpora and different metrics.

• Jabarian and Imas at UChicago Booth (NBER Working Paper 34223) ran 1,992 pre-2020 human passages plus AI counterparts across six genres and four models, and declared no conflicts of interest and no Pangram funding. Pangram scored a 0.001 false positive rate against 0.002 for Originality.ai and 0.007 for GPTZero, and was "the only tool to satisfy a strict cap (FPR ≤ 0.005) without sacrificing accuracy."
• A peer-reviewed study from Vrije Universiteit Brussel in the International Journal for Educational Integrity tested 160 academic papers of at least 4,000 words each. On fully human papers written by graduate students, Pangram returned 0% AI. On fully AI papers, Turnitin produced false negatives 100% of the time and Pangram's median was closest to the truth. Their conclusion: "from the four AI detection tools studied here, at this moment only [Pangram] produced satisfactory results."
• Epoch AI tested Pangram 3.3.2 on 495 human passages from 99 named pre-2022 authors and got 0 false positives.

So the direction is clear. Now the arithmetic, which is the part nobody says out loud.

No independent study is large enough to test a number as small as 0.0041%. Zero false positives in 1,992 passages bounds the rate at roughly 0.15%. Zero in 495 bounds it at roughly 0.6%. Every published independent dataset is one to two orders of magnitude too small, and that is a fact about sample sizes rather than a criticism of anyone's methodology.

And none of them tested Pangram 4. Epoch AI states version 3.3.2 explicitly. The UChicago study ran in mid-2025. Pangram 4 shipped on July 29, 2026, so the model carrying the headline figure has not been benchmarked by anyone outside the company.

There are three more findings a buyer should weigh:

• Epoch AI found Pangram 3.3.2 missed 10% of AI text written in a named author's style overall, and 25% of AI-generated scientific writing under style imitation, rising to 48% on Gemini-generated scientific passages. The false positive story is strong. The false negative story under adversarial prompting is not.
• A CMU-affiliated team (arXiv:2605.19516) showed base-model text reads as human while instruction-tuned text from the same model does not, and pushed individual passages from 0% human to 100% human with two rounds of paraphrasing. Their verdict is worth quoting: "current detectors are tracking artifacts of instruction tuning and local context more than any invariant notion of machine-generated text." Note the authors disclose Pangram and GPTZero donated the API credits.
• Pangram is absent from RAID, the only large detector leaderboard not run by a detector vendor. GPTZero is listed there at 0.984 AUROC. There is nowhere neutral to look Pangram's number up next to its competitors' on a dataset none of them controls.

Part of why that evidence base is thin is cost, and it is a structural point rather than a knock. A TU Darmstadt group (arXiv:2606.04906) dropped Pangram from a six-dataset benchmark outright, writing that testing the proprietary detectors "is infeasible with current pricing (over 1500$ in total)." When a detector's API pricing prices academics out of auditing it, the audit does not happen.

Worth saying plainly: Pangram's own eleven-benchmark table is labelled as third-party benchmarks, and the datasets genuinely are third-party, but the scoring is Pangram's own. That is a weaker claim than independent results, and two rows in Pangram's own table cut against the marketing. On DetectRL, a fine-tuned RoBERTa baseline beats Pangram 4 (99.75 against 96.93 F1 on the multi-domain split), and Pangram 4 scores below Pangram 3 on every DetectRL split. The upgrade is not monotone, and Pangram published that anyway.

What the false-positive stories have in common

Search Pangram on any writing forum and you will find people whose own work got flagged. The reflex is to dismiss these as cheaters seeking deniability. Having read a few hundred of them, I do not think that holds, because the stories cluster by register rather than by skill.

"Writing professor here. I've been teaching composition for 25 years. To be honest, I'm alarmed by Pangram. It said my writing is AI-generated, with a 100% degree level of certainty. I write comedy. Comedic writing has very specific rules, such as the "rule of three" (or comic triple), a classic comedic technique. As a comic writer, I have to tighten and re-tighten my writing over and over and over and over again, line by line. It's torturous. […] I might just abandon writing at this point."

Rare-Information5080, r/academia, Reddit

The mechanism there is checkable and it is not a vibe: prose that has been tightened toward a formal constraint converges on the same surface statistics as generated prose. The best-documented case anywhere is one variable, reported by a teacher who otherwise rates Pangram highest in its class.

"Turns out, I had a section in the text with three bullet points. I removed the bullets (literally just the bullets themselves, no actual text) and it passed as human."

pinnate, Hacker News

Same family: epigraphs written as clipped statements of fact, twenty years of marketing copy, academic prose with correct grammar and an em dash habit. If your writing has a deliberate house style, you are closer to the boundary than someone writing loosely. Which is a slightly grim thing to learn about your own craft, and it explains the perverse incentive people describe, where cleaner prose scores worse than error-free writing is supposed to.

That is also why "add some errors back in" keeps showing up as folk advice, and why an entire genre of detection-avoidance guides exists. Chasing a score that way is the wrong lever, and it makes the writing worse for the reader who was the point.

I would rather see people invest in a real editing process and in genuine personality in the writing. I reached the same conclusion about better prompts: fix the input, not the score.

The strongest criticism is not any of these anecdotes though. It is arithmetic, from a named academic.

"Suppose every instructor started using an AI detector on all student work. I'd estimate that students submit 500 – 1,000 written works in the course of a 4 year education […] If each of these were run through an AI detector with a FPR of 1 / 10,000, you'd have 5–10% of your student body falsely accused of cheating at least once."

Arvind Narayanan, LinkedIn

That argument survives contact with Pangram's numbers because it accepts them. Pangram's cofounder Bradley Emi replied to the same thread on X, so this is a two-sided exchange rather than a pile-on. The related statistical point from a Hacker News thread is that a false positive rate is not a false discovery rate, and the number people quote is not the number they mean: if you flag 50 true positives and 10 false ones, the FPR can be 1 in 10,000 while the chance a given flag is wrong is 1 in 6.

Pangram's own ethics section does not dodge this, and I would rather quote it than paraphrase:

"False accusations of AI usage can lead to serious consequences, including reputational damage, emotional trauma, and other undue harm. We acknowledge that our model has a non-zero error rate and its errors may result in such harms."

Pangram 4 model card

There is also a workflow in the community that actually works, and it is the most useful thing I read all week. One professor runs the assignment prompt through several models first, gathers everything else, and treats the detector as the last step rather than the first.

"I state that the AI detector is a "last chance at exoneration" and only comes after all the due diligance from before. If the detector says its real (even if it's a false negative) I drop it and don't report it. […] But when I point out that the detector is the last step and not the first step, this past year I have gone 24-for-24 in academic integrity submissions"

tilteddriveway, r/Professors, Reddit

Detector last, never first. That is the one thing both camps in this argument actually agree on.

Pricing: the 10x nobody announced

Pangram's plans are simple. The change underneath them is not.

Plagiarism detection is the paywall line: absent on Free, present on every paid tier. The $200 monthly API allowance sits on Professional only, not Individual and not Team. LMS integrations for Canvas, Brightspace, Moodle and Google Classroom are Institutional-only. Zero data retention and rate limits above 5 QPS are Enterprise-only, both unpriced.

Now the interesting bit. Pangram 4 changed the billable unit from one scan per 1,000 words rounded up, to one scan per 100 words. That is a straightforwardly good change, and Pangram framed it as "letting you make the most of your credits." At the same time, the monthly ceilings were halved: Individual from 600,000 words to 300,000, Professional from 3,000,000 to 1,500,000.

Those two changes pull in opposite directions, and the break-even is a 500-word document. Below that you win, up to 5x on a 100-word snippet. Above it you lose, down to half on anything 1,000 words or longer. Pangram's own framing of the API side matches: the change "represents a price change of 1-2x for short documents and up to 10x for long-form documents."

On the API the arithmetic is blunter. The dollar figure did not move, it is $0.05 either way, but the unit shrank tenfold.

The Pangram 4 API costs 7.5x more per word than the Individual subscription and 11.5x more than annual Professional. If your volume fits inside 1,500,000 words a month, a subscription is dramatically cheaper. The API is priced for embedding detection inside a product, not for bulk self-serve scanning. Worth sizing this against the rest of the stack, since a detector is rarely the biggest line in an AI blog writer's cost.

One trap the pricing page does not mention. Pangram 3 is supported until September 30, 2026, and until then API calls that omit the model argument keep routing to Pangram 3 on the old billing. After that date, the same call silently becomes ten times more expensive per word. If you have a script calling Pangram, go and pin model="pangram-4" now, and put a note in your calendar rather than finding out from an invoice. My writeup of the blog writer API has the wider version of that lesson about pinning versions.

Humanizer detection, and why I have skin in this

Here is where I stop being neutral, because eesel runs its own blog writer as its content engine, and Pangram is the last gate every draft goes through before it publishes. That is a few hundred drafts of firsthand experience with this specific detector, and the gate sits at a fixed point in my blog writing workflow rather than floating about.

The single most useful thing I learned is one Pangram deserves credit for. Rewriting for style does not move the score at all. Not voice, not rhythm, not first-person anecdotes, not scrubbing stock phrases. I tried all of it. What Pangram is reading is how predictable each token is, segment by segment, which means polished model output flags precisely because it keeps picking the most likely next word. That finding cost me weeks and it is, annoyingly, a point in Pangram's favour: the thing it measures is not a surface style you can paint over.

Pangram 4 goes a step further. The four-class humanizer head now labels text as Human, AI-Generated, AI-Edited, or Humanized-AI, and the API returns an is_humanized boolean that trips at a humanizer_score of 0.91 by default. Pangram reports catching AI involvement in humanized output 98.83% of the time across 13 commercial tools. So a whole category of workaround, the one that powers the content humanization tool market and things like the Surfer humanizer, is now its own labelled output field rather than an undetected gap. Pangram also disclosed something honest here: in ablations where the humanizer head's gradients reached the base model, the overall false positive rate held but the ablation "concentrated the failures in specific registers: academic writing and ESL essays." They froze the head to avoid that. Very few vendors publish the experiment that went wrong.

They also ran a red team properly. They handed an agent, Codex GPT 5.6 Sol, live API access for 24 hours with the explicit goal of engineering repeatable false negatives. It found exactly one bypass, by mimicking dictated surgical pathology notes, which Pangram then declared out of scope because the output was terse bullet points rather than open-ended prose. That carve-out is the softest moment on an otherwise rigorous page. The attack still worked.

And the conclusion I actually draw from all of it is not about detectors. eesel's traffic moved with search policy changes, not with detector scores. Google's guidance targets unhelpful content regardless of how it was produced, which is a different question from provenance, and I have the analytics to show the two do not track each other. A low Pangram score is hygiene. It is not a strategy, and it will not rescue a page that nobody needed. The honest lever is the boring one: research the thing properly, have a human edit it, and answer a real question. That is what my notes on E-E-A-T compliant content and on drafts that fail to rank are all about, and it is the same reason scaling content safely beats scaling it fast.

Pangram 4 versus the other detectors

Pangram is the most expensive option here by a distance, and on measured false positive rate it is also the best of the ones independent teams have tested. Both things are true.

A pattern worth naming: no vendor here publishes a credible accuracy figure and a matched-methodology false positive rate together. Winston claims the highest accuracy with no FPR at all. Copyleaks claims the lowest FPR but scopes accuracy to internal English testing. Turnitin publishes the best-validated FPR and no accuracy headline. Two of them undercut their own number on the same page. Pangram is the only one that publishes both with confidence intervals and sample sizes, and that transparency is most of why it earns the benefit of the doubt.

Verdict: Pangram if you need provenance and can afford it. GPTZero if you want a cheaper second opinion, and it is on RAID, which counts for something. Turnitin if you are an institution and workflow integration matters more than the score. Skip the free and open-source detectors: the UChicago study measured an open RoBERTa baseline at a 0.50 false positive rate and called it "unsuitable for high-stakes applications," which is generous.

Who should actually buy this

Buy it if you are a platform, a publisher, or a journal that needs to know how a submission was probably produced, in English, at length, and you have a corroboration step after the score. Pangram is the best tool for that job and it is not close.

Buy the subscription, not the API, unless you are embedding detection in a product. At 7.5x the per-word cost, API credits only make sense when you need the integration rather than the volume.

Think hard if you are an educator. The tool is strong and the base-rate math is still against you at institutional scale, which is why Pangram's own ethics section and its loudest critics land in the same place. Run it last, never first.

Skip it if your content is mostly non-English, mostly short, mostly templated, or mostly code and reference material. The published numbers do not describe that population, and Pangram says so itself.

Skip it if what you actually want is content that ranks. A detector tells you nothing about whether a page deserves to exist. For that, go read about AI content creation and pick a content generation tool instead.

If that last one is where you actually are, the useful comparisons are my roundup of AI content generators and my shortlist of AI writer tools. Teams building this into a repeatable system usually want a content pipeline tool rather than another point solution.

One last note on the review base, because it surprised me. Pangram's G2 listing exists with zero reviews. Capterra has no listing at all. The Chrome extension sits at 5.0 from 19 ratings with 20,000 users, and search listings put Trustpilot around 2.4 out of 5 from 13 reviews, though the page itself refused every attempt to load it. So there is no broad review corpus here in either direction, and anyone citing "the reviews" on this product is citing about thirty people. The real signal lives in academic papers and forum threads, which is unusual and, for a research-led company, sort of fitting.

Try eesel for the part a detector cannot do

Pangram tells you how a draft was probably written. It cannot tell you whether the draft was worth writing, and after a few hundred posts through that gate, I am confident the second question is the one that moves traffic.

That is the job eesel does. The eesel blog writer researches a topic from primary sources, writes with brand voice training, builds the internal link graph, and hands you a draft that a human editor can finish rather than rescue. It is the same pipeline that produced this post, detector gate included, which is about as honest a demo as I can offer.

You can see automating blog writing, compare it against the field in my AI blog writer review, and then try eesel free. If your bar is search performance specifically, start with the SEO blog writer angle instead.