How to Classify Search Intent for 50,000 Keywords and Know When You Are Wrong

Ask intent as a typed question that returns a probability across your own option list, not a label. Hand-label a few hundred keywords first. Find the confidence level where the classifier is right about as often as it claims to be. Auto-accept above that line, send the band below it to a person, and publish the error rate you measured.

That last part is the whole job. Everybody teaches the four buckets. Nobody tells you what to do with the 6,000 keywords in a 50,000 row export that sit on a boundary, and those are the rows that decide whether your content plan is right.

The short version

• The four standard buckets leak. Commercial investigation against transactional is the boundary that causes most misfiling, and local and comparison intents do not fit the four at all.
• Define your own option list. The option set is the design work. Content Harmony threw out the classic trichotomy and built nine SERP-aligned types instead.⁶
• Use a Choice question. One option from a list you defined, with probabilities across every option and a confidence value attached. Up to 255 options per call.²
• Split the judgment. On identical held-out data, one wide question scored 89.4 percent and the same judgment split into five narrow ones scored 95.0 percent. Claude Haiku moved the opposite way on the same test.⁴
• Calibrate the threshold from data. A calibrated probability means that of all the calls made at 0.8, roughly 80 percent are correct. You measure where that line sits. You do not guess it.
• Cheap loses on accuracy and can still win. An independent benchmark of 31,500 decisions put Jev at a Decision Score of 67.8 on pick-one questions against Gemini 3.8 Flash at 74.1, at roughly a twenty-eighth of the price.³
• Never let an unreviewed label drive a deletion or a redirect.

If you have not read the plain-English explainer on typed decision models, start there. This post goes straight at the part that breaks in production.

What are the types of search intent, and where does the standard list break?

The taught taxonomy is four buckets. Informational, someone wants to know. Navigational, someone wants to reach a specific site. Commercial investigation, someone is comparing before buying. Transactional, someone is buying now. It is a fine teaching device and a poor classification schema, for three reasons.

Commercial investigation and transactional share a border that nothing defines. "Best CRM for small business" is commercial. "Buy HubSpot" is transactional. Now place "HubSpot pricing." Half your team says commercial, half says transactional, and both are defensible. This is where most misfiling happens, because the two buckets differ by how close the searcher is to a decision, and closeness is a spectrum being forced into two boxes.

Local intent has nowhere to sit. "Emergency plumber near me" is transactional in that money changes hands and informational in that the searcher wants a list, when what they actually want is a map pack. Filing it under transactional tells you nothing about what page to build.

Comparison intent is not commercial investigation. "Shopify vs WooCommerce" wants a two-way comparison. "Best ecommerce platform" wants a ranked list. Both land in the same bucket and need different pages.

The distribution makes this worse. SE Ranking reports roughly 70 percent informational, 22 percent commercial, 7 percent navigational and 1 percent transactional across the queries it studied.⁷ The bucket that matters most commercially is the smallest, and it sits against the boundary your schema handles worst.

Why you should define your own intent list

The option list is not an inherited standard. It is the part of the system you design, and it determines everything downstream.

Content Harmony rejected the informational, navigational, transactional trichotomy outright and proposed nine SERP-feature-aligned types instead: Research, Answer, Transactional, Local, Visual, Video, Fresh or News, Branded, and Split Intent.⁶ Nine is not the correct number. The point is that the types align to what the SERP actually shows, so each label maps to a page you could build.

Three rules for writing your own list.

Every option must map to a distinct action. If two labels produce the same page type, merge them. If one label produces three page types depending on the keyword, split it. Intent labels that do not change what you build are decoration.

Include an explicit escape hatch. "Split intent" as a real option, not an absence. A keyword whose SERP shows three product pages and three how-to guides is genuinely split, and forcing it into one bucket manufactures an error your metrics will never catch.

Write the option descriptions as instructions, not nouns. "Transactional" is a word. "The searcher is ready to buy now and wants a page where a purchase can happen" is a rule two people can apply the same way. Every label you write is a written-down decision rule, which is more than almost any published guide on this subject offers.

Then write the question itself as a Choice: one option from your list, per keyword, with probabilities across every option and a confidence value on the result.¹² The probabilities are the mechanism for everything in the second half of this post. A model that hands you "commercial" and nothing else gives you no way to find the keywords it was unsure about.

Why five narrow questions beat one wide one

On 1,000 held-out emails, the same model was asked to make the same judgment two ways. One wide question scored 89.4 percent. The same judgment decomposed into five narrow questions and recombined in code scored 95.0 percent. Claude Haiku on the same data moved the opposite direction: its single best signal hit 94.2 percent and the five-question composite dropped to 93.2.⁴

Two things follow. Decomposition is model-dependent, so you test it rather than assume it. And on the model class this post is about, how you split the judgment was worth more than which model answered it.

Applied to intent, stop asking "what is the intent of this keyword." Ask several narrow things instead:

1. Is this someone buying now?
2. Is this someone comparing options before deciding?
3. Is this someone diagnosing a problem they have?
4. Is this someone looking for one specific brand or site?
5. Is this someone who wants a place rather than a page?

Each one is a yes or no with a probability. Your code combines them into a label. The combination rule is yours, it is deterministic, and you can read it, change it and version it, which is not true of a single opaque label.

Three gains come with this. Questions in one call run in parallel, so extra questions cost tokens but almost no time.² A keyword scoring high on both "buying now" and "comparing options" gets flagged as genuinely split. And when accuracy drops in month three, you can see which of the five questions drifted rather than staring at one number.

How do you know when your classifier is wrong?

Here is the part every other article skips.

A calibrated probability means something specific and testable. Of all the calls a classifier makes at 0.8, about 80 percent should be correct. Of the calls at 0.6, about 60 percent. If that holds, the number is a real measurement of doubt and you can build policy on it. If it does not hold, the number is decoration and any threshold you set from it is superstition. You cannot know which you have without a labeled sample. Build one.

Take 300 to 500 keywords, stratified. Not the top 300 by volume, because those are the easy head terms. Sample across volume bands and word counts, and deliberately oversample the boundary you know is weak, which for most sites is commercial against transactional.

Label them by hand, with two people where you can. Where the two disagree, the keyword is genuinely ambiguous and that disagreement rate is your ceiling. If your own team agrees only 88 percent of the time, no classifier will hit 95 against your labels, and expecting it to is how people conclude the tool is broken.

Run the classifier over the same sample and bucket by confidence. Group results into bands: 0.5 to 0.6, 0.6 to 0.7, and so on. In each band, compute the share that matched your human label. Now you have the curve, and it will not be the curve you assumed.

Set the auto-accept line from that curve, not from instinct. Pick the lowest band whose measured accuracy clears the bar your use case needs. Mapping keywords to page types for a content plan, 90 percent is plenty. Triggering a redirect, nothing below near-certainty should be automatic, and honestly it should not be automatic at all.

Then run three bands, not two. Above your measured line, auto-accept and write straight into the keyword map. In the middle band, route to a human queue sorted by search volume so the expensive keywords get read first. On a 50,000 row export that band is typically thousands of rows, which sounds bad until you compare it to reading 50,000. Below the floor, reject: do not average these in, hold them out of the plan. One open-source SEO implementation flags anything under 0.55 confidence for human review rather than letting it contribute to a score.¹⁰ Re-ask those with the actual top ten results attached rather than the keyword alone, and most resolve.

The gain here is not accuracy. It is that you can finish a sentence no content audit deck has ever contained: "42,000 of these are auto-accepted at a measured 93 percent, 6,400 are queued for review, 1,600 we could not call."

Is a cheap classifier accurate enough for 50,000 keywords?

Not on raw accuracy, no. Publish that anyway.

The independent jevals.com benchmark scored 31,500 decisions across seven models at 300 questions per task. On Choice, which is the primitive intent classification uses, Jev scored a Decision Score of 67.8 on pick-one questions against Gemini 3.8 Flash at 74.1, Jev ran at roughly one twenty-eighth of the price.³

Six points behind, on the exact question shape this post is built on. That is the honest number and it should change how you use the thing, not whether you use it. Across a 50,000 row export, six points is about 3,000 differently-labeled rows. That sounds decisive until you ask what the realistic alternative is, because it is rarely "label all 50,000 with the better model and check every one." It is usually "label a 2,000 keyword sample by hand and extrapolate," which gives you zero coverage on 48,000 rows and no error rate at all on them. Full coverage plus a measured error rate plus a review queue beats a confident spreadsheet covering 4 percent of your keywords.

Niko Alho ran the costed version of this comparison independently and published both sides. At his stated pace of 30 keywords an hour, labeling 5,000 keywords by hand is 166 hours and about 16,600 euros at 100 euros an hour. His automated run did 12,000 keywords in about 8 minutes for under 5 dollars in API credits, with 94 percent alignment against a human expert and a validation floor he set at 85 percent.⁵ Those are his numbers from his run, not ours. Note what he kept: a validation floor and an expert check. That is the three-band policy under a different name.

Two limits to hold onto. These models do not explain themselves, so when a label looks wrong you get a probability and no reasoning.⁸ And guaranteed schema conformance is not the same as a right answer. The zero-hallucination figure describes type safety, not correctness.⁹ Confidence summarizes the probability distribution. It does not guarantee the diagnosis.¹¹

What the classification is actually for

Intent labels are not the deliverable. Three things are.

Mapping keywords to page types. Comparison keywords get a comparison page. Local keywords get a location page. Diagnostic keywords get a problem-first article. If a label does not resolve to a page type, it should not be in your option list.

Finding intent pointed at the wrong page. Join your labels against the URL currently ranking for each keyword in Search Console. Transactional intent landing on a blog post is the most expensive mismatch on a content site: the searcher wanted to buy and got an explainer. Informational intent landing on a product page is the same failure reversed. This join is where the classification pays for itself, and it is the same technique as our yes or no approach to keyword cannibalization, which asks a different question over the same data.

Sizing the content gap. Group the auto-accepted labels by intent and compare against the pages you have. If 22 percent of your keywords are commercial and 4 percent of your pages serve commercial intent, that ratio is your roadmap, stated as a number rather than a hunch. The same approach works at the technical layer, covered in judging more pages than your crawler licence allows.

One note on the export. The Search Console API caps a query at 25,000 rows and does not guarantee it returns every row rather than the top ones.¹² A 50,000 keyword set means paginating by date or by page, or a third-party source. Plan the pull before you plan the classification.

What not to do

Never let an unreviewed classification drive a page deletion or a redirect. This is the one hard rule.

A wrong intent label on a content plan costs you a badly briefed article. A wrong label that removed a page or pointed a 301 at the wrong destination costs you rankings, links and, on a commercial page, revenue. Those two errors are not in the same category and no confidence number makes them equivalent. Ahrefs argues, correctly, against count-based triggers for consolidation and insists you confirm the overlap is actually costing traffic before you merge anything.¹³ Apply the same discipline here. A label is an input to a human decision on destructive changes, never the decision itself.

Three more, briefly. Do not reuse a threshold across sites, because the calibration curve is a property of your labels and your keyword mix. Do not average confidence into a single health number. And do not let labels go stale silently: SERPs shift intent, so re-run quarterly.

If you want this run against your own export rather than built from scratch, that is what our SEO work covers, and the wider approach sits in our guide to AI and LLM search.

Frequently asked questions

What are the four types of search intent?

Informational (the searcher wants to know something), navigational (the searcher wants to reach a specific site), commercial investigation (the searcher is comparing options), and transactional (the searcher is ready to act or buy). The four are a useful teaching frame, but in production they misfile local queries, comparison queries and anything sitting on the commercial to transactional boundary.

How do you classify search intent at scale?

Define your own intent option list, ask it as a typed Choice question that returns probabilities across every option, run it over the full export, and split the results into three confidence bands. Above your measured threshold, accept automatically. In the middle band, route to a human queue. Below the floor, hold out and re-ask with more SERP context.

What is a calibrated probability?

A probability is calibrated when it matches reality over many calls. If a classifier says 0.8 on a thousand keywords and about 800 of them turn out correct, the number is calibrated and you can set policy from it. If only 600 are correct, the number is overconfident and any threshold you derive from it will quietly under-deliver.

How many keywords do I need to hand-label?

Three hundred to five hundred, sampled across volume bands and word counts rather than taken off the top of the list, and oversampled on the boundary you already know is weak. Label with two people where you can. Where the two humans disagree, you have found your accuracy ceiling.

What is the difference between commercial investigation and transactional keywords?

Commercial investigation means the searcher is still choosing between options, so the page they need compares, ranks or reviews. Transactional means the searcher has chosen and wants to act, so the page they need lets them buy, book or sign up. The boundary is how close the decision is, which is a spectrum, which is why this pair produces most of the misfiling in bulk keyword categorization.

Is a cheap classifier accurate enough for keyword intent?

It is measurably less accurate than a frontier model. An independent benchmark of 31,500 decisions put the cheap typed model at a Decision Score of 67.8 on pick-one questions against 74.1 for the best model tested, at about one twenty-eighth of the price. It wins anyway when the realistic alternative is labeling a small sample by hand, because full coverage plus a measured error rate plus a review queue beats a confident spreadsheet covering 4 percent of your keywords.

Can I use search intent classification to delete pages?

No, not on its own. Deletions, merges and redirects on revenue pages get human approval regardless of confidence, because the cost of a wrong label there is rankings and revenue rather than a badly briefed article.

How often should I re-run intent classification?

Quarterly for the full set, and immediately for any cluster where the page type ranking in the SERP has changed. A label that was right in March can be wrong in September without anything on your site changing.

Figures and benchmarks checked on 22 September 2026. Independent evaluation of typed decision models is days old and moving. Where a number is a vendor claim rather than an independent measurement, it is labeled as one above.