The three things sold under one name
Almost every list of AI market research tools mixes three categories that solve different problems. Knowing which one you are looking at tells you more than any feature table.
- A general AI assistant. Fast, cheap, and answers from what it absorbed during training rather than from a document it read this morning. Excellent for framing a question; unsafe for a figure you will put in front of a board.
- A survey and consumer-insight platform with AI analysis. The data is primary - it is collected from respondents - and the AI summarises it. Strong where the question is what people think, weak where the question is what a company did.
- An agent research system. AI agents research named companies and markets against named sources on a schedule, and the output is a brief rather than a conversation. Strong on company, competitor and market activity; it does not ask anybody anything.
How the research behind a credible answer actually works
Any output you can defend has gone through five distinct stages, whether a tool names them or not. When a tool collapses them into one step, the step it has removed is almost always the third.
- Research. Reading the source document itself rather than a summary of it, against a named list of sources.
- Assessment. Weighing each source for relevance and for what it actually is - an official record, a company’s own statement, or independent reporting.
- Cross-checking. Counting how many independent sources carry a claim, after near-duplicate coverage of one story has been merged.
- Analysis. Reading the finding against what was already known, so a change is reported as a change rather than as news.
- Synthesis. Writing the few things that survive into something a person can act on, each line tracing back to its source.
The twelve capabilities worth testing
Four groups, in the order they matter. The first group is disqualifying: a tool that fails any of those three cannot be made safe by anything further down the list, because everything else is built on them.
Score each one yourself on a trial. A vendor who cannot demonstrate a capability in a live session does not have it, whatever the feature page says.
- 1. Traceability of information. Does every figure and claim open the specific dated document behind it, or only a domain name? A citation you cannot open is decoration. This is the one test that predicts all the others.
- 2. Information quality. Is each source classified for what it actually is - an official record, the company’s own statement, or independent reporting - or are all sources weighted the same? A filing and a blog post are not equivalent evidence, and a tool that treats them alike will mislead you confidently.
- 3. Cross-checking. Does the tool count how many independent sources carry a claim, and does it collapse ten reprints of one press release into one? Ask it to show you a finding with a source count of one.
- 4. Research depth. Does it read the document, or a summary of a summary? The test is whether it can quote a specific sentence from a filing or a careers page, rather than paraphrasing what such a document usually says.
- 5. Research methodology, stated. Can the vendor write down the stages between a question and an answer? A tool with no stated method has no method you can audit, and you will inherit its unexamined choices.
- 6. Company intelligence. Given a company name, can it assemble a dated account of what that company has done - filings, leadership, corporate activity - with each item sourced?
- 7. Competitor intelligence. Pricing changes shown as two dated captures of the rival’s own page, launches and positioning changes with a source count, not a feed of mentions.
- 8. Market analysis. Can it size a market from a named statistical series with the period stated, and does it tell you when no series measures your market rather than producing a figure anyway?
- 9. Technology intelligence. Can it infer a stack from a company’s own engineering evidence, and does it state its confidence per signal rather than asserting a stack as fact?
- 10. Pricing intelligence. Dated before-and-after captures, history you can read on day one rather than after weeks of waiting, and a clear answer on what it will not read - anything behind a login, a form or a sales call.
- 11. Hiring and talent intelligence. Requisition volume read as planned investment rather than a list of vacancies, normalised so a large company hiring ten people is not confused with a small one hiring ten.
- 12. Reporting. Output in the format the meeting actually runs on - a brief, a document, a spreadsheet, slides - not only a dashboard somebody has to log into. Plus the two things most reporting omits: what was held back, and what the system does not do.
How to score a trial against the framework
Twelve criteria is too many to weigh evenly, and weighing them evenly is how a procurement exercise picks the wrong tool. A workable split: the first three are pass-or-fail, the next three decide whether the output is defensible, and the last six are coverage - they matter only in proportion to the questions you actually ask.
If you only have an hour, test one, three and eight. Traceability tells you whether anything else can be trusted; cross-checking tells you whether the tool understands evidence; and asking it to size a market where no agency series exists tells you, faster than any other question, whether it would rather invent a number than admit a gap.
How to test one in an hour
Pick a company you already know well - ideally your own, or a competitor you have researched by hand. Ask the tool for an account of what changed at that company in the last quarter, then check the output against what you know.
Three things will separate a research system from a confident writer. Open three citations at random and see whether they say what the output claims. Find a claim you know to be contested and see whether the tool reports the disagreement or picks a side silently. Then ask it something there is genuinely no good source for, and see whether it says so or invents an answer.
Where every tool in this category still fails
Three limits are structural rather than a matter of which product you choose, and a vendor who does not volunteer them is worth less trust than one who does.
- Nothing in this category predicts. Growth rates and trend lines describe windows that have already closed. A forecast is a human judgement with assumptions attached.
- Agreement between sources is not truth. Three independent outlets can all be wrong, and the honest report of three independent sources is "reported independently", never "proven".
- Private information stays private. A tool reading what companies state on the record cannot tell you what was decided in a meeting, and a tool that claims to has usually bought something you would rather not be buying.