--- title: "AI Hotel Ranking — methodology & disclosures" description: "How the AI Hotel Ranking is measured: the guest questions asked, sample sizes and confidence intervals, what we exclude, the conflict-of-interest disclosure, and how to request a correction." url: "https://sigtrip.com/ai-hotel-ranking/methodology/" --- AI Hotel Ranking · Methodology # How we measure what AI recommends The AI Hotel Ranking is Sigtrip’s ongoing measure of which hotels AI recommends to travelers — one set of boards per market, built the same way in every city. We put the questions travelers actually ask to ChatGPT — with no hotel or brand names in them — through up to 4 traveler lenses, and repeat each one because the same question rarely gets the same answer twice. This page is the method behind every board: what we ask, how we count it, what we leave out, and where it can be wrong. Every number here comes straight from the answers we publish. 3 markets published 4 traveler lenses 1,498 hotels tracked 874 named by ChatGPT ## The questions We ask the questions travelers actually put to an AI — in their words, not ours, and never naming a hotel or brand. They span the ways people really search for a room. Citywide & top-of-mind “Best hotels in [city]?” District & neighborhood “Where to stay in [neighborhood]?” By budget “Budget-friendly hotels in [city]?” Near a landmark “Hotels near [landmark]?” Near an airport or station “Hotels near [airport]?” By amenity “Hotels with a rooftop bar in [city]?” Every market gets its own panel of 334 questions, generated from those same question types with that city’s own neighborhoods, landmarks and airports filled in. Keeping brand names out of them is the whole point: we want what the engine offers on its own, not what it says when we lead it. Each broad citywide question is asked 8× and each more specific one 6×, so a single lucky or unlucky answer can’t define a hotel’s standing. ## Traveler lenses The same question lands differently depending on who's asking. Where a market's run captures them, we put the whole panel through four traveler lenses, so a hotel AI recommends to families but not to business travelers reads as exactly that. - General— no trip type mentioned — what a traveler sees by default - Business— asked as someone on a business trip - Family— asked as someone planning a family trip - Leisure— asked as someone traveling for leisure On the board, the Blendedlens combines every lens that market captured — the fullest picture of who ChatGPT names — and you can switch to any single one to see how the picture shifts. Only the framing of the ask changes between lenses; the questions themselves stay word-for-word the same. Not every run captures all four: where a market’s wave ran a single lens, its board has no Lens control and Blended is that one lens. ## How a placement is measured We record which hotels the engine names, and how it ranks them — then read those answers three ways, each stricter than the last. A hotel’s placement ratefor a board is the share of that board’s eligible answers in which the engine names the hotel. Recommended rate is stricter — how often it actively recommends the hotel, not just mentions it in passing — and top-pick rateis stricter still: how often the hotel is the first or #1 choice. The “Rank by” menu on the board switches between them. When two hotels tie on the metric being ranked, the stricter measures break the tie — a hotel the engine picks first more often sits higher. A market’s overall board uses citywide questions (“best hotels in the city”); each district, neighborhood, and budget board uses that segment’s questions (77 boards across every market we publish). A hotel appears on a board because the engine named it there — not because of any tag we assign. An answer counts toward a board whether or not it names a hotel. Some questions a traveler would really ask are ones the engine answers with a neighborhood, or with a question of its own, rather than a property. Those answers stay in the denominator. A placement rate therefore reads “of the times a traveler asks this, how often does this hotel come back” — and a question the engine won’t answer straight correctly holds every hotel’s rate on that board down. Dropping those answers would quietly flatter every board that contains one, so we don’t. The rule is the same in every market. We keep two signals separate and never merge them into one score: how often a hotel is named (the ranking) and what sources the engine cites (the sources board). One is reputation; the other is where that reputation comes from. ## Sample size & confidence We show percentages, not raw counts — but every percentage carries a sample size and a confidence range underneath. Ask an AI the same question twice and you can get different answers, so a placement rate is an estimate, not a fixed number. We attach a 95% confidence rangeto each one — the band the true rate most likely sits in, given how many answers back it. A narrow band means we’re fairly sure; a wide band means read it as a rough signal. A citywide board rests on 88–352 answers depending on the market, and the district, neighborhood, and budget boards run from 15 to 96 — a market whose wave captured more traveler lenses gathers proportionally more answers on every board. Boards with fewer than 12 eligible answers are withheld as too small to publish. Wilson 95% score intervals on each placement rate (n = the board's eligible answers). Repeats cluster by prompt, so intervals are approximate. One caution when reading down a board: a hotel named in just one or two answers sits inside that run-to-run noise. Treat the head of each board as the real signal, and low single-digit percentages as “occasionally mentioned,” not a firm rank. ## Which hotels are in the running The ranking can only name hotels it knows exist. Here's how that list is built — and why being on it says nothing about whether a hotel is a Sigtrip customer. Each market’s list is assembled from public directory data — 1,498 hotels across the 3 markets we publish today, with no property’s permission needed, and being on the list doesn’t make a hotel a customer. Of those, ChatGPT named 874 at least once across our questions; the rest simply never came up in an answer a traveler would read. A further 107 hotels are held off the public boards on purpose — a validation set kept aside to check our own accuracy against answers we didn’t use to build the ranking. ## What we exclude A ranking is only as trustworthy as what it leaves out. - Blind-validation hold-out hotels (reserved to validate the method — never ranked) - Segments with fewer than 12 eligible answers (sample too small) ## Conflict of interest & corrections Sigtrip sells AI-visibility tools to hotels, including some that appear on these boards. To keep the ranking honest, being a Sigtrip customer never changes where a hotel lands — we don’t add, remove, or reorder any property, so a customer appears on a board only when the engine names it, exactly like everyone else. We record which hotels are customers for transparency, not to promote them. These figures are a snapshot of what an AI engine said at a point in time, not a judgment of any hotel. If something looks wrong, tell us and we’ll review it against the underlying answers: hello@sigtrip.com (mailto:hello@sigtrip.com) . The AI Hotel Ranking is Sigtrip’s own measurement, produced from our ChatGPT runs. Percentages are placement rates over eligible answers. [← Back to the rankings](https://sigtrip.com/ai-hotel-ranking/)