Google and ChatGPT disagree in local searches

If your company ranks well in Google, you cannot assume that it will also be visible in ChatGPT. I tested 35 commercial searches in Sweden and compared the companies appearing in Google’s organic results with the companies ChatGPT recommended when asked the corresponding buying question. The Google dataset contained 269 qualifying company-and-query combinations. ChatGPT mentioned 159 of them at least once. The remaining 40.9 percent were never mentioned by ChatGPT, even though I asked each question in five different ways and repeated every version three times.

What surprised me more was that the actual Google position made very little difference. Companies ranking in positions 1–3 were mentioned by ChatGPT in 60.2 percent of cases. For positions 4–10, the figure was 58.6 percent. That is a difference of only 1.6 percentage points. There were 32 companies ranking number one in Google, and ten of them never appeared in ChatGPT at all. If ChatGPT were mainly taking Google’s first page and rearranging it, I would expect a much clearer advantage for the companies at the top. I did not find one.

This is a Swedish study, which is worth explaining if you are not familiar with the market. Sweden has around 10.6 million people and very high internet usage, but the commercial market is much smaller than the US, UK or Germany. Stockholm is by far the largest city, followed by Gothenburg and Malmö, while cities such as Uppsala, Umeå and Växjö represent smaller regional markets. The study was carried out in Swedish, with local searches in those six cities and national searches for products and services where geography matters less. That gives us a fairly contained market to study: one language, one country and a mix of large and small local markets.

Being number one in Google did not help much

The individual searches make the difference easier to understand than the averages do. For the Swedish equivalent of “moving company Uppsala”, Svartbackens ranked number one in Google and was mentioned by ChatGPT zero times out of 15. Flyttfirma Östra Aros ranked number ten and appeared in 12 of the 15 responses. The company at the bottom of Google’s first page was therefore recommended repeatedly, while the company at the top disappeared completely.

I found the same thing in accounting. For “accounting firm Umeå”, BDO ranked ninth in Google but appeared in 13 of 15 ChatGPT responses. Several companies above BDO in Google were mentioned much less often or not at all. This does not prove that a low Google ranking helps in ChatGPT. It shows something more useful: within this dataset, the Google position itself is a poor way of predicting whether ChatGPT will mention the company.

The most extreme cases are where ChatGPT recommends companies that did not qualify for the Google dataset at all. For car dealers in Stockholm, Haninge Bilpark ranked first in Google and appeared zero times in ChatGPT. Riddermark Bil was not among the qualifying direct companies in Google’s first ten organic results, yet ChatGPT mentioned it in 13 of 15 responses. Bilia followed a similar pattern and appeared frequently in ChatGPT despite not being part of the qualifying Google set.

The SEO agency search produced an example that is particularly close to home. Viva Media ranked number one in Google and was not mentioned once by ChatGPT. Pineberry ranked fifth and Topdog seventh; both appeared in 11 of 15 responses. Brath or Magnus Bråth did not qualify for the Google comparison set but appeared in 13 of the 15 ChatGPT answers. When I say a company was “not in Google”, I do not mean that Google cannot find it. I mean that it was not one of the direct providers that qualified from the first ten organic results for that particular search. That distinction matters because Google often fills its results with directories, publishers, comparison services and other intermediaries. I removed those rather than pretending they were companies competing for the same customer.

Google and ChatGPT disagree more in some verticals

The amount of overlap changes considerably from one industry to another. For real estate agents, 83.3 percent of the companies in the Google dataset were mentioned by ChatGPT at least once. For cleaning companies, only 42 percent were. Accounting firms reached 60.4 percent, car dealers 61.5 percent and moving companies 52 percent. Home insurance was much higher at 85.7 percent.

Home insurance also gave me some of the most stable ChatGPT results in the study. Länsförsäkringar and Folksam are two major Swedish insurance brands, and both appeared in all 15 responses. The Stockholm cleaning market looked completely different. Of eight companies that qualified from Google, only two appeared in ChatGPT at all, while Hemfrid, a large Swedish home-services company, appeared in every one of the 15 responses.

There was a large geographical difference as well. Across the local categories, Uppsala had 72.7 percent overlap between Google and ChatGPT. Gothenburg was at 65 percent and Umeå at 65.1 percent. Malmö and Växjö were both around 54 percent. Stockholm was only 37.5 percent: Google produced 40 qualifying company results across the local Stockholm searches, and ChatGPT mentioned just 15 of them at least once.

There is a tempting explanation for that. Stockholm is a larger market, which gives ChatGPT more plausible companies to choose from and may make it less likely that its recommendations match Google’s first page. That is a hypothesis, not a result from the study. I did not test why the difference occurs. What I can say is that there is no single Google-to-ChatGPT relationship that holds across all of these Swedish markets.

The wording of the question changes the result

There is another problem if you want to measure ChatGPT visibility: the prompt matters.

I did not ask each question once. I used five natural versions of the same buying question and ran each version three times. One formulation asked which companies were “worth comparing”. Across its three repetitions, that prompt mentioned 42.8 percent of the companies in the Google dataset. Another asked which company ChatGPT “would personally consider”. That version reached only 27.5 percent.

The second prompt also caused ChatGPT to avoid giving company recommendations much more often. Only 83 of 105 responses using that formulation contained at least one company name. A more direct version along the lines of “Can you recommend some good…” produced a company in all 105 responses. Across the entire study, 492 of the 525 ChatGPT answers contained at least one concrete company recommendation.

This makes single-prompt visibility tests difficult to take seriously. If you ask one question once and turn the answer into a ranking, a large part of what you are measuring may simply be the wording of that question and the randomness of that particular run. ChatGPT does not behave like a static search result where company number four is reliably company number four. For this kind of measurement, I think frequency is much more useful: how often does the company appear when you ask several reasonable versions of the same question?

How I collected the data

I chose ten types of purchases where a consumer could reasonably either search Google or ask ChatGPT for recommendations. Five were local services: accounting firms, moving companies, cleaning companies, real estate agents and car dealers. I tested those in Stockholm, Gothenburg, Malmö, Uppsala, Umeå and Växjö. The five national categories were SEO agencies, electricity providers, home insurance, mobile operators and home-security companies.

For Google, I used the first ten organic results returned by SISTRIX for each Swedish query. I classified those ten results and removed directories, comparison sites, affiliates, publishers and other intermediaries. I did not go to page two to replace them. If only six of the first ten results were actual companies selling the service, those six became the Google set for that search. This is why the number of Google companies varies between queries.

I then asked the corresponding recommendation questions through the OpenAI API. Each query received five prompt formulations, and every formulation was run three times. The runs were independent, with no previous conversation carried into the next one, and the prompts did not contain lists of companies from Google. That produced 15 ChatGPT responses for each query and 525 responses in total.

One of the 35 searches, real estate agents in Växjö, did not have usable SISTRIX data. I still have the ChatGPT responses, but there is nothing reliable to compare them with on the Google side. I therefore excluded that search from the overlap calculations. The Google-versus-ChatGPT figures in this article are based on 34 comparable searches and 269 qualifying Google company-and-query entries.

This study is about ChatGPT. It does not tell you how Gemini, Claude, Perplexity or every other language model behaves, and it does not prove that ChatGPT will produce the same distributions for every market or language. It also cannot tell us why ChatGPT chooses Riddermark Bil while Google chooses Haninge Bilpark, or why Stockholm shows much less overlap than Uppsala but it does tell us something practical. If you use Google rankings as a proxy for ChatGPT visibility, you are measuring the wrong thing. In this dataset, a company could be first in Google and completely absent from ChatGPT, or barely visible in Google and recommended in almost every ChatGPT response. If you want to know how visible a company is in ChatGPT, you need to measure ChatGPT.

Go through all the data in the explorer below, and please tell me if you disagree with something.