Mum Asks: Why Does AI Give Different Answers to the Same Question?

Mum asks:

I understand why ChatGPT might give two people different answers. But why can I ask the same ChatGPT the same question today and tomorrow — and get a different answer?

Because ChatGPT doesn’t always have one fixed answer waiting for every question.

If you ask what two plus two is, there is only one correct answer. But if you ask, “Which laptop should I buy?” or “Where should I go for the weekend?”, there may be several good answers.

ChatGPT generates a new response each time you ask. If several options fit your question, it doesn’t have to choose the same one every time.

This is one of the important differences between AI Search and the kind of search we’re used to. We explain this in more detail in Mum Asks: What’s the Difference Between Google Search and AI Search?

The same question, a different answer

Let’s say you’re staying at a hotel and ask at reception:

“Where should we go nearby today?”

They recommend one place.

The next day, you ask the same person the same question. This time, they suggest somewhere else.

That doesn’t seem particularly strange. There may be several good places to visit. Yesterday they suggested one; today they suggest another. Perhaps the weather has changed or there’s an interesting event happening nearby. Or perhaps both places are equally good choices.

Something similar happens with ChatGPT. If several answers make sense, it may choose a different one the next time you ask.

Context can change the answer

Now imagine a young couple asks the same person at reception where they should go nearby.

They might suggest an old town where the couple can walk around and spend the day together.

A few minutes later, a family with two young children asks almost exactly the same question. This time, the person at reception might suggest somewhere more suitable for a family.

The situation in front of them has changed, and so has the context.

ChatGPT also uses the context of the conversation. If you’ve already mentioned that you’re travelling with children, that information can affect the answer.

Even a few extra words in the question can make a difference.

“Where should I go for the weekend?”

and

“Where should I go for the weekend with my partner?”

The questions are almost identical, but the second gives ChatGPT another piece of information to work with.

Sometimes the information changes too

There’s another reason today’s answer might be different from yesterday’s: ChatGPT may have access to slightly different information.

Perhaps new information about a place has appeared, or the system has found different up-to-date sources.

We explain where ChatGPT can get information about businesses in Mum Asks: Where Does ChatGPT Get Information About My Business?

Even when the question, context and available information are almost identical, the answer can still vary. If several options fit, ChatGPT may choose a different one when it generates the response again.

Different answers don’t always mean a mistake

If ChatGPT tells you today that the capital of France is Paris and tomorrow names a different city, that’s a mistake. There is one correct factual answer to that question.

If ChatGPT suggests one place for a weekend away today and another tomorrow, both suggestions may still be perfectly reasonable. Some questions simply allow for several good answers.

Variation like this is a normal part of how ChatGPT responds to questions with more than one reasonable answer.

The useful thing to look at is what actually changed. A different factual answer and a different recommendation are very different kinds of change.

Share this article