Understand AI Better that 90% of People, Part 4: Temperature

 

Understand AI Better than 90% of People in 2 Minutes
Part 4: Temperature

If you've ever asked an AI the same question twice and received two completely different answers, you might think it's unpredictable and maybe even a little scary.

It isn't. You've just met temperature.

Temperature is a setting that controls how much randomness an AI uses when generating a response. The AI is constantly predicting the most likely next word (or more accurately, the next token, see Part 1). Temperature influences how closely it sticks to the most likely choice versus exploring less likely alternatives.

Think of it like choosing a restaurant. At a low temperature, the AI picks the safe, reliable favorite, like Olive Garden. At a higher temperature, the AI might say, "You know what? Let's try that place in the alley that serves tacos from a converted school bus."

Once the temperature gets muy caliente, things can get a little more interesting. Sometimes that leads to amazing discoveries. Sometimes it leads to a long night of regret.

Here's a simple example. Suppose the AI is asked to complete this sentence:

"The sky is..."

The AI might calculate that the most likely next word is:

• blue (80%)
• cloudy (10%)
• gray (7%)
• the limit on your awesome potential! (1%)

With a low temperature, the AI will almost always choose "blue." With a higher temperature, it becomes more willing to consider the less likely options. That's when responses can become more creative, surprising, or occasionally a little weird.

Here's another example. 

"My boss is..."

• Low temperature: "helpful."
• Medium temperature: "helpful and supportive" 
• Higher temperature: "a mysterious life form powered entirely by coffee and calendar invites."

You can probably see where this is going.

A Hot AI Isn't a Smarter AI

One common misconception is that higher temperature makes AI smarter. It doesn't.

Temperature doesn't change what the AI knows. It changes how adventurous it is when selecting words. A higher temperature can produce greater creativity, interesting metaphors, and unexpected connections. It can also increase the chances of mistakes or hallucinations (see Part 3) because the AI is more willing to wander away from the most likely answer.

This is why different AI tools use different temperature settings for different tasks.

Writing code? Lower temperature.

Writing a legal document? Lower temperature.

Brainstorming marketing ideas, story plots, or funny social media posts? Higher temperature can be useful and a lot more fun.

Think of temperature as the AI's willingness to improvise.

But Sadly, You Can't Adjust It... Anymore.

Interestingly, temperature used to be a setting that users could adjust themselves. Early versions of AI chatbots had a simple slider that allowed users to adjust it themselves. Nowadays, most AI applications hide this setting and adjust it behind the scenes depending on what it's being used for.

For example, an AI assistant writing code has a lower temperature set to improve consistency, while a creative writing assistant may have a higher temperature to generate more imaginative responses. 

So, the next time an AI gives you a surprisingly creative answer, suggests tacos from a school bus in a dark alley, or tells you that the sky is the limit on your awesome potential, you'll know exactly what's happening behind the scenes.

You've just met temperature.

Next up, Part 5: Training vs. Learning

---

Thanks for reading and if you have a question, please leave a comment.

With over 25 years of technical industry experience, Sarah Writtenhouse shares her insights about job market trends, emerging technology and software development.

Find her on GitHub, LinkedIn and Medium.