Lesson 2.6MathCode~40 min

Very big & very small numbers

Scientific notation, orders of magnitude, log scales — and the surprising ways computers get arithmetic wrong.

Module 2 · 0/9 done

From tiny to gigantic

AI deals with numbers at both extremes at once. A learning rate might be 0.0003, while a model has 175 000 000 000 parameters. Writing all those zeros is slow and easy to get wrong, so we use scientific notation: a number between 1 and 10, times a power of 10.

175\,000\,000\,000 = 1.75 \times 10^{11}

0.0003 = 3 \times 10^{-4}

The power says how many places to move the decimal point: right for positive powers, left for negative ones. In code it’s written with an e: 1.75e11 and 3e-4.

Fig 2.10Real numbers from AI, from 0.00000001 to 15 trillion. On an ordinary axis they’re invisible; on a log axis every factor of ten gets the same room.