Neural networks · perceptron · training · tokens · client-side

how AI worksneural network explainerwhat is AIperceptron calculatorAI training explainedtokens explainedhow ChatGPT works

//// Educational · AI

How AI Actually Works

AI isn't magic. It's math. Here's the actual math behind neural networks, training, and language models.

401(k) Limit 2024$23,000(+$7,500 catch-up 50+)

Roth IRA Limit$7,000(MAGI phase-out applies)

S&P 500 Avg Return~10%/yr(nominal, historical)

🧠 What is a Neural Network?

A neural network is a bunch of "neurons" (nodes) connected in layers. Each connection has a weight — a number that says how much that input matters. Drag the slider and watch how fast the connections multiply!

Neurons in hidden layer4

Input

4 nodes

Hidden

3 nodes

Output

1 nodes

Total connections

4×3 + 3×1

Formula

4×3+3×1

connections = n_in × n_out per layer

connections = neurons_in × neurons_out ← try changing the slider!

GPT-4 has roughly 1.8T parameters (weights). That's 1800B connections, and training it cost ~$100M.

🏋️ What is Training?

Training is when the network sees thousands of examples and adjusts its weights to get better. Each pass through the data is called an "epoch." Watch accuracy climb toward ~95% and loss fall toward 0 as the model learns!

Epoch

/ 50

Accuracy

0.0%

1 − e^(−0.1×epoch)

Loss

1.000

1 − accuracy

Model accuracy0.0%

accuracy ≈ 1 − e^{(−0.1 × epoch)}

🔤 What are Tokens?

AI models don't read words or letters — they read tokens. A token is roughly a word or word-piece. GPT-4 charges per token. Type something and see it split!

Thequickbrownfoxjumpsoverthelazydog

Tokens

Characters

Chars/token

4.8

≈ 4 chars typical

GPT-4 has a context window of ~128,000 tokens. That's roughly a 300-page book at once.

⚙️ The Math Behind It: Perceptron

A perceptron is the simplest neural network: multiply each input by its weight, add a bias, then squash through sigmoid to get a 0–1 probability. This is the same operation repeated billions of times inside GPT-4.

Input x1

Input x2

Input x3

Weight w11.0

Weight w2-0.5

Weight w30.8

Bias0.2

Weighted sum: 0.9900 + bias: 1.1900

Output: 76.7%

→ Prediction: YES (class 1)

Weighted sum

Σ(xᵢ × wᵢ) = (0.5 × 1.0) + (0.3 × -0.5) + (0.8 × 0.8)

= 0.9900

Add bias

0.9900 + 0.2

= 1.1900

Sigmoid activation

σ(z) = 1 / (1 + e^(−1.190))

= 76.7% probability

#ShowYourWork