Computer Vision · NLP · Reinforcement Learning

Three specialisations — each built on the neural network foundation from Topic 08. Real models running live in your browser. No cloud, no API key, no cost.

Real-time object detection Live NLP inference Whisper speech-to-text RL agent simulation

Live AI Demos

Computer Vision

Upload any photo or use your webcam. Detect and count every object with bounding boxes. Classify images into 1000 ImageNet categories.

Object detection Counting Classification Webcam live

Natural Language Processing

Four NLP tasks with one Transformer. Sentiment, zero-shot classification, named entity recognition, and semantic similarity on your own text.

Sentiment Zero-shot NER Similarity

Speech — STT & TTS

Record your voice and transcribe it with Whisper, entirely in-browser. Or type any text and hear it spoken with full voice/pitch control.

Speech-to-Text Text-to-Speech Multi-language

Reinforcement Learning — Live Agent

No model download needed. Watch a Q-Learning agent learn to balance a CartPole, escape a grid maze, and play Catch — trained from scratch in your browser.

CartPole Balance

Q-Learning from scratch · Pure JS

0
Episode
0
Best score
1.00
ε (explore)

Grid Maze — Q-Table

Agent learns optimal path · ε-greedy

0
Episode
Best steps
0
Total reward
Click Train to start Q-Learning…

Core Concepts

Computer Vision

CNN Architecture

Convolutional layers detect local patterns: edges → textures → shapes → objects. Pooling reduces spatial size. Fully-connected layers classify.

Input image → Conv(3×3) → ReLU → MaxPool → Conv(3×3) → ReLU → MaxPool → Flatten → Dense(256) → Softmax → class probabilities
NLP / Transformers

Attention Mechanism

Each token attends to every other token, weighted by relevance. This parallelisable design replaced RNNs and powers BERT, GPT, and all modern LLMs.

Q, K, V = linear(token_embeddings) # Scaled dot-product attention Attention = softmax(Q @ K.T / √d) @ V
Reinforcement Learning

Q-Learning

Agent observes state → takes action → receives reward → updates Q-table. The Bellman equation drives learning: Q(s,a) ← r + γ·max Q(s',a').

# Q-table update (Bellman) Q[s][a] += α * (r + γ * max(Q[s']) - Q[s][a]) # ε-greedy: explore vs exploit a = random() if ε > rand else argmax(Q[s])
Speech — Whisper

Audio → Text Pipeline

Audio is chunked into 30s windows → converted to log-Mel spectrogram → encoded by Transformer → decoded autoregressively into text tokens.

audio → resample(16kHz) → mel_spectrogram(80 bins) → encoder() → decoder() → text
Object Detection

DETR Architecture

Detection Transformer uses a CNN backbone to extract features, then a Transformer encoder-decoder to simultaneously predict all bounding boxes and classes in one pass.

image → ResNet50 → feature map → Transformer encoder → 100 object queries (decoder) → [class, box] per query
Deep RL

DQN — Deep Q-Network

Replace Q-table with a neural network that takes state as input and outputs Q-values for all actions. Uses experience replay and target networks for stability.

Q(s, a; θ) ≈ NeuralNet(s) # Target: r + γ · max Q(s'; θ⁻) # Replay buffer: random minibatch

Prerequisites (Topic 08)

Neural networks and layers
Forward pass & matrix ops
Backpropagation & gradient descent
Activation functions (ReLU, Softmax)

What comes next (Topic 10)

Large Language Models & GPT
RLHF — learning from human feedback
Hallucination & AI safety
Prompt engineering techniques
Continue to Topic 10 →