Neural Networks Basics
A neural network is layers of simple units that learn to map inputs to outputs. A perceptron is one unit, a network stacks them. Depth lets it learn non-linear patterns that linear models miss.
You will not implement backprop from scratch; libraries do it. What you need is intuition: forward pass makes a prediction, loss measures error, backward pass nudges weights. Repeat many times.
Scaling and learning rate matter more than architecture at first. Without scaling, gradients explode. With too big a learning rate, loss bounces.
A tiny network
Use sklearn MLPClassifier for tabular data. One hidden layer can already solve many problems.
from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(hidden_layer_sizes=(64,), max_iter=500, random_state=42)
mlp.fit(X_train_scaled, y_train)
print(mlp.score(X_test_scaled, y_test))Learning rate and scaling
Always scale before neural nets. Start with learning_rate_init 0.001 and watch loss.
MLPClassifier(hidden_layer_sizes=(100,), learning_rate_init=0.001, max_iter=300)Syntax
from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(hidden_layer_sizes=(64,32), max_iter=400)
mlp.fit(X_train, y_train)Unscaled data makes neural nets diverge
Feeding raw 0-255 pixels and 0-1 targets without scaling makes gradients explode and loss becomes NaN. Always StandardScaler or /255. This Stack Overflow top answer fixes most “my MLP loss is nan”.
X_scaled = X / 255.0 # or StandardScaler().fit_transform(X)Good to know
- Perceptron is one unit, network stacks them.
- Backprop nudges weights to reduce loss.
- Scale and tune learning rate first.