The artificial neuron is the mathematical unit every neural network is built from — a small function that takes several numeric inputs, combines them, and produces one output. This note defines it precisely, using the linear algebra and calculus notation from the previous two categories.
The Computation, Step by Step
| Symbol | Meaning | Shape/Type |
|---|---|---|
| \(\mathbf{x}\) | Input vector — the features fed to this neuron | \((n,)\) |
| \(\mathbf{w}\) | Weight vector — learned importance of each input | \((n,)\) |
| \(b\) | Bias — a learned offset, shifts the decision independent of the inputs | scalar |
| \(z\) | The weighted sum ("pre-activation") | scalar |
| \(\phi\) | Activation function — introduces non-linearity (see the Activation Functions category next) | function |
| \(y\) | The neuron's final output | scalar |
Numerical Example
Diagram — One Neuron
Each input is multiplied by its own weight, summed with a bias, and passed through an activation function to produce the neuron's output.
Code
import numpy as np
def neuron(x, w, b, activation):
z = np.dot(w, x) + b
return activation(z)
sigmoid = lambda z: 1 / (1 + np.exp(-z))
x = np.array([2, 3])
w = np.array([0.5, -1])
b = 1
print(neuron(x, w, b, sigmoid)) # approximately 0.269import torch
import torch.nn as nn
neuron = nn.Linear(in_features=2, out_features=1) # computes w^T x + b automatically
x = torch.tensor([[2.0, 3.0]])
z = neuron(x)
y = torch.sigmoid(z)
print(y)Common Mistakes
- Forgetting the bias term — without \(b\), the neuron's decision boundary is forced through the origin, a real limitation on what it can represent (this becomes concrete in Limitations of Perceptron).
- Skipping the activation function \(\phi\) — without it, a neuron (and any network built from stacking such neurons) reduces to a plain linear transformation, as shown in Linear Transformations.
Interview Relevance
Q: "Write out the computation a single artificial neuron performs, and name each term." \(y = \phi(\mathbf{w}^\top\mathbf{x}+b)\) — a weighted sum of the inputs \(\mathbf{x}\) using learned weights \(\mathbf{w}\), shifted by a learned bias \(b\), then passed through a non-linear activation function \(\phi\) to produce the output \(y\).
Practice Question
For \(\mathbf{x}=[1,-2,3]\), \(\mathbf{w}=[0.2,0.4,-0.1]\), \(b=0.5\), compute the pre-activation \(z\) by hand.
