Convolutional Neural Networks (CNNs/ConvNets) are similar to regular neural networks except there are less connections, which reduces the amount of parameters in the network. People also think of CNNs as having filters of a predefined size convolving (ie. sliding) across the previous layer with a particular stride.
- Convolutional Neural Networks (CNNs/ConvNets) (karpathy, 2016)
- A Beginner’s Guide To Understanding Convolutional Neural Networks (Adit Deshpande, 2016)
Components of CNNs
- Input layer takes in the input features
- Convolutional layer are connected to local regions in the input.
(Think: filter to compute on local region) - Pooling layer for downsizing
(Think: pooling local regions of the convolutional layer) - Fully-connected layer is use to compute the output class scores
Lasagne
Code taken from craffel@github’s Lasagne Tutorial & the Lasagne documentation is found here.
# Input layer
l_in = lasagne.layers.InputLayer(
shape=(n_examples, n_channels, width, height))
# Convolutional layer
# (ReLU is the common nonlinearity in cnns)
l_conv = lasagne.layers.Conv2DLayer(
l_in, num_filters=32, filter_size=(5, 5),
nonlinearity=lasagne.nonlinearities.rectify)
# Pooling layer
l_pool = lasagne.layers.MaxPool2DLayer(
l_conv, pool_size=(2, 2))
# (You can add more conv+pool layers)
# Output layer
l_output = lasagne.layers.DenseLayer(
l_pool, num_units=n_classes,
nonlinearity=lasagne.nonlinearities.softmax)
Hitting a wall
Like most people doing research, sometimes you succeed and sometimes you hit a wall. Today I hit a wall. Originally I wanted to add a functional CNN to yesterday’s script, but I kept hitting errors. Maybe it’s due to syntax, maybe I’m out of memory, some flawed logic, or the input is not formatted correctly. All I know is the code won’t compile and I have some debugging to do. I think tomorrow I’ll simplify. I’ll implement a simple functioning CNN and build up from there.