Convolutional Neural Network
A convolution is an integral that expresses the amount of overlap of one function as it is shifted over another function1.
Problem of Fully Connected Deep Neural Networks
Theoretically, fully connected (or dense) deep neural networks are sufficient to study any input/output relationships2.
However, using a dense neural network is not without costs.
In a dense neural network, we would have too many
activation functions to optimize.
In addition, while we connect all the inputs to the output in a dense neural network,
noises will also go though the feedfward pass.
Because of this, it is hard to train and optimize a dense neural network.
To solve this, convolutional neural network (CNN) leverage some distinct features of image data
to better find the important features and drop the unnecessary features more faster.
Consider learning an image
While we human beings recognize an object from a picture,
we focus more on a certain important area rather than the whole image.
This is because in an image, important features are adjacent.
We call the regional adjacent features
To better learn the local features, we develop several
smaller neural networks,
rather than use a single dense layer to take all the input.
While the local features in a same category may exists in any area,
we can shift those
smaller neural networks around the whole image.
This is why we name this layer as
What’s more, we may also reduce the resolution of the inputs. This is because reducing the resolution of an image to a certain level will not affect the performance of recognizing it. Benefit from the reduced the data size, we can then reduce the total number of required neural.