Convolutional Neural Network

A convolution is an integral that expresses the amount of overlap of one function as it is shifted over another function1.

Problem of Fully Connected Deep Neural Networks

Theoretically, fully connected (or dense) deep neural networks are sufficient to study any input/output relationships2. However, using a dense neural network is not without costs. In a dense neural network, we would have too many weights and activation functions to optimize. In addition, while we connect all the inputs to the output in a dense neural network, the noises will also go though the feedfward pass. Because of this, it is hard to train and optimize a dense neural network. To solve this, convolutional neural network (CNN) leverage some distinct features of image data to better find the important features and drop the unnecessary features more faster.

Consider learning an image

While we human beings recognize an object from a picture, we focus more on a certain important area rather than the whole image. This is because in an image, important features are adjacent. We call the regional adjacent features local features. To better learn the local features, we develop several smaller neural networks, rather than use a single dense layer to take all the input.

cnn detector

While the local features in a same category may exists in any area, we can shift those smaller neural networks around the whole image. This is why we name this layer as convolutional layer.

cnn conv

What’s more, we may also reduce the resolution of the inputs. This is because reducing the resolution of an image to a certain level will not affect the performance of recognizing it. Benefit from the reduced the data size, we can then reduce the total number of required neural.

cnn pool

CNN Case Study