Convolutional Neural Networks

refer to http://cs231n.stanford.edu/

1. Let’s focus on a small area only (5x5x3)

1. Get one number using the filter

1. Let’s look at other areas with the same filter (w)

1. In practice: Common to zero pad the border

1. Swiping the entire image

1. Convolution layers

1. Pooling layer (sampling) & MAX POOLING

1. Fully Connected Layer (FC layer) Contains neurons that connect to the entire input volume, as in ordinary Neural Networks

1. ConvNetJS demo: training on CIFAR-10]

refer to http://cs.stanford.edu/people/karpathy/convnetjs/demo/cifar10.html

CNN Case Study

Case Study: LeNet-5 [LeCun et al., 1998]

Case Study: AlexNet [Krizhevsky et al. 2012]

Case Study: GoogLeNet [Szegedy et al., 2014]

Case Study: ResNet [He et al., 2015]

• Slide from Kaiming He’s recent presentation https://www.youtube.com/watch?v=1PGLj-uKT1w
• ILSVRC 2015 winner (3.6% top 5 error)
• 2-3 weeks of training on 8 GPU machine
• at runtime: faster than a VGGNet! (even though it has 8x more layers)