Convolutional Neural Networks
One of the important areas of Deep learning is Convolutional Neural Network. CNN deals in analysing visual images. These types of neural networks works by processing, classifying and segmenting images. CNN algorithm learns about images and then are able to predict about a given image when present.
Convolutional Neural Networks
What are Convolutional Neural Networks:
One of the important areas of Deep learning is Convolutional Neural Network. CNN deals in analysing visual images. These types of neural networks works by processing, classifying and segmenting images. CNN algorithm learns about images and then are able to predict about a given image when present. CNN algorithms requires to be trained with tons of images and their possible predictor class. CNN are powerful tools for processing data having grid like topology. CNN involves convolution which means cross-corelation operations (instead of a fully connected layer) as one of its layers.
These neural networks are successful in many different real-life case studies and applications like:
- Image classification, Object detection, segmentation, face recognition
- Self driving cars that leverage CNN based vision systems
- Classificationof crystal structure using a convolutional neural network
Convolution is the key principle applied: in convolutional neural network architecture. Convolution is a way to identify patterns in data that is directly tied to space or time. Assume we have a one dimentional array of numbers say Input = [a,b,c,d,e,f,g] and another set of numbers say K = [p,q]. Then convolution involves sliding the lower size list over the higher size list and then multiplying corresponding values and adding them therefore after convoluting K over Input we get [ap+bq, bp+cq, cp+dq, dp+eq, fp+gq]. The result identifies one of the feature of the Input. The result is known as the convolution of the Input and in practice only non-zero values are choosen for feature selection.If the input is a grid structure or in matrix form (in case of images) then the kernal is also choosen as the matrix of lower dimension and in this case matrix multiplication is perfomed to get the resultant convolution of the input image.
Computer see images as matrices:Grayscale images have single channel (gray).
So, we can represent grayscale images in the form of 2D matrix, where each element represents the intensity of brightness in that particular pixel,
where 0 means black and 255 means white. Color images have 3 channels RGB (red, green, blue).
Color images can be represented as a 3D matrix with the depth of 3.
For example: Shape of a matrix representing a 480px by 852px color image will be (480, 852, 3) Each pixel of the color image has three numbers (ranging from 0 to 255) associated with it. These numbers shows the intensity of red, green and blue color in that particular pixel.
import os
import cv2
import matplotlib.pyplot as plt # (optional) for plotting and showing images inline
IMAGES_FOLDER = os.path.join('../../static/img/main') # images for visuals
earth_fname = os.path.join(IMAGES_FOLDER,'earth.jpg')
earth_img1 = cv2.imread(earth_fname)
print(earth_img1.shape)## (480, 852, 3)The CIFAR 10 dataset has the Input layer of 60000 32x32 colour images in 10 categories, with 6000 images per class.
In a covnolutional neural network:
Input layer: Takes an image as input and preserves its spatial structure Convolution layer: extracts feature maps from the input, each responding to a specific pattern ReLU layer: Introduces non-linearities in the network by putting the negative pixels to 0.
Kernel, stride and padding
Filters also known as kernals, convolve square blocks of pixels into scalars in subsequent convolutional layers. In the animation above, we have a 3 x 3 filter with ones running on the diagonal and off-diagonal, scanning an image from left to right, top to bottom.
Throughout the process, the filter performs element-wise multiplication and sums up all products, into a single value passed to the subsequent convolutional layer. Note that the filter is moving a pixel at a time. This is the stride, the stepsize of the sliding window the filter uses to convolve. Larger size of strides indicates more granular and smaller convolved features.
Example Convolution layer in Python
The first layer takes input as set of images specified with input_shape. filters, kernal size, strides and padding are the most important parameters to keras Cov2D. The parameter filters denote the number of filters. The task of a filter is to detect a feature in the image.
from keras.models import Sequential## Using TensorFlow backend.
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\python\framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_qint8 = np.dtype([("qint8", np.int8, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\python\framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\python\framework\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_qint16 = np.dtype([("qint16", np.int16, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_qint32 = np.dtype([("qint32", np.int32, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## np_resource = np.dtype([("resource", np.ubyte, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_qint8 = np.dtype([("qint8", np.int8, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_qint16 = np.dtype([("qint16", np.int16, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## _np_qint32 = np.dtype([("qint32", np.int32, 1)])
## C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
## np_resource = np.dtype([("resource", np.ubyte, 1)])from keras.layers import Conv2D
model = Sequential()
model.add(Conv2D(filters=16, kernel_size = 3, padding = 'same',activation = 'relu',input_shape=(32,32,3)))
model.summary()## Model: "sequential_1"
## _________________________________________________________________
## Layer (type) Output Shape Param #
## =================================================================
## conv2d_1 (Conv2D) (None, 32, 32, 16) 448
## =================================================================
## Total params: 448
## Trainable params: 448
## Non-trainable params: 0
## _________________________________________________________________Example Convolution layer in R
library(tensorflow)
library(keras)
model <- keras_model_sequential()
model%>%
layer_conv_2d(filters = 16, kernel_size = c(3,3),activation = 'relu',input_shape = c(32,32,3))
summary(model)## Model: "sequential"
## ________________________________________________________________________________
## Layer (type) Output Shape Param #
## ================================================================================
## conv2d (Conv2D) (None, 30, 30, 16) 448
## ================================================================================
## Total params: 448
## Trainable params: 448
## Non-trainable params: 0
## ________________________________________________________________________________pooling layer: Down-samples the rectified feature maps, thus reducing the spatial dimensionality and retaining important features. This prevents overfitting.
Example max pooling layer in R
model %>%
layer_max_pooling_2d(pool_size = c(2,2)) %>%
layer_dropout(0.25)
summary(model)## Model: "sequential"
## ________________________________________________________________________________
## Layer (type) Output Shape Param #
## ================================================================================
## conv2d (Conv2D) (None, 30, 30, 16) 448
## ________________________________________________________________________________
## max_pooling2d (MaxPooling2D) (None, 15, 15, 16) 0
## ________________________________________________________________________________
## dropout (Dropout) (None, 15, 15, 16) 0
## ================================================================================
## Total params: 448
## Trainable params: 448
## Non-trainable params: 0
## ________________________________________________________________________________Example max pooling layer in Python
from keras.layers import MaxPooling2D
from keras.layers import Dropout
model.add(MaxPooling2D(pool_size=(2, 2)))## WARNING:tensorflow:From C:\Users\slaxm\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\keras\backend\tensorflow_backend.py:4070: The name tf.nn.max_pool is deprecated. Please use tf.nn.max_pool2d instead.model.add(Dropout(0.25))
model.summary()## Model: "sequential_1"
## _________________________________________________________________
## Layer (type) Output Shape Param #
## =================================================================
## conv2d_1 (Conv2D) (None, 32, 32, 16) 448
## _________________________________________________________________
## max_pooling2d_1 (MaxPooling2 (None, 16, 16, 16) 0
## _________________________________________________________________
## dropout_1 (Dropout) (None, 16, 16, 16) 0
## =================================================================
## Total params: 448
## Trainable params: 448
## Non-trainable params: 0
## _________________________________________________________________Fully-Connected layer: Learns non-linear combinations of the features and performs the classification task FlatteningFlattening converts last convolutional layer into a one-dimensional NN layer.
Example
from keras.layers import Dense, Flatten
from keras.layers.normalization import BatchNormalization
from keras.layers import Dense, Activation
model.add(Flatten())
model.add(Dense(512, kernel_initializer="uniform"))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Dropout(0.5))
# softmax classifier
model.add(Activation("softmax"))Object Detection using Convolutional Neural Network
We are going to use another Keras dataset, which contains numerous images of ten different categories. These are the following:
[‘Plane’ , ‘Car’ , ‘Bird’ , ‘Cat’ , ‘Deer’ , ‘Dog’ , ‘Frog’ , ‘Horse’ , ‘Ship’ , ‘Truck’ ]
This dataset contains tens of thousands of images of different objects with their respective class. Our goal here is to train a convolutional neural network on that data, in order to then classify other images that the model has never seen before.
Importing libraries
import cv2 as cv
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras import datasets, layers, models(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()## Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz
##
## 8192/170498071 [..............................] - ETA: 1:22:49
## 24576/170498071 [..............................] - ETA: 1:22:51
## 57344/170498071 [..............................] - ETA: 47:21
## 106496/170498071 [..............................] - ETA: 31:53
## 172032/170498071 [..............................] - ETA: 23:43
## 237568/170498071 [..............................] - ETA: 20:01
## 368640/170498071 [..............................] - ETA: 14:44
## 663552/170498071 [..............................] - ETA: 9:11
## 991232/170498071 [..............................] - ETA: 6:17
## 1236992/170498071 [..............................] - ETA: 5:27
## 1581056/170498071 [..............................] - ETA: 4:21
## 1941504/170498071 [..............................] - ETA: 3:36
## 2285568/170498071 [..............................] - ETA: 3:07
## 2416640/170498071 [..............................] - ETA: 3:03
## 2629632/170498071 [..............................] - ETA: 2:51
## 2940928/170498071 [..............................] - ETA: 2:35
## 3317760/170498071 [..............................] - ETA: 2:20
## 3629056/170498071 [..............................] - ETA: 2:10
## 3989504/170498071 [..............................] - ETA: 2:00
## 4251648/170498071 [..............................] - ETA: 2:08
## 5529600/170498071 [..............................] - ETA: 1:39
## 5857280/170498071 [>.............................] - ETA: 1:35
## 6430720/170498071 [>.............................] - ETA: 1:27
## 6889472/170498071 [>.............................] - ETA: 1:24
## 7249920/170498071 [>.............................] - ETA: 1:21
## 7593984/170498071 [>.............................] - ETA: 1:18
## 7970816/170498071 [>.............................] - ETA: 1:15
## 8265728/170498071 [>.............................] - ETA: 1:13
## 8347648/170498071 [>.............................] - ETA: 1:19
## 10395648/170498071 [>.............................] - ETA: 1:06
## 11493376/170498071 [=>............................] - ETA: 1:00
## 11837440/170498071 [=>............................] - ETA: 58s
## 12197888/170498071 [=>............................] - ETA: 57s
## 12312576/170498071 [=>............................] - ETA: 1:01
## 14753792/170498071 [=>............................] - ETA: 51s
## 15065088/170498071 [=>............................] - ETA: 50s
## 15097856/170498071 [=>............................] - ETA: 51s
## 15441920/170498071 [=>............................] - ETA: 51s
## 15802368/170498071 [=>............................] - ETA: 50s
## 16162816/170498071 [=>............................] - ETA: 49s
## 16179200/170498071 [=>............................] - ETA: 52s
## 17948672/170498071 [==>...........................] - ETA: 46s
## 18063360/170498071 [==>...........................] - ETA: 47s
## 18325504/170498071 [==>...........................] - ETA: 46s
## 18604032/170498071 [==>...........................] - ETA: 46s
## 18833408/170498071 [==>...........................] - ETA: 46s
## 19046400/170498071 [==>...........................] - ETA: 46s
## 19243008/170498071 [==>...........................] - ETA: 45s
## 19456000/170498071 [==>...........................] - ETA: 45s
## 19734528/170498071 [==>...........................] - ETA: 45s
## 19996672/170498071 [==>...........................] - ETA: 45s
## 20160512/170498071 [==>...........................] - ETA: 45s
## 20422656/170498071 [==>...........................] - ETA: 44s
## 20652032/170498071 [==>...........................] - ETA: 44s
## 20930560/170498071 [==>...........................] - ETA: 44s
## 21192704/170498071 [==>...........................] - ETA: 44s
## 21340160/170498071 [==>...........................] - ETA: 44s
## 21585920/170498071 [==>...........................] - ETA: 44s
## 21848064/170498071 [==>...........................] - ETA: 43s
## 22077440/170498071 [==>...........................] - ETA: 43s
## 22355968/170498071 [==>...........................] - ETA: 43s
## 22618112/170498071 [==>...........................] - ETA: 43s
## 22863872/170498071 [===>..........................] - ETA: 42s
## 23158784/170498071 [===>..........................] - ETA: 42s
## 23355392/170498071 [===>..........................] - ETA: 42s
## 23568384/170498071 [===>..........................] - ETA: 42s
## 23830528/170498071 [===>..........................] - ETA: 42s
## 24059904/170498071 [===>..........................] - ETA: 42s
## 24338432/170498071 [===>..........................] - ETA: 41s
## 24535040/170498071 [===>..........................] - ETA: 41s
## 24780800/170498071 [===>..........................] - ETA: 41s
## 25042944/170498071 [===>..........................] - ETA: 41s
## 25305088/170498071 [===>..........................] - ETA: 41s
## 25600000/170498071 [===>..........................] - ETA: 40s
## 25796608/170498071 [===>..........................] - ETA: 40s
## 26091520/170498071 [===>..........................] - ETA: 40s
## 26337280/170498071 [===>..........................] - ETA: 40s
## 26583040/170498071 [===>..........................] - ETA: 40s
## 26828800/170498071 [===>..........................] - ETA: 40s
## 27123712/170498071 [===>..........................] - ETA: 39s
## 27402240/170498071 [===>..........................] - ETA: 39s
## 27631616/170498071 [===>..........................] - ETA: 39s
## 27910144/170498071 [===>..........................] - ETA: 39s
## 28090368/170498071 [===>..........................] - ETA: 39s
## 28434432/170498071 [====>.........................] - ETA: 39s
## 28696576/170498071 [====>.........................] - ETA: 38s
## 28909568/170498071 [====>.........................] - ETA: 38s
## 29188096/170498071 [====>.........................] - ETA: 38s
## 29417472/170498071 [====>.........................] - ETA: 38s
## 29777920/170498071 [====>.........................] - ETA: 38s
## 30023680/170498071 [====>.........................] - ETA: 38s
## 30220288/170498071 [====>.........................] - ETA: 38s
## 30466048/170498071 [====>.........................] - ETA: 37s
## 30760960/170498071 [====>.........................] - ETA: 37s
## 31072256/170498071 [====>.........................] - ETA: 37s
## 31285248/170498071 [====>.........................] - ETA: 37s
## 31547392/170498071 [====>.........................] - ETA: 37s
## 31760384/170498071 [====>.........................] - ETA: 37s
## 32047104/170498071 [====>.........................] - ETA: 36s
## 32317440/170498071 [====>.........................] - ETA: 36s
## 32563200/170498071 [====>.........................] - ETA: 36s
## 32825344/170498071 [====>.........................] - ETA: 36s
## 33071104/170498071 [====>.........................] - ETA: 36s
## 33284096/170498071 [====>.........................] - ETA: 36s
## 33611776/170498071 [====>.........................] - ETA: 36s
## 33857536/170498071 [====>.........................] - ETA: 35s
## 34136064/170498071 [=====>........................] - ETA: 35s
## 34357248/170498071 [=====>........................] - ETA: 35s
## 34578432/170498071 [=====>........................] - ETA: 35s
## 34889728/170498071 [=====>........................] - ETA: 35s
## 35151872/170498071 [=====>........................] - ETA: 35s
## 35414016/170498071 [=====>........................] - ETA: 35s
## 35627008/170498071 [=====>........................] - ETA: 35s
## 35848192/170498071 [=====>........................] - ETA: 35s
## 36167680/170498071 [=====>........................] - ETA: 34s
## 36478976/170498071 [=====>........................] - ETA: 34s
## 36708352/170498071 [=====>........................] - ETA: 34s
## 36970496/170498071 [=====>........................] - ETA: 34s
## 37249024/170498071 [=====>........................] - ETA: 34s
## 37511168/170498071 [=====>........................] - ETA: 34s
## 37838848/170498071 [=====>........................] - ETA: 33s
## 38035456/170498071 [=====>........................] - ETA: 33s
## 38297600/170498071 [=====>........................] - ETA: 33s
## 38559744/170498071 [=====>........................] - ETA: 33s
## 38854656/170498071 [=====>........................] - ETA: 33s
## 39165952/170498071 [=====>........................] - ETA: 33s
## 39378944/170498071 [=====>........................] - ETA: 33s
## 39591936/170498071 [=====>........................] - ETA: 33s
## 39886848/170498071 [======>.......................] - ETA: 33s
## 40181760/170498071 [======>.......................] - ETA: 32s
## 40493056/170498071 [======>.......................] - ETA: 32s
## 40706048/170498071 [======>.......................] - ETA: 32s
## 40919040/170498071 [======>.......................] - ETA: 32s
## 41230336/170498071 [======>.......................] - ETA: 32s
## 41525248/170498071 [======>.......................] - ETA: 32s
## 41869312/170498071 [======>.......................] - ETA: 32s
## 42098688/170498071 [======>.......................] - ETA: 32s
## 42311680/170498071 [======>.......................] - ETA: 32s
## 42557440/170498071 [======>.......................] - ETA: 32s
## 42835968/170498071 [======>.......................] - ETA: 31s
## 43196416/170498071 [======>.......................] - ETA: 31s
## 43425792/170498071 [======>.......................] - ETA: 31s
## 43655168/170498071 [======>.......................] - ETA: 31s
## 43966464/170498071 [======>.......................] - ETA: 31s
## 44130304/170498071 [======>.......................] - ETA: 31s
## 44457984/170498071 [======>.......................] - ETA: 31s
## 44736512/170498071 [======>.......................] - ETA: 31s
## 44933120/170498071 [======>.......................] - ETA: 31s
## 45195264/170498071 [======>.......................] - ETA: 30s
## 45473792/170498071 [=======>......................] - ETA: 30s
## 45785088/170498071 [=======>......................] - ETA: 30s
## 46063616/170498071 [=======>......................] - ETA: 30s
## 46227456/170498071 [=======>......................] - ETA: 30s
## 46522368/170498071 [=======>......................] - ETA: 30s
## 46784512/170498071 [=======>......................] - ETA: 30s
## 47128576/170498071 [=======>......................] - ETA: 30s
## 47374336/170498071 [=======>......................] - ETA: 30s
## 47685632/170498071 [=======>......................] - ETA: 30s
## 47931392/170498071 [=======>......................] - ETA: 29s
## 48259072/170498071 [=======>......................] - ETA: 29s
## 48537600/170498071 [=======>......................] - ETA: 29s
## 48701440/170498071 [=======>......................] - ETA: 29s
## 48979968/170498071 [=======>......................] - ETA: 29s
## 49291264/170498071 [=======>......................] - ETA: 29s
## 49586176/170498071 [=======>......................] - ETA: 29s
## 49881088/170498071 [=======>......................] - ETA: 29s
## 50044928/170498071 [=======>......................] - ETA: 29s
## 50339840/170498071 [=======>......................] - ETA: 29s
## 50675712/170498071 [=======>......................] - ETA: 28s
## 50880512/170498071 [=======>......................] - ETA: 28s
## 51191808/170498071 [========>.....................] - ETA: 28s
## 51355648/170498071 [========>.....................] - ETA: 28s
## 51699712/170498071 [========>.....................] - ETA: 28s
## 52011008/170498071 [========>.....................] - ETA: 28s
## 52240384/170498071 [========>.....................] - ETA: 28s
## 52535296/170498071 [========>.....................] - ETA: 28s
## 52781056/170498071 [========>.....................] - ETA: 28s
## 53092352/170498071 [========>.....................] - ETA: 28s
## 53370880/170498071 [========>.....................] - ETA: 27s
## 53649408/170498071 [========>.....................] - ETA: 27s
## 53862400/170498071 [========>.....................] - ETA: 27s
## 54059008/170498071 [========>.....................] - ETA: 27s
## 54370304/170498071 [========>.....................] - ETA: 27s
## 54681600/170498071 [========>.....................] - ETA: 27s
## 54976512/170498071 [========>.....................] - ETA: 27s
## 55205888/170498071 [========>.....................] - ETA: 27s
## 55386112/170498071 [========>.....................] - ETA: 27s
## 55664640/170498071 [========>.....................] - ETA: 27s
## 55992320/170498071 [========>.....................] - ETA: 27s
## 56221696/170498071 [========>.....................] - ETA: 27s
## 56516608/170498071 [========>.....................] - ETA: 26s
## 56778752/170498071 [========>.....................] - ETA: 26s
## 57040896/170498071 [=========>....................] - ETA: 26s
## 57384960/170498071 [=========>....................] - ETA: 26s
## 57630720/170498071 [=========>....................] - ETA: 26s
## 57827328/170498071 [=========>....................] - ETA: 26s
## 58089472/170498071 [=========>....................] - ETA: 26s
## 58368000/170498071 [=========>....................] - ETA: 26s
## 58679296/170498071 [=========>....................] - ETA: 26s
## 58925056/170498071 [=========>....................] - ETA: 26s
## 59138048/170498071 [=========>....................] - ETA: 26s
## 59383808/170498071 [=========>....................] - ETA: 26s
## 59678720/170498071 [=========>....................] - ETA: 25s
## 60006400/170498071 [=========>....................] - ETA: 25s
## 60317696/170498071 [=========>....................] - ETA: 25s
## 60465152/170498071 [=========>....................] - ETA: 25s
## 60776448/170498071 [=========>....................] - ETA: 25s
## 61087744/170498071 [=========>....................] - ETA: 25s
## 61349888/170498071 [=========>....................] - ETA: 25s
## 61628416/170498071 [=========>....................] - ETA: 25s
## 61816832/170498071 [=========>....................] - ETA: 25s
## 62070784/170498071 [=========>....................] - ETA: 25s
## 62414848/170498071 [=========>....................] - ETA: 25s
## 62660608/170498071 [==========>...................] - ETA: 25s
## 62971904/170498071 [==========>...................] - ETA: 24s
## 63102976/170498071 [==========>...................] - ETA: 24s
## 63414272/170498071 [==========>...................] - ETA: 24s
## 63741952/170498071 [==========>...................] - ETA: 24s
## 64020480/170498071 [==========>...................] - ETA: 24s
## 64282624/170498071 [==========>...................] - ETA: 24s
## 64430080/170498071 [==========>...................] - ETA: 24s
## 64790528/170498071 [==========>...................] - ETA: 24s
## 65150976/170498071 [==========>...................] - ETA: 24s
## 65495040/170498071 [==========>...................] - ETA: 24s
## 65658880/170498071 [==========>...................] - ETA: 24s
## 65904640/170498071 [==========>...................] - ETA: 24s
## 66232320/170498071 [==========>...................] - ETA: 24s
## 66527232/170498071 [==========>...................] - ETA: 23s
## 66854912/170498071 [==========>...................] - ETA: 23s
## 67002368/170498071 [==========>...................] - ETA: 23s
## 67297280/170498071 [==========>...................] - ETA: 23s
## 67624960/170498071 [==========>...................] - ETA: 23s
## 67969024/170498071 [==========>...................] - ETA: 23s
## 68198400/170498071 [==========>...................] - ETA: 23s
## 68411392/170498071 [===========>..................] - ETA: 23s
## 68722688/170498071 [===========>..................] - ETA: 23s
## 69066752/170498071 [===========>..................] - ETA: 23s
## 69427200/170498071 [===========>..................] - ETA: 23s
## 69558272/170498071 [===========>..................] - ETA: 23s
## 69771264/170498071 [===========>..................] - ETA: 23s
## 70066176/170498071 [===========>..................] - ETA: 22s
## 70361088/170498071 [===========>..................] - ETA: 22s
## 70721536/170498071 [===========>..................] - ETA: 22s
## 70918144/170498071 [===========>..................] - ETA: 22s
## 71163904/170498071 [===========>..................] - ETA: 22s
## 71409664/170498071 [===========>..................] - ETA: 22s
## 71720960/170498071 [===========>..................] - ETA: 22s
## 72065024/170498071 [===========>..................] - ETA: 22s
## 72228864/170498071 [===========>..................] - ETA: 22s
## 72376320/170498071 [===========>..................] - ETA: 22s
## 72704000/170498071 [===========>..................] - ETA: 22s
## 73031680/170498071 [===========>..................] - ETA: 22s
## 73375744/170498071 [===========>..................] - ETA: 22s
## 73637888/170498071 [===========>..................] - ETA: 21s
## 73768960/170498071 [===========>..................] - ETA: 21s
## 73998336/170498071 [============>.................] - ETA: 21s
## 74309632/170498071 [============>.................] - ETA: 21s
## 74702848/170498071 [============>.................] - ETA: 21s
## 74964992/170498071 [============>.................] - ETA: 21s
## 75210752/170498071 [============>.................] - ETA: 21s
## 75522048/170498071 [============>.................] - ETA: 21s
## 75849728/170498071 [============>.................] - ETA: 21s
## 76144640/170498071 [============>.................] - ETA: 21s
## 76341248/170498071 [============>.................] - ETA: 21s
## 76587008/170498071 [============>.................] - ETA: 21s
## 76947456/170498071 [============>.................] - ETA: 21s
## 77307904/170498071 [============>.................] - ETA: 20s
## 77586432/170498071 [============>.................] - ETA: 20s
## 77848576/170498071 [============>.................] - ETA: 20s
## 78028800/170498071 [============>.................] - ETA: 20s
## 78389248/170498071 [============>.................] - ETA: 20s
## 78749696/170498071 [============>.................] - ETA: 20s
## 79011840/170498071 [============>.................] - ETA: 20s
## 79224832/170498071 [============>.................] - ETA: 20s
## 79437824/170498071 [============>.................] - ETA: 20s
## 79781888/170498071 [=============>................] - ETA: 20s
## 80142336/170498071 [=============>................] - ETA: 20s
## 80420864/170498071 [=============>................] - ETA: 20s
## 80617472/170498071 [=============>................] - ETA: 20s
## 80928768/170498071 [=============>................] - ETA: 20s
## 81256448/170498071 [=============>................] - ETA: 19s
## 81616896/170498071 [=============>................] - ETA: 19s
## 81928192/170498071 [=============>................] - ETA: 19s
## 82059264/170498071 [=============>................] - ETA: 19s
## 82370560/170498071 [=============>................] - ETA: 19s
## 82616320/170498071 [=============>................] - ETA: 19s
## 82960384/170498071 [=============>................] - ETA: 19s
## 83304448/170498071 [=============>................] - ETA: 19s
## 83517440/170498071 [=============>................] - ETA: 19s
## 83763200/170498071 [=============>................] - ETA: 19s
## 84074496/170498071 [=============>................] - ETA: 19s
## 84369408/170498071 [=============>................] - ETA: 19s
## 84713472/170498071 [=============>................] - ETA: 19s
## 84992000/170498071 [=============>................] - ETA: 18s
## 85221376/170498071 [=============>................] - ETA: 18s
## 85467136/170498071 [==============>...............] - ETA: 18s
## 85794816/170498071 [==============>...............] - ETA: 18s
## 86138880/170498071 [==============>...............] - ETA: 18s
## 86417408/170498071 [==============>...............] - ETA: 18s
## 86687744/170498071 [==============>...............] - ETA: 18s
## 86925312/170498071 [==============>...............] - ETA: 18s
## 87252992/170498071 [==============>...............] - ETA: 18s
## 87597056/170498071 [==============>...............] - ETA: 18s
## 87875584/170498071 [==============>...............] - ETA: 18s
## 88154112/170498071 [==============>...............] - ETA: 18s
## 88399872/170498071 [==============>...............] - ETA: 18s
## 88727552/170498071 [==============>...............] - ETA: 17s
## 89055232/170498071 [==============>...............] - ETA: 17s
## 89350144/170498071 [==============>...............] - ETA: 17s
## 89645056/170498071 [==============>...............] - ETA: 17s
## 89890816/170498071 [==============>...............] - ETA: 17s
## 90169344/170498071 [==============>...............] - ETA: 17s
## 90513408/170498071 [==============>...............] - ETA: 17s
## 90873856/170498071 [==============>...............] - ETA: 17s
## 91168768/170498071 [===============>..............] - ETA: 17s
## 91414528/170498071 [===============>..............] - ETA: 17s
## 91693056/170498071 [===============>..............] - ETA: 17s
## 92020736/170498071 [===============>..............] - ETA: 17s
## 92348416/170498071 [===============>..............] - ETA: 17s
## 92659712/170498071 [===============>..............] - ETA: 16s
## 92954624/170498071 [===============>..............] - ETA: 16s
## 93249536/170498071 [===============>..............] - ETA: 16s
## 93528064/170498071 [===============>..............] - ETA: 16s
## 93872128/170498071 [===============>..............] - ETA: 16s
## 94232576/170498071 [===============>..............] - ETA: 16s
## 94494720/170498071 [===============>..............] - ETA: 16s
## 94838784/170498071 [===============>..............] - ETA: 16s
## 95019008/170498071 [===============>..............] - ETA: 16s
## 95346688/170498071 [===============>..............] - ETA: 16s
## 95690752/170498071 [===============>..............] - ETA: 16s
## 96034816/170498071 [===============>..............] - ETA: 16s
## 96411648/170498071 [===============>..............] - ETA: 15s
## 96624640/170498071 [================>.............] - ETA: 15s
## 96985088/170498071 [================>.............] - ETA: 15s
## 97312768/170498071 [================>.............] - ETA: 15s
## 97656832/170498071 [================>.............] - ETA: 15s
## 98000896/170498071 [================>.............] - ETA: 15s
## 98263040/170498071 [================>.............] - ETA: 15s
## 98557952/170498071 [================>.............] - ETA: 15s
## 98918400/170498071 [================>.............] - ETA: 15s
## 99262464/170498071 [================>.............] - ETA: 15s
## 99622912/170498071 [================>.............] - ETA: 15s
## 99917824/170498071 [================>.............] - ETA: 15s
## 100212736/170498071 [================>.............] - ETA: 15s
## 100540416/170498071 [================>.............] - ETA: 14s
## 100900864/170498071 [================>.............] - ETA: 14s
## 101261312/170498071 [================>.............] - ETA: 14s
## 101539840/170498071 [================>.............] - ETA: 14s
## 101883904/170498071 [================>.............] - ETA: 14s
## 102162432/170498071 [================>.............] - ETA: 14s
## 102522880/170498071 [=================>............] - ETA: 14s
## 102883328/170498071 [=================>............] - ETA: 14s
## 103227392/170498071 [=================>............] - ETA: 14s
## 103538688/170498071 [=================>............] - ETA: 14s
## 103882752/170498071 [=================>............] - ETA: 14s
## 104194048/170498071 [=================>............] - ETA: 14s
## 104521728/170498071 [=================>............] - ETA: 13s
## 104882176/170498071 [=================>............] - ETA: 13s
## 105226240/170498071 [=================>............] - ETA: 13s
## 105570304/170498071 [=================>............] - ETA: 13s
## 105914368/170498071 [=================>............] - ETA: 13s
## 106274816/170498071 [=================>............] - ETA: 13s
## 106586112/170498071 [=================>............] - ETA: 13s
## 106962944/170498071 [=================>............] - ETA: 13s
## 107307008/170498071 [=================>............] - ETA: 13s
## 107651072/170498071 [=================>............] - ETA: 13s
## 108093440/170498071 [==================>...........] - ETA: 13s
## 109142016/170498071 [==================>...........] - ETA: 12s
## 109600768/170498071 [==================>...........] - ETA: 12s
## 109862912/170498071 [==================>...........] - ETA: 12s
## 111337472/170498071 [==================>...........] - ETA: 12s
## 111566848/170498071 [==================>...........] - ETA: 12s
## 111878144/170498071 [==================>...........] - ETA: 12s
## 112058368/170498071 [==================>...........] - ETA: 12s
## 112254976/170498071 [==================>...........] - ETA: 12s
## 112402432/170498071 [==================>...........] - ETA: 12s
## 112566272/170498071 [==================>...........] - ETA: 12s
## 112828416/170498071 [==================>...........] - ETA: 12s
## 113025024/170498071 [==================>...........] - ETA: 12s
## 113188864/170498071 [==================>...........] - ETA: 11s
## 113336320/170498071 [==================>...........] - ETA: 11s
## 113532928/170498071 [==================>...........] - ETA: 11s
## 113795072/170498071 [===================>..........] - ETA: 11s
## 113991680/170498071 [===================>..........] - ETA: 11s
## 114122752/170498071 [===================>..........] - ETA: 11s
## 114319360/170498071 [===================>..........] - ETA: 11s
## 114647040/170498071 [===================>..........] - ETA: 11s
## 114843648/170498071 [===================>..........] - ETA: 11s
## 115007488/170498071 [===================>..........] - ETA: 11s
## 115236864/170498071 [===================>..........] - ETA: 11s
## 115417088/170498071 [===================>..........] - ETA: 11s
## 115761152/170498071 [===================>..........] - ETA: 11s
## 115924992/170498071 [===================>..........] - ETA: 11s
## 116121600/170498071 [===================>..........] - ETA: 11s
## 116269056/170498071 [===================>..........] - ETA: 11s
## 116580352/170498071 [===================>..........] - ETA: 11s
## 116826112/170498071 [===================>..........] - ETA: 11s
## 116940800/170498071 [===================>..........] - ETA: 11s
## 117137408/170498071 [===================>..........] - ETA: 11s
## 117301248/170498071 [===================>..........] - ETA: 11s
## 117612544/170498071 [===================>..........] - ETA: 11s
## 117841920/170498071 [===================>..........] - ETA: 11s
## 118022144/170498071 [===================>..........] - ETA: 11s
## 118185984/170498071 [===================>..........] - ETA: 11s
## 118464512/170498071 [===================>..........] - ETA: 11s
## 118759424/170498071 [===================>..........] - ETA: 10s
## 118874112/170498071 [===================>..........] - ETA: 10s
## 119070720/170498071 [===================>..........] - ETA: 10s
## 119234560/170498071 [===================>..........] - ETA: 10s
## 119545856/170498071 [====================>.........] - ETA: 10s
## 119791616/170498071 [====================>.........] - ETA: 10s
## 119971840/170498071 [====================>.........] - ETA: 10s
## 120152064/170498071 [====================>.........] - ETA: 10s
## 120414208/170498071 [====================>.........] - ETA: 10s
## 120692736/170498071 [====================>.........] - ETA: 10s
## 120889344/170498071 [====================>.........] - ETA: 10s
## 121069568/170498071 [====================>.........] - ETA: 10s
## 121249792/170498071 [====================>.........] - ETA: 10s
## 121446400/170498071 [====================>.........] - ETA: 10s
## 121790464/170498071 [====================>.........] - ETA: 10s
## 121987072/170498071 [====================>.........] - ETA: 10s
## 122134528/170498071 [====================>.........] - ETA: 10s
## 122413056/170498071 [====================>.........] - ETA: 10s
## 122593280/170498071 [====================>.........] - ETA: 10s
## 122888192/170498071 [====================>.........] - ETA: 10s
## 123084800/170498071 [====================>.........] - ETA: 10s
## 123248640/170498071 [====================>.........] - ETA: 10s
## 123461632/170498071 [====================>.........] - ETA: 10s
## 123691008/170498071 [====================>.........] - ETA: 9s
## 124002304/170498071 [====================>.........] - ETA: 9s
## 124166144/170498071 [====================>.........] - ETA: 9s
## 124428288/170498071 [====================>.........] - ETA: 9s
## 124592128/170498071 [====================>.........] - ETA: 9s
## 124919808/170498071 [====================>.........] - ETA: 9s
## 125149184/170498071 [=====================>........] - ETA: 9s
## 125296640/170498071 [=====================>........] - ETA: 9s
## 125542400/170498071 [=====================>........] - ETA: 9s
## 125689856/170498071 [=====================>........] - ETA: 9s
## 126050304/170498071 [=====================>........] - ETA: 9s
## 126246912/170498071 [=====================>........] - ETA: 9s
## 126427136/170498071 [=====================>........] - ETA: 9s
## 126640128/170498071 [=====================>........] - ETA: 9s
## 126787584/170498071 [=====================>........] - ETA: 9s
## 127098880/170498071 [=====================>........] - ETA: 9s
## 127311872/170498071 [=====================>........] - ETA: 9s
## 127524864/170498071 [=====================>........] - ETA: 9s
## 127721472/170498071 [=====================>........] - ETA: 9s
## 127852544/170498071 [=====================>........] - ETA: 9s
## 128212992/170498071 [=====================>........] - ETA: 9s
## 128425984/170498071 [=====================>........] - ETA: 8s
## 128671744/170498071 [=====================>........] - ETA: 8s
## 128819200/170498071 [=====================>........] - ETA: 8s
## 128999424/170498071 [=====================>........] - ETA: 8s
## 129359872/170498071 [=====================>........] - ETA: 8s
## 129556480/170498071 [=====================>........] - ETA: 8s
## 129769472/170498071 [=====================>........] - ETA: 8s
## 129933312/170498071 [=====================>........] - ETA: 8s
## 130129920/170498071 [=====================>........] - ETA: 8s
## 130416640/170498071 [=====================>........] - ETA: 8s
## 130719744/170498071 [======================>.......] - ETA: 8s
## 130899968/170498071 [======================>.......] - ETA: 8s
## 131031040/170498071 [======================>.......] - ETA: 8s
## 131325952/170498071 [======================>.......] - ETA: 8s
## 131604480/170498071 [======================>.......] - ETA: 8s
## 131866624/170498071 [======================>.......] - ETA: 8s
## 132030464/170498071 [======================>.......] - ETA: 8s
## 132177920/170498071 [======================>.......] - ETA: 8s
## 132440064/170498071 [======================>.......] - ETA: 8s
## 132767744/170498071 [======================>.......] - ETA: 8s
## 132980736/170498071 [======================>.......] - ETA: 8s
## 133128192/170498071 [======================>.......] - ETA: 8s
## 133373952/170498071 [======================>.......] - ETA: 7s
## 133668864/170498071 [======================>.......] - ETA: 7s
## 133980160/170498071 [======================>.......] - ETA: 7s
## 134144000/170498071 [======================>.......] - ETA: 7s
## 134275072/170498071 [======================>.......] - ETA: 7s
## 134553600/170498071 [======================>.......] - ETA: 7s
## 134848512/170498071 [======================>.......] - ETA: 7s
## 135110656/170498071 [======================>.......] - ETA: 7s
## 135241728/170498071 [======================>.......] - ETA: 7s
## 135487488/170498071 [======================>.......] - ETA: 7s
## 135766016/170498071 [======================>.......] - ETA: 7s
## 136060928/170498071 [======================>.......] - ETA: 7s
## 136257536/170498071 [======================>.......] - ETA: 7s
## 136388608/170498071 [======================>.......] - ETA: 7s
## 136650752/170498071 [=======================>......] - ETA: 7s
## 136880128/170498071 [=======================>......] - ETA: 7s
## 137191424/170498071 [=======================>......] - ETA: 7s
## 137338880/170498071 [=======================>......] - ETA: 7s
## 137519104/170498071 [=======================>......] - ETA: 7s
## 137863168/170498071 [=======================>......] - ETA: 7s
## 138092544/170498071 [=======================>......] - ETA: 6s
## 138371072/170498071 [=======================>......] - ETA: 6s
## 138485760/170498071 [=======================>......] - ETA: 6s
## 138698752/170498071 [=======================>......] - ETA: 6s
## 138977280/170498071 [=======================>......] - ETA: 6s
## 139272192/170498071 [=======================>......] - ETA: 6s
## 139452416/170498071 [=======================>......] - ETA: 6s
## 139632640/170498071 [=======================>......] - ETA: 6s
## 139812864/170498071 [=======================>......] - ETA: 6s
## 140156928/170498071 [=======================>......] - ETA: 6s
## 140419072/170498071 [=======================>......] - ETA: 6s
## 140582912/170498071 [=======================>......] - ETA: 6s
## 140779520/170498071 [=======================>......] - ETA: 6s
## 141058048/170498071 [=======================>......] - ETA: 6s
## 141369344/170498071 [=======================>......] - ETA: 6s
## 141582336/170498071 [=======================>......] - ETA: 6s
## 141795328/170498071 [=======================>......] - ETA: 6s
## 141893632/170498071 [=======================>......] - ETA: 6s
## 142254080/170498071 [========================>.....] - ETA: 6s
## 142581760/170498071 [========================>.....] - ETA: 6s
## 142761984/170498071 [========================>.....] - ETA: 5s
## 142909440/170498071 [========================>.....] - ETA: 5s
## 143089664/170498071 [========================>.....] - ETA: 5s
## 143417344/170498071 [========================>.....] - ETA: 5s
## 143695872/170498071 [========================>.....] - ETA: 5s
## 143908864/170498071 [========================>.....] - ETA: 5s
## 144007168/170498071 [========================>.....] - ETA: 5s
## 144261120/170498071 [========================>.....] - ETA: 5s
## 144580608/170498071 [========================>.....] - ETA: 5s
## 144859136/170498071 [========================>.....] - ETA: 5s
## 144990208/170498071 [========================>.....] - ETA: 5s
## 145121280/170498071 [========================>.....] - ETA: 5s
## 145448960/170498071 [========================>.....] - ETA: 5s
## 145727488/170498071 [========================>.....] - ETA: 5s
## 146006016/170498071 [========================>.....] - ETA: 5s
## 146104320/170498071 [========================>.....] - ETA: 5s
## 146284544/170498071 [========================>.....] - ETA: 5s
## 146546688/170498071 [========================>.....] - ETA: 5s
## 146874368/170498071 [========================>.....] - ETA: 5s
## 147103744/170498071 [========================>.....] - ETA: 5s
## 147234816/170498071 [========================>.....] - ETA: 5s
## 147431424/170498071 [========================>.....] - ETA: 4s
## 147742720/170498071 [========================>.....] - ETA: 4s
## 148103168/170498071 [=========================>....] - ETA: 4s
## 148217856/170498071 [=========================>....] - ETA: 4s
## 148398080/170498071 [=========================>....] - ETA: 4s
## 148594688/170498071 [=========================>....] - ETA: 4s
## 148905984/170498071 [=========================>....] - ETA: 4s
## 149217280/170498071 [=========================>....] - ETA: 4s
## 149348352/170498071 [=========================>....] - ETA: 4s
## 149512192/170498071 [=========================>....] - ETA: 4s
## 149807104/170498071 [=========================>....] - ETA: 4s
## 150134784/170498071 [=========================>....] - ETA: 4s
## 150331392/170498071 [=========================>....] - ETA: 4s
## 150511616/170498071 [=========================>....] - ETA: 4s
## 150708224/170498071 [=========================>....] - ETA: 4s
## 151003136/170498071 [=========================>....] - ETA: 4s
## 151330816/170498071 [=========================>....] - ETA: 4s
## 151461888/170498071 [=========================>....] - ETA: 4s
## 151642112/170498071 [=========================>....] - ETA: 4s
## 151920640/170498071 [=========================>....] - ETA: 4s
## 152215552/170498071 [=========================>....] - ETA: 3s
## 152461312/170498071 [=========================>....] - ETA: 3s
## 152641536/170498071 [=========================>....] - ETA: 3s
## 152821760/170498071 [=========================>....] - ETA: 3s
## 153051136/170498071 [=========================>....] - ETA: 3s
## 153313280/170498071 [=========================>....] - ETA: 3s
## 153575424/170498071 [==========================>...] - ETA: 3s
## 153772032/170498071 [==========================>...] - ETA: 3s
## 153985024/170498071 [==========================>...] - ETA: 3s
## 154214400/170498071 [==========================>...] - ETA: 3s
## 154574848/170498071 [==========================>...] - ETA: 3s
## 154755072/170498071 [==========================>...] - ETA: 3s
## 154951680/170498071 [==========================>...] - ETA: 3s
## 155181056/170498071 [==========================>...] - ETA: 3s
## 155328512/170498071 [==========================>...] - ETA: 3s
## 155688960/170498071 [==========================>...] - ETA: 3s
## 155918336/170498071 [==========================>...] - ETA: 3s
## 156131328/170498071 [==========================>...] - ETA: 3s
## 156344320/170498071 [==========================>...] - ETA: 3s
## 156573696/170498071 [==========================>...] - ETA: 3s
## 156868608/170498071 [==========================>...] - ETA: 2s
## 157048832/170498071 [==========================>...] - ETA: 2s
## 157261824/170498071 [==========================>...] - ETA: 2s
## 157458432/170498071 [==========================>...] - ETA: 2s
## 157638656/170498071 [==========================>...] - ETA: 2s
## 157966336/170498071 [==========================>...] - ETA: 2s
## 158244864/170498071 [==========================>...] - ETA: 2s
## 158441472/170498071 [==========================>...] - ETA: 2s
## 158605312/170498071 [==========================>...] - ETA: 2s
## 158900224/170498071 [==========================>...] - ETA: 2s
## 159260672/170498071 [===========================>..] - ETA: 2s
## 159473664/170498071 [===========================>..] - ETA: 2s
## 159653888/170498071 [===========================>..] - ETA: 2s
## 159735808/170498071 [===========================>..] - ETA: 2s
## 160096256/170498071 [===========================>..] - ETA: 2s
## 160456704/170498071 [===========================>..] - ETA: 2s
## 160669696/170498071 [===========================>..] - ETA: 2s
## 160800768/170498071 [===========================>..] - ETA: 2s
## 160948224/170498071 [===========================>..] - ETA: 2s
## 161275904/170498071 [===========================>..] - ETA: 2s
## 161603584/170498071 [===========================>..] - ETA: 1s
## 161849344/170498071 [===========================>..] - ETA: 1s
## 161964032/170498071 [===========================>..] - ETA: 1s
## 162177024/170498071 [===========================>..] - ETA: 1s
## 162521088/170498071 [===========================>..] - ETA: 1s
## 162848768/170498071 [===========================>..] - ETA: 1s
## 163045376/170498071 [===========================>..] - ETA: 1s
## 163143680/170498071 [===========================>..] - ETA: 1s
## 163471360/170498071 [===========================>..] - ETA: 1s
## 163815424/170498071 [===========================>..] - ETA: 1s
## 164126720/170498071 [===========================>..] - ETA: 1s
## 164208640/170498071 [===========================>..] - ETA: 1s
## 164356096/170498071 [===========================>..] - ETA: 1s
## 164700160/170498071 [===========================>..] - ETA: 1s
## 165011456/170498071 [============================>.] - ETA: 1s
## 165322752/170498071 [============================>.] - ETA: 1s
## 165421056/170498071 [============================>.] - ETA: 1s
## 165683200/170498071 [============================>.] - ETA: 1s
## 165994496/170498071 [============================>.] - ETA: 0s
## 166305792/170498071 [============================>.] - ETA: 0s
## 166518784/170498071 [============================>.] - ETA: 0s
## 166649856/170498071 [============================>.] - ETA: 0s
## 166961152/170498071 [============================>.] - ETA: 0s
## 167272448/170498071 [============================>.] - ETA: 0s
## 167600128/170498071 [============================>.] - ETA: 0s
## 167731200/170498071 [============================>.] - ETA: 0s
## 167911424/170498071 [============================>.] - ETA: 0s
## 168206336/170498071 [============================>.] - ETA: 0s
## 168468480/170498071 [============================>.] - ETA: 0s
## 168828928/170498071 [============================>.] - ETA: 0s
## 168960000/170498071 [============================>.] - ETA: 0s
## 169205760/170498071 [============================>.] - ETA: 0s
## 169451520/170498071 [============================>.] - ETA: 0s
## 169762816/170498071 [============================>.] - ETA: 0s
## 170090496/170498071 [============================>.] - ETA: 0s
## 170237952/170498071 [============================>.] - ETA: 0s
## 170500096/170498071 [==============================] - 37s 0us/steptrain_images, test_images = train_images / 255.0 , test_images / 255.0This time we load the cifat10 dataset with the load_data method. We also normalize this data immediately after that, by dividing all values by 255. Since we are dealing with RGB values, and all values lie in between 0 and 255, we end up with values in between 0 and 1.
Next, we define the possible class names in a list, so that we can label the final numerical results later on. The neural network will again produce a softmax result, which means that we will use the argmax function, to figure out the class name.
class_names = [ 'Plane' , 'Car' , 'Bird' , 'Cat' , 'Deer' ,
'Dog' , 'Frog' , 'Horse' , 'Ship' , 'Truck' ]Now we can visualize a section of the data, to see what this dataset looks like.
For this we run a for loop with 16 iterations and create a 4x4 grid of subplots. The x-ticks and the y-ticks will be set to empty lists, so that we don’t have annoying coordinates. After that, we use the imshow method, to visualize the individual images. The label of the image will then be the respective class name.
This dataset contains a lot of images.
Here for example we only use the first 20,000 of the training images and the first 4,000 of the test images. Of course your model will be way more accurate if you use all the images. However, for weak computers this might take forever.
BUILDING NEURAL NETWORK
Now that we have prepared our data, we can start building the neural network.
model = models.Sequential()
model.add(layers.Conv2D( 32 , ( 3 , 3 ), activation = 'relu' ,
input_shape =( 32 , 32 , 3 )))
model.add(layers.MaxPooling2D(( 2 , 2 )))
model.add(layers.Conv2D( 64 , ( 3 , 3 ), activation = 'relu' ))
model.add(layers.MaxPooling2D(( 2 , 2 )))
model.add(layers.Conv2D( 64 , ( 3 , 3 ), activation = 'relu' ))
model.add(layers.Flatten())
model.add(layers.Dense( 64 , activation = 'relu' ))
model.add(layers.Dense( 10 , activation = 'softmax' ))Here we again define a Sequential model. Our inputs go directly into a convolutional layer (Conv2D ). This layer has 32 filters or channels in the shape of 3x3 matrices. The activation function is the ReLU function, which we already know and the input shape is 32x32x3. This is because we our images have a resolution of 32x32 pixels and three layers because of the RGB colors. The result is then forwarded into a MaxPooling2D layer that simplifies the output. Then the simplified output is again forwarded into the next convolutional layer. After that into another max-pooling layer and into another convolutional layer. This result is then being flattened by the Flatten layer, which means that it is transformed into a one-dimensional vector format. Then we forward the results into one dense hidden layer before it finally comes to the softmax output layer. There we find the final classification probabilities.
TRAINING AND TESTING
Now we are almost done. We just need to train and test the model before we can use it.
model.compile( optimizer = 'adam' ,
loss = 'sparse_categorical_crossentropy' ,
metrics =[ 'accuracy' ])Here we again use the adam optimizer and the sparse categorical crossentropy loss function.
model.fit(train_images,
train_labels,
epochs = 10 ,
validation_data =(test_images, test_labels))We now train our model on our training data in ten epochs. Remember: This means that our model is going to see the same data ten times over and over again.
test_loss, test_acc = model.evaluate(test_images,
test_labels,
verbose = 2 )We use the evaluate function to test our model and get the loss and accuracy values. We set the parameter verbose to 2, so that we get as much information as possible.
- 1s - loss: 0.8139 - acc: 0.7090
CLASSIFYING OWN IMAGES
However, the interesting part starts now. Since our model is trained, we can now go ahead and use our own images of cars, planes, horses etc. for classification.
The important thing is that we get these images down to 32x32 pixels because this is the required input format of our model. For this you can use any software like Gimp or Paint. You can either crop the images or scale them.
Now we just have to load these images into our script, using OpenCV.
img1 = cv.imread( 'car.jpg' )
img1 = cv.cvtColor(img1, cv.COLOR_BGR2RGB)
img2 = cv.imread( 'horse.jpg' )
img2 = cv.cvtColor(img2, cv.COLOR_BGR2RGB)
plt.imshow(img1, cmap =plt.cm.binary)
plt.show()The function imread loads the image into our script. Then we use the cvtColor method, in order to change the default color scheme of BGR (blue, green, red) to RGB (red, green, blue).
plt.imshow(img1, cmap =plt.cm.binary)
plt.show()With the imshow function, we can show the image in our script, using Matplotlib.
We can now use the loaded images as the input for our model, in order to get a prediction.
prediction = model.predict(np.array([img1]) / 255 )
index = np.argmax(prediction)
print (class_names[index])First we use the predict function to get the softmax result. Notice that we are converting our image into a NumPy array and dividing it by 255. This is because we need to normalize it, since our model was trained on normalized values. Then we use the argmax function to get the index of the highest softmax activation value. Finally, we print the class name of that index as a result.
Car Horse
The results speak for themselves. These pictures were classified absolutely correct.
