Generative-Adversarial-Networks

Generative Adversarial Networks for Generating MNIST digits.

Before explaining the Project, I would like to give a brief introduction about GAN's

What is GAN's ?

Generative Adversarial Networks, or GANs are a model architecture for training a generative model, and it is most common to use deep learning models in this architecture. Yann LeCun described it as “the most interesting idea in the last 10 years in Machine Learning”

GANs are generative models: they create new data instances that resemble your training data. For example, GANs can create images that look like photographs of human faces, even though the faces don't belong to any real person just like the Monalisa.

Here are some fake images generated by High End GAN's

GAN Architecture

GAN basically contains two networks a Generator, a Discriminator competining against each other.

• The generator network is trained to fool the discriminator, it wants to output data that looks as close as possible to real, training data.
• The discriminator is a classifier that is trained to figure out which data is real and which is fake.

The general structure of a GAN is shown in the diagram above, using MNIST images as data. The latent sample is a random vector that the generator uses to construct its fake images.This latent vector come from the normal distribution. This is often called a latent vector and that vector space is called latent space. As the generator trains, it figures out how to map latent vectors to recognizable images that can fool the discriminator.

Explaining my GAN network for MNIST digits Generation.

Neccessary import for this model:

* Tensoflow
* keras
* Numpy

For this network i would recommend to GPU for faster training

Defining the input shape

img_rows = 28
img_cols = 28
channels = 1    # Gray Scale Images
img_shape = (img_rows,img_cols,channels)

Generator Model

The input of the generator will be noise of shape (100,1) from the Normal distribution and it will output an image of same size as that of training data.

model.add(Dense(256,input_shape=noise_shape))
model.add(LeakyReLU(alpha=0.2))
model.add(BatchNormalization(momentum=0.8))

The generator model had the Dense layer with Leaky ReLU as the activation function to avoid the problem of dead ReLU or dying ReLU The output of generator model is 28*28 = 786 pixels I had used tanh activation function to our output layer because generator has been found to perform the best with 𝑡𝑎𝑛ℎ for the generator output, which scales the output to be between -1 and 1, instead of 0 and 1.

Discriminator Model

The input of the discriminator is an image and the output is the validity, the likelihood of the image being real.OR Simply Binary Classification.

model= Sequential()
model.add(Flatten(input_shape=img_shape))
model.add(Dense(512))
model.add(LeakyReLU(alpha=0.2))
model.add(Dense(256))
model.add(LeakyReLU(alpha=0.2))
model.add(Dense(1,activation='sigmoid'))
model.summary()

Training Phase

After creating both the generator model and the discriminator model now is the time to squeeze them together and start Generator Vs Discriminator training.

To start the training, first of all we would re-scaling our training images in between -1 to 1 and setting the ground truths.

x_train = (x_train.astype(np.float32)-127.5)/127.5

After that as the shape of the Mnist images is of (28 x 28), we would expand it to (28 x 28 x 1) by adding a channel to match it as per our model.

x_train = np.expand_dims(x_train,axis=3)

Discriminator Training

Selecting images from the training sample of size equal to half of the batch size.

idx = np.random.randint(0, X_train.shape[0], half_batch)
imgs = X_train[idx]

For the discriminator to classify we would require some fake images, for that we will use the generator model( although initally it would be not trained ) by using noise from the numpy normal distribution for half of the batch size images.

noise = np.random.normal(0,1,(half_batch,100)) 
gen_imgs = generator.predict(noise)

Now as we have the fake images. We would train our discriminator on the real images and fake images seperately and get the loss. One thing to note that we are labeling our real images as 1 and fake images as 0. After that averging the loss from real and fake data.

d_loss_real = discriminator.train_on_batch(imgs,np.ones((half_batch,1)))
d_loss_fake = discriminator.train_on_batch(gen_imgs,np.zeros((half_batch,1)))
d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

Generator Training

For training the generator within the same epoch we had defined combined model (stacked generator and discriminator) takes noise as input => generates images => determines validity. One important point to note here is that we are generating the fake images using the generator and labeling them as real images and them passing it through the disctiminator to classify them as real or fake. after this the discriminator will gives us his own classification and we will calculate the loss according to the images that we had labled as real but discriminator correctely predicted it as fake.

noise = np.random.normal(0,1,(batch_size,100))
valid_y = np.array([1]*batch_size)
g_loss = combined.train_on_batch(noise,valid_y)

The Conclusion...

EPOCH #0	EPOCH #400

EPOCH #1000	EPOCH #5000

EPOCH #10000	EPOCH #15000

EPOCH #85000	EPOCH #99400