Can you explain why batch_input size ?

[bikachu]

Hello,

I am a beginner to neural networks. Can you explain why
batch_input_ = tf.transpose(batch_input, perm=[2, 0, 1])
X = tf.transpose(batch_input_) ?
I know this is equivalent to

X = tf.transpose(batch_input, perm = [1,0,2]

but why do we need to reshape batch_input size and can you explain more about batch input size?

Thank you

[bikachu]

This is in the vanilla RNN

[kanthagirish-rit]

Hello @bikachu,

@KnHuq is using tf.scan() to iterate through time steps in the data and compute hidden state. If you take a look at the method vanilla_rnn() of the class, you would notice hidden states being computed for a single time step. This method is passed to tf.scan() as the function.

The first argument in method definition is previous_hidden_state h(t-1). tf.scan() initializes this with initial_hidden which is created in the __init__(). The computed current_hidden_state is h(t).

Second argument x, which would take the shape (# samples, input_dim) is unwrapped by tf.scan() from self.processed_input (time_step, # samples, input_dim) along the first dimension, hence he has reshaped the data to have the first dimension as the time step. vanilla_rnn() would be run for each x, which is the time step and hidden states are computed.

He is considering each column of pixels in an MNIST image as one time step.