Actually, I plan to do a machine translation project with this seq2seq module. Before that, I just did a simple test and got a very bad result. I don't know where goes wrong. Pls help me...
Here's the process:
#1. traning set
def generate_sequence(length, n_unique):
return [randint(1, n_unique-1) for _ in range(length)]
x = np.array(generate_sequence(100000,100)).reshape(10000,10)
y = np.array(generate_sequence(50000,100)).reshape(10000,5)
x_encoder_input_data = to_categorical(x)
y_decoder_target_data = to_categorical(y)
#x_encoder_input_data.shape = (10000, 10, 100)
#10000 training data, x_input_length=10,x_input_dim=100
#y_decoder_target_data.shape = (10000, 5, 100)
#2. building&training model
model = SimpleSeq2Seq(input_dim=100,
input_length=10,
output_dim=100,
hidden_dim=128,
output_length=5)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])
model.fit(x_encoder_input_data, y_decoder_target_data, batch_size=128, epochs=500)
#3. the partial losses
Epoch 1/500
10000/10000 - 44s 4ms/step - loss: 7.2350 - acc: 0.0108
Epoch 2/500
10000/10000 - 20s 2ms/step - loss: 6.9500 - acc: 0.0104
Epoch 3/500
10000/10000 - 22s 2ms/step - loss: 10.7297 - acc: 0.0103
Epoch 4/500
10000/10000 - 20s 2ms/step - loss: 8.6834 - acc: 0.0096
Epoch 5/500
10000/10000 - 21s 2ms/step - loss: 8.4943 - acc: 0.0099
Epoch 6/500
10000/10000 - 19s 2ms/step - loss: 8.4487 - acc: 0.0100
Epoch 7/500
10000/10000 - 19s 2ms/step - loss: 8.6318 - acc: 0.0099
Epoch 8/500
10000/10000 - 18s 2ms/step - loss: 8.5765 - acc: 0.0099
Epoch 9/500
10000/10000 - 20s 2ms/step - loss: 8.4753 - acc: 0.0099
Epoch 10/500
10000/10000 - 20s 2ms/step - loss: 8.3738 - acc: 0.0099
Epoch 11/500
10000/10000 - 20s 2ms/step - loss: 8.3999 - acc: 0.0098
Epoch 12/500
10000/10000 - 19s 2ms/step - loss: 8.3108 - acc: 0.0099
Epoch 13/500
10000/10000 - 19s 2ms/step - loss: 8.3457 - acc: 0.0099
Epoch 14/500
10000/10000 - 19s 2ms/step - loss: 8.4852 - acc: 0.0098
Epoch 15/500
10000/10000 - 18s 2ms/step - loss: 8.4749 - acc: 0.0099
Epoch 16/500
10000/10000 - 18s 2ms/step - loss: 8.5881 - acc: 0.0098
Epoch 17/500
10000/10000 - 19s 2ms/step - loss: 8.3868 - acc: 0.0099
Epoch 18/500
10000/10000 - 21s 2ms/step - loss: 8.2499 - acc: 0.0098
Epoch 19/500
10000/10000 - 20s 2ms/step - loss: 8.4659 - acc: 0.0099
Epoch 20/500
10000/10000 - 20s 2ms/step - loss: 7.8421 - acc: 0.0099
Epoch 21/500
10000/10000 - 21s 2ms/step - loss: 7.6197 - acc: 0.0099
Epoch 22/500
10000/10000 - 20s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 23/500
10000/10000 - 19s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 24/500
10000/10000 - 21s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 25/500
10000/10000 - 19s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 26/500
10000/10000 - 22s 2ms/step - loss: 7.6193 - acc: 0.0099
Epoch 27/500
10000/10000 - 22s 2ms/step - loss: 7.6193 - acc: 0.0099
··· ···
#4. predicting
for seq_index in range(6):
predictions = model.predict(x_encoder_input_data[seq_index:seq_index+1])
predicted_list=[]
for prediction_vector in predictions:
for pred in prediction_vector:
next_token = np.argmax(pred)
predicted_list.append(next_token)
print('-')
print('Input sentence:', X[seq_index])
print('Decoded sentence:', predicted_list)
print('Target sentence:', y[seq_index])
#5. the predicting results:
-
Input sentence: [28, 2, 46, 12, 21, 6] # x
Decoded sentence: [78, 78, 78, 78, 66] # y_predict
Target sentence: [82 22 82 41 27] # y
-
Input sentence: [12, 20, 45, 28, 18, 42]
Decoded sentence: [78, 78, 66, 66, 66]
Target sentence: [43 36 30 13 64]
-
Input sentence: [3, 43, 45, 4, 33, 27]
Decoded sentence: [78, 78, 66, 66, 66]
Target sentence: [90 20 56 23 32]
-
Input sentence: [34, 50, 21, 20, 11, 6]
Decoded sentence: [78, 78, 78, 78, 66]
Target sentence: [27 57 50 57 81]
-
Input sentence: [47, 42, 14, 2, 31, 6]
Decoded sentence: [78, 78, 78, 78, 66]
Target sentence: [77 94 47 26 67]
-
Input sentence: [20, 24, 34, 31, 37, 25]
Decoded sentence: [78, 78, 66, 66, 66]
Target sentence: [11 48 99 67 66]
Actually, I plan to do a machine translation project with this seq2seq module. Before that, I just did a simple test and got a very bad result. I don't know where goes wrong. Pls help me...
Here's the process:
#1. traning set
#2. building&training model
#3. the partial losses
#4. predicting
#5. the predicting results: