[WIP] Implement 2nd pass training using 1-best decoding results from the 1st pass network

[csukuangfj]

It implements #106 (comment)

BTW, since it seems this is hard to get to work, if you feel like it you could work on a simpler idea.
In training time we'd take the best-path alignment and using some kind of RNN or masked attention
we'd predict the next label in that best-path. (We'd probably take in the output of the 1st network
as an input to that). The sequence length here is the same as the original sequence length.
In test time the way this would work at least initially, is we'd run this on n-best lists obtained from the
1st-pass decoding and use the scores to decide which of the n-best paths to keep. There are more accurate
decoding methods we could look into later.

---

The training objf is decreasing and seems to be converging. Will post the decoding results later.

[csukuangfj]

TODO(fangjun): Use cross attention here

• query: x2
• key and value: padded_acoustics

and masked self-attention

• key, query, and value: x2

[csukuangfj]

common2.py is the same as common.py, except that it has some code supporting
the second pass model. To avoid conflicts with the master, a new file is used.

The same goes for the following xxx2.py files, e.g., lm_rescore2.py, mmi2.py.

[csukuangfj]

@danpovey

This file implements the Nbest class proposed in
#232 (comment)
Please have a review if it matches the proposal.

[danpovey]

That's great! Yes it looks like what I had in mind.
I assume you would separate it from this PR though? Or maybe even submit it to k2? Since there's a lot going on here.

[csukuangfj]

Will move it to k2.