### Prerequisites - [X] I have tried updating UtilsRL to the newest version. - [X] I have checked both open and closed issues but found nothing related to my request. ### UtilsRL verison when proposing this request 0.3.13 ### What I am expecting Full support for RNN backend. Currently (v0.3.13) in UtilsRL, actors and critics are fed with `backend` which is typed as `nn.Module`, and theoretically this can decouple feature extraction and decision making. However, when using RNN to extract features, actors/critics often require extra inputs apart from the RNN's output. For example, in MAPLE, RNN output $o_t$ is computed with previous state-actions and the actor needs both $s_t$ and $o_t$ as input. We need to unify the design when the type of `backend` varies among (possibly) MLP, CNN, RNN. ### Possible solutions 1. check the output dimension of `backend` and compare it against the input of `output_layer`. If these two don't match, we concatenate extra inputs (such as $s_t$) to the output of `backend` along dimension -1. 2. create another class like RecurrentActor and RecurrentCritic, like what tianshou did. ### Any additional messages which might help _No response_ ### Urgency Urgent, will bring significant improvement and should be considered in next main version.