model_to_max_input_tokens = { "google/flan-t5-xxl": 8192, "google/flan-t5-xl": 8192, "google/flan-t5-large": 8192, "google/flan-t5-base": 8192, "google/flan-t5-small": 8192, "google/flan-ul2": 8192, "bigscience/T0pp": 8192, }

[zhenghuawang6]

These models cannot support such a long context length for inference. Moreover, in the logic of the process_model_input function, should some space be reserved for the inference length to prevent exceeding the context window?