Fix implicit padding in split_between_processes when apply_padding=False and num_samples < num_processes

[3manifold]

What does this PR do?

Function split_between_processes applies implicit padding even if apply_padding=False. This results to redundant calculations during distributed inference since the same inference (i.e. prompt) is called twice etc. Apart from wasting resources, this can cause conflicts among competing processes, for instance, disk I/O.

The same bug is duplicated across two code branches of the same function. Both stem from the identical incorrect assumption — "a process always receives at least one element" — and both produce identical wrong behaviour (silent padding when apply_padding=False). The only difference is syntax:

• Bug 1 (list/Tensor): guards with result = inputs[-1:]
• Bug 2 (Dataset): guards with start_index = len(inputs) - 1

How to reproduce

Example with 1 sample and 2 GPUs

Process	start_index	end_index	Normal slice
0	0	1	inputs[0:1] → [sample_0] ✓
1	1	1	start_index >= len(inputs) triggers! → inputs[-1:] = [sample_0] ✗

Process 1's start_index (1) equals len(inputs) (1), triggering the guard. Instead of returning an empty slice, it silently returns the last element — regardless of apply_padding.

def _split_values(inputs, start_index, end_index):
    if isinstance(inputs, (list, tuple, torch.Tensor)):
        if start_index >= len(inputs):      
            result = inputs[-1:]              # ← always returns last element
        else:
            result = inputs[start_index:end_index]
        if apply_padding:
            ...  

resolves #4053

Who can review?

@BenjaminBossan @SunMarc

[3manifold]

Python slicing returns [] naturally for out-of-bounds, which is the correct behavior when apply_padding=False.

[3manifold]

Use inputs[-1] instead of result[-1]: result may be empty when this process received no samples.

[3manifold]

Clamp both indices to valid range without artificially inflating start_index to len-1 when apply_padding is False.

[3manifold]

Pad using the last real element (len(inputs)-1), not end_index-1 which can be wrong when result_idcs is empty.