RuntimeError: selected index k out of range

[L-JunJie]

query_indices = (k_states_query_token_L1_norm.topk(pivot_text_token).indices + image_token_start_index + image_token_length).tolist()

RuntimeError: selected index k out of range

[ZichenWen1]

Could you share your experimental setup and experiment scripts with me?

[L-JunJie]

CUDA_VISIBLE_DEVICES=0 bash scripts/v1_5/eval/vqav2.sh 0.778 128

vqav2.sh file as below：

#!/bin/bash

export CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7

gpu_list="${CUDA_VISIBLE_DEVICES:-0}"
IFS=',' read -ra GPULIST <<< "$gpu_list"

CHUNKS=${#GPULIST[@]}

CKPT="llava-v1.5-13b"
MODEL='VLM/llava-v1.5-7b'

reduction_ratio=$1
max_num_trunction=$2
SPLIT="llava_vqav2_mscoco_test-dev2015"

for IDX in $(seq 0 $((CHUNKS-1))); do
CUDA_VISIBLE_DEVICES=${GPULIST[$IDX]} python -m llava.eval.model_vqa_loader
--model-path $MODEL
--question-file ./playground/data/eval/vqav2/$SPLIT.jsonl
--image-folder ./playground/data/eval/vqav2/test2015
--answers-file ./playground/data/eval/vqav2/answers/$SPLIT/$CKPT/${CHUNKS}_${IDX}.jsonl
--num-chunks $CHUNKS
--chunk-idx $IDX
--temperature 0
--conv-mode vicuna_v1
--sparse
--attn_implementation sdpa
--pruned_layer 2
--image_token_start_index 35
--image_token_length 576
--reduction_ratio $reduction_ratio
--max_num_trunction $max_num_trunction
--pivot_image_token 4
--pivot_text_token 4 &
done

wait

output_file=./playground/data/eval/vqav2/answers/$SPLIT/$CKPT/merge.jsonl

Clear out the output file if it exists.

"$output_file"

Loop through the indices and concatenate each file.

for IDX in $(seq 0 $((CHUNKS-1))); do
cat ./playground/data/eval/vqav2/answers/$SPLIT/$CKPT/${CHUNKS}_${IDX}.jsonl >> "$output_file"
done

python scripts/convert_vqav2_for_submission.py --split $SPLIT --ckpt $CKPT

[L-JunJie]

I tried to print the shape of k_states_query_token_L1_norm, but found that k_states_query_token_L1_norm.shape: torch.Size([0]).

[Sugar929]

@L-JunJie have you solved this problem?