hooks/pyramid_attention_broadcast: fix redundant recompute at iteration 0 and free stale cache when outside timestep range#13467

GitGlimpse895 wants to merge 1 commit intohuggingface:mainhuggingface/diffusers:mainfrom

GitGlimpse895:fix/pab-cache-logicGitGlimpse895/diffusers:fix/pab-cache-logicCopy head branch name to clipboard