Thread per-frame `frame_times` through the VLM eval pipeline

[amazloumi]

What

The per-frame timestamp embedding (FrameTimeEmbedding) is threaded through training — scripts/train.py passes frame_times into VLMWrapper.forward — but the VLM eval pipeline does not thread it. A video checkpoint trained with a time embedding is therefore evaluated without its learned temporal signal.

The drop is silent: VLMWrapper.forward ignores the temporal signal when frame_times is None and raises no error (see the inference-contract note in its docstring, kempnerforge/model/vlm.py). In the eval path:

• kempnerforge/eval/vlm/adapter.py::_render_request decodes (frames, times) but discards times.
• the batch-stacking path builds frame_mask but never builds a (B, F) frame_times tensor.
• _generate_batch calls model(pixel_values, input_ids, frame_mask=frame_mask) with no frame_times, so it defaults to None. `

Because time_embedding.type defaults to "sinusoidal" (on), most video checkpoints are affected — a train/eval mismatch that quietly degrades / misrepresents eval numbers.

Scope

eval/vlm/adapter.py only:

• _render_request: surface the decoded per-frame times alongside frames.
• batch construction (where frames_to_clip_tensor / frame_mask are built and stacked): build and stack frame_times to (B, F).
• _generate_batch: add a frame_times parameter and pass it to model(...).
• Tests (tests/unit/eval/vlm/test_adapter.py): assert frame_times is built with the right shape and passed into the model forward (spy on the forward call).

No training-side or config changes.

Backward compatibility

• Image checkpoints, and video checkpoints trained with time_embedding.type = "none": unchanged — the model ignores frame_times regardless.
• Video checkpoints trained with a time embedding (the default): this fixes a silent mismatch so eval uses the same temporal signal as training. Eval outputs change for these checkpoints, but toward train/eval parity. No config defaults change.