Skip to content

vllm.v1.worker.gpu.model_states.interface

Classes:

ModelSpecificAttnMetadata

Base class for model-specific attention metadata.

Source code in vllm/v1/worker/gpu/model_states/interface.py 
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
class ModelSpecificAttnMetadata:
    """Base class for model-specific attention metadata."""

    def get_extra_common_attn_kwargs(
        self,
        kv_cache_group_id: int,
        num_reqs: int,
    ) -> dict[str, Any]:
        return {}

    def get_extra_attn_kwargs(
        self,
        attn_metadata_builder: Any,
        num_reqs: int,
    ) -> dict[str, Any]:
        return {}