Skip to content

vllm.renderers.inputs.tokenize

Schemas and utilities for tokenization inputs.

Classes:

Attributes:

DecoderOnlyTokPrompt = TokensPrompt | EmbedsPrompt module-attribute

A DecoderOnlyDictPrompt that has been tokenized.

DecoderTokPrompt = TokensPrompt module-attribute

A DecoderDictPrompt that has been tokenized.

EncoderTokPrompt = TokensPrompt module-attribute

A EncoderDictPrompt that has been tokenized.

SingletonTokPrompt = DecoderOnlyTokPrompt | EncoderTokPrompt | DecoderTokPrompt module-attribute

A SingletonDictPrompt that has been tokenized.

TokPrompt = DecoderOnlyTokPrompt | EncoderDecoderTokPrompt module-attribute

A DictPrompt that has been tokenized.

EncoderDecoderTokPrompt

Bases: TypedDict

A EncoderDecoderDictPrompt that has been tokenized.

Source code in vllm/renderers/inputs/tokenize.py 
32
33
34
35
36
37
38
39
40
41
class EncoderDecoderTokPrompt(TypedDict):
    """
    A
    [`EncoderDecoderDictPrompt`][vllm.renderers.inputs.preprocess.EncoderDecoderDictPrompt]
    that has been tokenized.
    """

    encoder_prompt: EncoderTokPrompt

    decoder_prompt: DecoderTokPrompt | None