vllm.logits_process ¶
Attributes:
-
LogitsProcessor(TypeAlias) –LogitsProcessor is a function that takes a list
LogitsProcessor = Callable[[list[int], torch.Tensor], torch.Tensor] | Callable[[list[int], list[int], torch.Tensor], torch.Tensor] module-attribute ¶
LogitsProcessor is a function that takes a list of previously generated tokens, the logits tensor for the next token and, optionally, prompt tokens as a first argument, and returns a modified tensor of logits to sample from.