Skip to content

vllm.ir.util

Functions:

  • hash_source –

    Utility method to hash the sources of functions or objects.

  • weak_cache –

    Simple weak equivalent to functools.cache

  • weak_lru_cache –

    LRU Cache decorator that keeps a weak reference to 'self'.

hash_source(*srcs)

Utility method to hash the sources of functions or objects. Args: srcs: strings or objects to add to the hash. Objects and functions have their source inspected.

Source code in vllm/ir/util.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
def hash_source(*srcs: str | Any) -> str:
    """
    Utility method to hash the sources of functions or objects.
    Args:
        srcs: strings or objects to add to the hash.
            Objects and functions have their source inspected.
    """
    hasher = hashlib.sha256()
    for src in srcs:
        if src is None:
            src_str = "None"
        elif isinstance(src, str):
            src_str = src
        elif isinstance(src, Path):
            src_str = src.read_text()
        elif isinstance(src, (types.FunctionType, type)):
            src_str = inspect.getsource(src)
        else:
            # object instance
            src_str = inspect.getsource(src.__class__)
        hasher.update(src_str.encode("utf-8"))
    return hasher.hexdigest()

weak_cache(user_function)

Simple weak equivalent to functools.cache

Source code in vllm/ir/util.py 
59
60
61
def weak_cache(user_function, /):
    """Simple weak equivalent to functools.cache"""
    return weak_lru_cache(maxsize=None)(user_function)

weak_lru_cache(maxsize=128, typed=False)

LRU Cache decorator that keeps a weak reference to 'self'. This avoids memory leakage, which happens when functools.lru_cache stores a reference to self in the global cache.

Taken from: https://stackoverflow.com/a/68052994/5082708

Source code in vllm/ir/util.py 
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
def weak_lru_cache(maxsize: int | None = 128, typed: bool = False):
    """
    LRU Cache decorator that keeps a weak reference to 'self'.
    This avoids memory leakage, which happens when functools.lru_cache
    stores a reference to self in the global cache.

    Taken from: https://stackoverflow.com/a/68052994/5082708
    """

    def wrapper(func):
        @functools.lru_cache(maxsize, typed)
        def _func(_self, *args, **kwargs):
            return func(_self(), *args, **kwargs)

        @functools.wraps(func)
        def inner(self, *args, **kwargs):
            return _func(weakref.ref(self), *args, **kwargs)

        return inner

    return wrapper