Function Cacher API

This page contains the autodoc for the main Function Cacher user-facing APIs. The two most important functions are the Cacher constructor and the .get function.

class functionCacher.Cacher.Cacher(cachePath: Path | str | None = None, cacheFolder: str = 'caches', shouldCache: bool = True, compressionType: CompressionType = CompressionType.ZSTD, maxCacheAge: timedelta = datetime.timedelta(days=1), createCacheFolder: bool = True, deleteOutdatedCacheFile: bool = True, deleteThrowingCacheFile: bool = True, reformatSourceOnGenerateHash: bool = True, num_workers: int = 1, name: str | None = None, hmac_cache: bool = True, hmac_key: bytes = b'fancy-hmac-key', **kwargs)

Note on reformatSourceOnGenerateHash: Used if Cacher.get is invoked with parameter generate_cache_id set (default: True). If set, then the Cacher will automatically generate a fitting cache ID. If hash_source_code is also set (default: True), then the source code of the cached function is also included in the cache ID; If the source code of the cached function changes between cache attempts, then the cache will automatically fail. If reformatSourceOnGenerateHash is set (default: True), then the source code is reformatted using autopep8. This protects against false cache misses if e.g. a newline was added or removed to the cached source code.

Parameters:

  • cachePath – Path to cache folder (default: tempfile.gettempdir() / $USER / cacheFolder)

  • cacheFolder – Folder for caches (default: caches)

  • shouldCache – Whether to activate the cache (default: True)

  • compressionType – Compression Type (default: DEFAULT == ZSTD)

  • maxCacheAge – Age threshold (default: 24h)

  • createCacheFolder – Whether to create cache folder if not exists (default: True)

  • deleteOutdatedCacheFile – Whether to delete outdated cache file (age(cache) > maxCacheAge) (default: True)

  • deleteThrowingCacheFile – Whether to delete cache file causing an Exception during read (default: True)

  • reformatSourceOnGenerateHash – Whether to reformat source code of cached function during cache ID generation (see notes) (default: True)

  • num_workers – Number of workers for compression library (only respected for CompressionType.ZSTD) (default: 1)

  • name – Optional cacher name, currently used for logger name (default: None)

  • hmac_cache – Whether to HMAC the cache file (default: True)

  • hmac_key – HMAC key (default: b"fancy-hmac-key")

  • kwargs – key-word arguments, currently used for backwards-compatibility

cache(func=None, **ckwargs)

Use as decorator for function with @self.cache. Any parameters set in ckwargs are passed to underlying .get call.

Parameters:

func – function to be cached - this is automatically handled by the decorator mechanism.

cacheExists(cacheId: str) → bool

Check whether a cache file for cacheId exists.

Parameters:

cacheId – ID of cache to check

Returns:

True if exists

get(callback, args: tuple | None = None, kwargs: dict | None = None, cacheId: str | None = None, generate_cache_id: bool = True, cache_response_callback: Callable | None = None, exclude_kwargs: list[str] | None = None, exclude_args: list[int] | None = None, additional_cache_id_data: str | None = None, slow_cache_path: bool = False, hash_source_code: bool = True, **ckwargs) → Any

Try to get a cached function call result. If successful, return the cache file. Otherwise, invoke callback(*args, **kwargs), cache the result, and return the data.

Either provide a unique cacheId used for lookup, or set generate_cache_id, which will automatically derive a cacheId based on the callback and args/kwargs. Use exclude_{args, kwargs} to remove unstable arguments from automatic caching, such as database connection metadata.

Note on slow_cache_path: In some scenarios, the caching (i.e. pickling) process might change the underlying data in subtle ways. This is caused by some data structure serialisers/deserialisers acting weird (e.g. pandas). In case you encounter this issue, set slow_cache_path: On cache miss, this executes the callback(*args, **kwargs), caches (i.e. pickle) the result, then loads that result back from disk, and return this value. As opposed to the intuitive optimisation of returning the callback result immediately, this approach will always return the same type of data, despite any serialisers acting funny.

Parameters:

  • cacheId – Cache ID for saving cache files (needs to be unique for each cache)

  • callback – Function to call if cache file does not exist

  • args – Tuple with non-keyworded arguments to callback

  • kwargs – Dictionary containing all necessary callback function parameters

  • generate_cache_id – Whether to generate hash based on function name, arguments, and source code if hash_source_code is set (default: True)

  • cache_response_callback – Callable with signature cache_response_callback(bool), called with True on cache hit, False on cache miss

  • exclude_kwargs – List of kwargs of callback to exclude while caching

  • exclude_args – List of indices into args of callback to exclude

  • additional_cache_id_data – Additional data considered while generating the automatic cache ID. This could be data not present

  • slow_cache_path – If set and on cache miss: execute callback, cache results, read cached results, return cached results (see notes) (default: False)

  • hash_source_code – Whether to include hash of cached function’s source code in cache ID (default: True)

  • ckwargs – Key-worded args for internal cacher-functionality, currently unused

Returns:

Result from callback or cache file

invalidate(cacheId: str)

Invalidate / remove cache for a given cacheId.

Parameters:

cacheId – cache ID to invalidate / remove

class functionCacher.CacheEntry.CacheEntry(file_path: Path, timestamp: datetime, compression_type: CompressionType, hmac: bytes | None)

CacheEntry stores metadata for cache entry.

Parameters:

  • file_path – Stores location of cache on FS

  • timestamp – Stores last modification time of cache entry

  • compression_type – Stores compression type

  • hmac – Optionally stores hmac, if enabled in Cacher

class functionCacher.CompressionType.CompressionType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

CompressionType stores the currently used cache file compression algorithm.

NONE = 1
LZMA = 2
ZSTD = 3
DEFAULT = 3