Model catalog

Client for the models.dev catalog, providing per-model context and output token limits with an in-memory + on-disk cache and graceful fallback when the catalog is unavailable.

Client for the models.dev model catalog.

Fetches https://models.dev/api.json (a ~3MB JSON mapping of provider -> model -> properties) and exposes the per-model token limits used to bound LLM output token reservations. This is needed because some providers (e.g. HuggingFace) reserve the whole context window as output when no max-token parameter is set, which leads to spurious usage limits and rate limiting.

The catalog is fetched lazily on first use, kept in memory for the process lifetime (lru_cache), and mirrored to an on-disk cache ({user_cache_dir}/klea/models-dev.json) with a one-day TTL so that restarts do not need to re-download it.

Providers without a catalog entry (local ollama, unknown custom endpoints) and models missing from the catalog resolve to None so callers can fall back gracefully instead of failing.

File: klea_utils/models_catalog.py

Copyright 2026 Ankur Sinha Author: Ankur Sinha <sanjay DOT ankur AT gmail DOT com>

klea_utils.models_catalog.DEFAULT_MODELS_DEV_URL = 'https://models.dev/api.json'

Default models.dev catalog URL. Overridable via the KLEA_MODELS_DEV_URL environment variable (e.g. for offline mirrors or enterprise proxies).

klea_utils.models_catalog.DISK_CACHE_FILE = 'models-dev.json'

File name of the on-disk catalog cache, under the platformdirs cache dir.

klea_utils.models_catalog.DISK_CACHE_TTL_SECONDS = 86400

Time-to-live for the on-disk catalog cache, in seconds (1 day).

klea_utils.models_catalog.FETCH_TIMEOUT_SECONDS = 15.0

this is only hit on the first use after the disk cache expires.

Type:

HTTP timeout for fetching the catalog. Short

class klea_utils.models_catalog.ModelLimits(context: int | None = None, input: int | None = None, output: int | None = None)[source]

Bases: NamedTuple

Token limits for a single model from the catalog.

All fields are optional: the catalog always carries context and output, while input is only defined for a subset of models.

context: int | None

Alias for field number 0

input: int | None

Alias for field number 1

output: int | None

Alias for field number 2

klea_utils.models_catalog.get_model_context_limit(provider: str, model_name: str) → int | None[source]

Return the context window size for a provider + model.

Convenience wrapper around get_model_limits().

Parameters:

  • provider – Klea provider id.

  • model_name – Model identifier.

Returns:

The model’s limit.context, or None if unknown.

klea_utils.models_catalog.get_model_limits(provider: str, model_name: str) → ModelLimits | None[source]

Return the token limits for a provider + model, or None.

Returns None when the provider has no catalog entry (local ollama, custom endpoints), the model is missing from the catalog, or the catalog could not be fetched. Callers should treat None as “no information available”.

Parameters:

  • provider – Klea provider id (e.g. "huggingface").

  • model_name – Model identifier, e.g. "gpt-4o" or "Qwen/Qwen3-Coder-30B-A3B-Instruct".

Returns:

ModelLimits with whatever fields the catalog defines.

klea_utils.models_catalog.get_model_output_limit(provider: str, model_name: str) → int | None[source]

Return the max output token limit for a provider + model.

Convenience wrapper around get_model_limits().

Parameters:

  • provider – Klea provider id.

  • model_name – Model identifier.

Returns:

The model’s limit.output, or None if unknown.