scorable.client

Classes

Beta	Beta API features namespace
Scorable	Scorable API Python client.

Module Contents

class scorable.client.Beta(get_client_context: Callable[[], AsyncContextManager[scorable.generated.openapi_aclient.ApiClient]] | Callable[[], ContextManager[scorable.generated.openapi_client.ApiClient]])

Beta API features namespace

Parameters:

get_client_context (Union[Callable[[], AsyncContextManager[scorable.generated.openapi_aclient.ApiClient]], Callable[[], ContextManager[scorable.generated.openapi_client.ApiClient]]])

property judges: scorable.judges.Judges

Get Judges API (Beta)

Return type:

scorable.judges.Judges

class scorable.client.Scorable(api_key: str | None = None, *, run_async: bool = False, _api_client: scorable.generated.openapi_aclient.ApiClient | None | scorable.generated.openapi_client.ApiClient | None = None, base_url: str | None = None)

Scorable API Python client.

The API key must be provided via one of the following methods - the code uses the first one that is found:

1. as an argument to Scorable constructor,

2. environment variable SCORABLE_API_KEY, or

3. .env file containing SCORABLE_API_KEY=

Parameters:

• api_key (Optional[str]) – Scorable API Key (if not provided from environment)

• run_async (bool) – Whether to run the API client asynchronously

• _api_client (Union[Optional[scorable.generated.openapi_aclient.ApiClient], Optional[scorable.generated.openapi_client.ApiClient]])

• base_url (Optional[str])

api_key = None

base_url = None

property beta: Beta

Get Beta API features

Return type:

Beta

property datasets: scorable.datasets.DataSets

Get DataSets API

Return type:

scorable.datasets.DataSets

property evaluators: scorable.skills.Evaluators

Get Evaluators API

Return type:

scorable.skills.Evaluators

property execution_logs: scorable.execution_logs.ExecutionLogs

Get Execution Logs API

Return type:

scorable.execution_logs.ExecutionLogs

property get_client_context: Callable[[], AsyncContextManager[scorable.generated.openapi_aclient.ApiClient]] | Callable[[], ContextManager[scorable.generated.openapi_client.ApiClient]]

Return type:

Union[Callable[[], AsyncContextManager[scorable.generated.openapi_aclient.ApiClient]], Callable[[], ContextManager[scorable.generated.openapi_client.ApiClient]]]

property judges: scorable.judges.Judges

Get Judges API

Return type:

scorable.judges.Judges

property models: scorable.models.Models

Get Models API

Return type:

scorable.models.Models

property objectives: scorable.objectives.Objectives

Get Objectives API

Return type:

scorable.objectives.Objectives

run_async = False