scorable.annotations

Classes

Annotations

Annotations API.

Module Contents

class scorable.annotations.Annotations(client_context: scorable.utils.ClientContextCallable)

Annotations API.

An annotation is a human label attached to exactly one target - a dataset item or an execution log - using a score config. For binary/categorical configs pass category (the label); for continuous configs pass value. Omitting score_config_id defaults to the global identity “Score” config, so a raw score can be labelled with just value.

Note

The construction of the API instance should be handled by accessing an attribute of a scorable.client.Scorable instance.

Parameters:

client_context (scorable.utils.ClientContextCallable)

async acreate(*, dataset_item_id: str | None = None, execution_log_id: str | None = None, value: float | None = None, category: str | None = None, rationale: str = '', status: str = 'published', score_config_id: str | None = None, _request_timeout: int | None = None, _client: scorable.generated.openapi_aclient.ApiClient) → scorable.generated.openapi_aclient.models.annotation.Annotation

Asynchronously create an annotation.

Parameters:

  • dataset_item_id (Optional[str])

  • execution_log_id (Optional[str])

  • value (Optional[float])

  • category (Optional[str])

  • rationale (str)

  • status (str)

  • score_config_id (Optional[str])

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_aclient.ApiClient)

Return type:

scorable.generated.openapi_aclient.models.annotation.Annotation

async adelete(annotation_id: str, *, _request_timeout: int | None = None, _client: scorable.generated.openapi_aclient.ApiClient) → None

Asynchronously delete an annotation.

Parameters:

  • annotation_id (str)

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_aclient.ApiClient)

Return type:

None

async aget(annotation_id: str, *, _request_timeout: int | None = None, _client: scorable.generated.openapi_aclient.ApiClient) → scorable.generated.openapi_aclient.models.annotation.Annotation

Asynchronously get an annotation by ID.

Parameters:

  • annotation_id (str)

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_aclient.ApiClient)

Return type:

scorable.generated.openapi_aclient.models.annotation.Annotation

async alist(*, dataset: str | None = None, score_config: str | None = None, status: str | None = None, dataset_item: str | None = None, execution_log: str | None = None, limit: int = 100) → AsyncIterator[scorable.generated.openapi_aclient.models.annotation.Annotation]

Asynchronously iterate through annotations.

Parameters:

  • dataset (Optional[str])

  • score_config (Optional[str])

  • status (Optional[str])

  • dataset_item (Optional[str])

  • execution_log (Optional[str])

  • limit (int)

Return type:

AsyncIterator[scorable.generated.openapi_aclient.models.annotation.Annotation]

async aupdate(annotation_id: str, *, value: float | None = None, category: str | None = None, rationale: str | None = None, status: str | None = None, _request_timeout: int | None = None, _client: scorable.generated.openapi_aclient.ApiClient) → scorable.generated.openapi_aclient.models.annotation.Annotation

Asynchronously update an annotation.

Parameters:

  • annotation_id (str)

  • value (Optional[float])

  • category (Optional[str])

  • rationale (Optional[str])

  • status (Optional[str])

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_aclient.ApiClient)

Return type:

scorable.generated.openapi_aclient.models.annotation.Annotation

create(*, dataset_item_id: str | None = None, execution_log_id: str | None = None, value: float | None = None, category: str | None = None, rationale: str = '', status: str = 'published', score_config_id: str | None = None, _request_timeout: int | None = None, _client: scorable.generated.openapi_client.ApiClient) → scorable.generated.openapi_client.models.annotation.Annotation

Create an annotation.

Parameters:

  • dataset_item_id (Optional[str]) – The dataset item to annotate (mutually exclusive with execution_log_id).

  • execution_log_id (Optional[str]) – The execution log to annotate (mutually exclusive with dataset_item_id).

  • value (Optional[float]) – The score for continuous configs.

  • category (Optional[str]) – The label for binary/categorical configs.

  • rationale (str) – Optional free-text justification.

  • status (str) – draft or published (default published).

  • score_config_id (Optional[str]) – The score config; defaults to the global identity “Score” config.

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_client.ApiClient)

Return type:

scorable.generated.openapi_client.models.annotation.Annotation

delete(annotation_id: str, *, _request_timeout: int | None = None, _client: scorable.generated.openapi_client.ApiClient) → None

Delete an annotation.

Parameters:

  • annotation_id (str)

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_client.ApiClient)

Return type:

None

get(annotation_id: str, *, _request_timeout: int | None = None, _client: scorable.generated.openapi_client.ApiClient) → scorable.generated.openapi_client.models.annotation.Annotation

Get an annotation by ID.

Parameters:

  • annotation_id (str)

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_client.ApiClient)

Return type:

scorable.generated.openapi_client.models.annotation.Annotation

list(*, dataset: str | None = None, score_config: str | None = None, status: str | None = None, dataset_item: str | None = None, execution_log: str | None = None, limit: int = 100, _client: scorable.generated.openapi_client.ApiClient) → Iterator[scorable.generated.openapi_client.models.annotation.Annotation]

Iterate through annotations.

Parameters:

  • dataset (Optional[str]) – Filter by the dataset the annotated items belong to.

  • score_config (Optional[str]) – Filter by score config id.

  • status (Optional[str]) – Filter by draft or published.

  • dataset_item (Optional[str]) – Filter by dataset item id.

  • execution_log (Optional[str]) – Filter by execution log id.

  • limit (int) – Number of entries to iterate through at most.

  • _client (scorable.generated.openapi_client.ApiClient)

Return type:

Iterator[scorable.generated.openapi_client.models.annotation.Annotation]

update(annotation_id: str, *, value: float | None = None, category: str | None = None, rationale: str | None = None, status: str | None = None, _request_timeout: int | None = None, _client: scorable.generated.openapi_client.ApiClient) → scorable.generated.openapi_client.models.annotation.Annotation

Update an annotation. The target is immutable; value/category are re-derived.

Parameters:

  • annotation_id (str)

  • value (Optional[float])

  • category (Optional[str])

  • rationale (Optional[str])

  • status (Optional[str])

  • _request_timeout (Optional[int])

  • _client (scorable.generated.openapi_client.ApiClient)

Return type:

scorable.generated.openapi_client.models.annotation.Annotation

client_context