Sampler API and execution settings

Start with the Sampler guide for authoring and composition.

Public definitions and results

chatsnack.Sampler(*, name=None, data=None, questions=None, params=None, expand=True, model=_UNSET, timeout=_UNSET, retry=_UNSET, base_url=_UNSET, api_key_env=_UNSET)

Bases: Asset

An authored evaluation definition; ask always returns a resolved Sample.

Keep construction in memory, with convenient authored execution options.

yaml property

Inspect authored YAML without requiring a name or writing a file.

lookup_root property

Find sibling collections from an explicit path or the bound root stash.

model property writable

Convenient view of the authored Jev model setting.

load(path=None)

Load explicitly, preserving native snapclass collection behavior.

authored(*, include_name=True)

Return readable authored values without resolving their live fillings.

save(path=None)

Persist only names that the saved-result filling grammar can address.

from_sample(sample, *, name=None) classmethod

Reconstruct the evaluated inputs literally, independent of live assets.

ask(question=_UNSET, **kwargs)

Evaluate inline or saved questions and return a Sample in every case.

ask_a(question=_UNSET, *, questions=_UNSET, data=_UNSET, model=_UNSET, timeout=_UNSET, retry=_UNSET, base_url=_UNSET, api_key_env=_UNSET, **fillings) async

Async evaluation; override values belong only to this invocation.

close()

Close the retained sync provider client opened by this Sampler.

close_a() async

Close the retained sync provider client from async code.

compile(question=_UNSET, **kwargs)

Inspect a request; executable dependency fillings may still run.

compile_a(question=_UNSET, *, questions=_UNSET, data=_UNSET, model=_UNSET, timeout=_UNSET, retry=_UNSET, base_url=_UNSET, api_key_env=_UNSET, **fillings) async

Resolve and validate without evaluating this Sampler itself.

chatsnack.Question

Bases: Asset

A reusable judgment; choices or levels select its kind implicitly.

yaml property

Inspect authored YAML without requiring a name or writing a file.

lookup_root property

Find sibling collections from an explicit path or the bound root stash.

kind property

Infer the provider primitive without adding a discriminator to YAML.

load(path=None)

Load explicitly, preserving native snapclass collection behavior.

authored(*, include_name=True)

Return the authored definition, retaining structured criteria and order.

save(path=None)

Persist only names that remain addressable as assets and result keys.

compile()

Validate and translate one resolved question into Jev vocabulary.

score_levels()

Keep stable authored names alongside Jev's ordered level descriptions.

chatsnack.Sample(data, questions, answers, model, usage)

Resolved evaluation inputs and results, always in authored question order.

Keep results outside persistence tracking when used as new Sampler data.

question property

The first resolved question, also available as questions[0].

answer property

The first answer, also available as answers[0], even in a batch.

to_dict()

Project a prior Sample as ordinary structured evaluation data.

Execution settings

model, timeout, retry, base_url, and api_key_env can be authored in SamplerParams, passed conveniently to the constructor, or overridden for one call. Precedence is call override → saved params → SDK/environment defaults.

sampler = Sampler(data="popcorn", model="jev-latest")
sample = sampler.ask("Crunchy?", timeout=20, retry={"max_retries": 0})

sampler.model is backed by sampler.params.model. Unset models follow TYPESAFE_DEFAULT_MODEL, then jev-latest; the result retains the reported model. Model aliases can move. Pin a provider version for calibrated applications.

params:
  model: jev-latest
  timeout: 20
  retry:
    max_retries: 3

timeout is an HTTP-operation timeout in seconds. retry supports the SDK's serializable fields: max_retries, backoff_initial, backoff_max, backoff_jitter, http_statuses, respect_retry_after, api_connection_error, api_timeout_error, and its total retry-budget timeout. Callables and exception classes are excluded. The SDK owns retry execution.

api_key_env names a credential variable; credentials are never saved. Without it, the SDK uses TYPESAFE_API_KEY. base_url selects an alternate endpoint; otherwise the SDK honors TYPESAFE_BASE_URL and its default endpoint.

Client lifetime

Like Chat, a reusable Sampler opens its synchronous provider client lazily and retains it across ask() calls. Call sampler.close() when a long-running synchronous application shuts down. ask_a() creates and closes a scoped async client for each evaluation so event-loop-bound transports are never retained on the Sampler. close_a() remains the async-compatible way to close any retained sync client when code uses both execution styles.

Changing model for one call reuses the existing connection because the model is part of the request. Call-time timeout, retry, base_url, or api_key_env settings use a scoped client for that call. They do not replace the Sampler's authored synchronous connection. If authored connection settings change after a synchronous evaluation, close the Sampler before evaluating with the new settings.

Request inspection

compile() and compile_a() return the resolved state, model, and questions request. They do not evaluate that Sampler, but executable dependency fillings can make provider calls during resolution. Ordinary .yaml inspection preserves live authored references and does not execute them.

Structured instructions, criteria, and data remain structured. Choice lists compile to null descriptions; Score level names are retained locally while the ordered descriptions go to Jev.

Explicit Question names must be nonempty strings without the filling metacharacters ., [, ], {, }, !, or :. Those characters delimit parts of named result fillings or have special meaning to the filling formatter. Saved Sampler names similarly exclude {, }, !, and :; dots and brackets remain valid in the asset-name segment.

Advanced resolver

from chatsnack import resolve_fillings_a

values = await resolve_fillings_a(
    ["sampler.SnackCheck.crunchy.score"],
    variables={"snack": "popcorn"},
    allow_sampler=True,
)
score = values["sampler"]["SnackCheck.crunchy.score"]

allow_sampler=False is the default for this public resolver. allow_chat independently authorizes direct/transitive Chat fillings. Ordinary Chat.ask() and Sampler.ask() need no authority flags. question.Name resolution returns a typed Question. Explicit namespace overrides retain the resolver's opaque-value behavior. Resolver-scoped Sampler work has a 16-evaluation limit; repeated reads of one evaluation count once. Existing depth and expansion bounds still apply.

Answer details and replay

Yes/no ties select yes with confidence zero. Choice preserves the provider's selection even if another probability is larger. Score selects the first maximal probability in authored level order. Provider numeric values are not rewritten.

from_sample() records expand: false to keep resolved inputs literal across save/load. Ordinary callers need no expansion controls. Samples hold results in memory; this release does not introduce independently saved Sample assets.

Saving a follow-up Sampler(data=sample) automatically quotes literal braces in the projected data. Reloading preserves that data while its newly authored questions can still use fillings.

chatsnack.SamplerParams(model: str | None = None, timeout: float | None = None, retry: dict | None = None, base_url: str | None = None, api_key_env: str | None = None) dataclass

Serializable execution choices; None delegates to SDK/environment defaults.

authored()

Keep absent SDK defaults and credentials out of saved definitions.

validate()

Fail invalid serializable options before dependent model work starts.

chatsnack.Answer(choice: str, score: float, probabilities: dict[str, float], confidence: float) dataclass

One decoded judgment with its provider probabilities and primary score.

chatsnack.YesNoAnswer(choice: str, score: float, probabilities: dict[str, float], confidence: float) dataclass

Bases: Answer

Binary judgment with readable Boolean views of the selected outcome.

yes property

Whether the selected outcome is yes, including an exact tie.

no property

Whether the selected outcome is no.

chatsnack.ChoiceAnswer(choice: str, score: float, probabilities: dict[str, float], confidence: float) dataclass

Bases: Answer

Selection among authored alternatives; preserve Jev's selected key.

chatsnack.ScoreAnswer(choice: str, score: float, probabilities: dict[str, float], confidence: float) dataclass

Bases: Answer

Ordered-rubric judgment with Jev's probability-weighted numeric score.

Provider contract

Verified on 2026-09-17 against TypeSafe's HTTP API, structured question entries, async Python client, and retry policy. Chatsnack 0.9.0 includes typesafe-sdk>=0.6.0,<0.7.0 as a standard dependency.