Skip to content

[BUG] - PreferenceDataset #136

Description

@patrickfleith

Bug Description

When running a PreferenceDataset.generate()

Problem 1
Inconsistent expected number of rows printed vs logged.

Expected 78 preference pairs
2025-11-20 11:46:47 | INFO     | datafast.datasets:generate - Starting PreferenceDataset.generate() | Expected rows: 234

Problem 2
When a validation error is thrown during llm.generate() the PreferenceDataset generation loop is broken. We need to catch, log, and gracefully continue

Metadata

Metadata

Assignees

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions