Skip to content

Pull requests: huggingface/lighteval

New pull request New
172 Open 699 Closed
172 Open 699 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

fix(metrics): add actionable BERTScore baseline guidance
#1373 opened Aug 31, 2026 by LarryHu0217 • Draft
1
fix(tasks): skip few-shot reuse warning for zero-shot tasks
#1372 opened Aug 31, 2026 by LarryHu0217 • Draft
1
fix(judge): honor concurrent_requests for the OpenAI/TGI judge backend
#1371 opened Aug 30, 2026 by mohansree14 Loading…
1
Key results and details output paths on the model revision
#1370 opened Aug 30, 2026 by thisisreyy Loading…
1
Pin prediction cache to resolved model revisions
#1369 opened Aug 30, 2026 by LarryHu0217 Loading…
fix(cache): make multi-split document IDs unique
#1368 opened Aug 30, 2026 by linhongyu510 Loading…
1
feat: add OrcaRouter as a first-class provider in the LiteLLM backend
#1367 opened Aug 30, 2026 by kuswardhanietidims-svg Loading…
fix: resolve SamplingMetric string normalize argument
#1366 opened Aug 29, 2026 by Abelo9996 Loading…
1
fix: honor task-level num_samples and warn when it has no effect
#1361 opened Aug 27, 2026 by Abelo9996 • Draft
ci: bump pr_style_bot to current style-bot-action reusable workflow
#1360 opened Aug 27, 2026 by Abelo9996 Loading…
Fix Dyck scoring to preserve bracket order
#1358 opened Aug 25, 2026 by Excelius-Wang Loading…
1
Fix continuation tokens after distributed gathering
#1357 opened Aug 25, 2026 by Excelius-Wang Loading…
1
Fix DROP exact match: the length check compares gold to gold
#1356 opened Aug 25, 2026 by Ag3497120 Loading…
fix(sglang): size batch truncation from the longest input in the batch, not the first
#1355 opened Aug 24, 2026 by shoemoney Loading…
don't score an empty F1 gold reference as zero
#1354 opened Aug 24, 2026 by gyanu2507 Loading…
1
docs: fix broken code examples in evaluating-a-custom-model.mdx
#1352 opened Aug 22, 2026 by mohansree14 Loading…
1
Fix duplicate custom task module imports
#1351 opened Aug 21, 2026 by yangweigbh Loading…
1
fix: keep every task's responses when reloading them from details
#1350 opened Aug 19, 2026 by tonycoder-hub Loading…
2 tasks done
Fix relational comparisons with different condition counts
#1349 opened Aug 19, 2026 by Excelius-Wang Loading…
Parse hellaswag_arabic endings with ast.literal_eval
#1348 opened Aug 19, 2026 by Nakul-Sinha Loading…
1
Add add_generation_prompt=True when tokenizing judge prompts
#1347 opened Aug 19, 2026 by Nakul-Sinha Loading…
1
fix(metrics): avg@n scores raw text instead of post-processed text, and skips preprocessing
#1335 opened Aug 16, 2026 by arthi-arumugam-git Loading…
Fix few-shot sampler isolation
#1334 opened Aug 16, 2026 by atirna Loading…
2
Fix BBEH generative evaluation
#1333 opened Aug 15, 2026 by lakshayxi Loading…
Fix DetailsLogger compatibility with xxhash 4.0
#1332 opened Aug 15, 2026 by Honorllq Loading…
1
1
ProTip! Updated in the last three days: updated:>2026-08-29.