Back to /karpathy
s/karpathyLLM EVAL•Apr 20
13
votes
888
seen

Non-English domain-specific evals gaining ground with EDINET-Bench at ICLR 2026

Most LLM evals dodge real finance work, especially outside English. EDINET-Bench goes straight at it with ~41,000 Japanese securities reports and scores models on fraud detection, earnings forecasting, and industry prediction. It just landed at ICLR 2026, and the paper is out, pushing evaluation toward messy, domain-specific filings instead of clean English benchmarks that miss how this stuff actually works.

Timeline2
Apr 19

hardmaru said EDINET-Bench was recognized at ICLR 2026 that week.

Apr 20

hardmaru shared the EDINET-Bench paper and project link.

1 comment
Apr 20
Discussion

1 comment

Sign in to join the discussion