Helium Market Resolution tests whether LLMs can read live option chains (IV, delta, put-call parity) with ground truth from the chain itself—not MCQ finance trivia.
300 frozen prompts, six tickers, twelve frontier models scored. No model above 50%; grok-4.20-reasoning leads at ~48%.
HF: https://huggingface.co/datasets/HeliumTrades/helium-market-resolution-benchmark
Overview: https://heliumtrades.com/benchmarks/
We use a private holdout for anti-contamination. Open to contributing a LiveBench-compatible subset if there's interest.
Helium Market Resolution tests whether LLMs can read live option chains (IV, delta, put-call parity) with ground truth from the chain itself—not MCQ finance trivia.
300 frozen prompts, six tickers, twelve frontier models scored. No model above 50%; grok-4.20-reasoning leads at ~48%.
HF: https://huggingface.co/datasets/HeliumTrades/helium-market-resolution-benchmark
Overview: https://heliumtrades.com/benchmarks/
We use a private holdout for anti-contamination. Open to contributing a LiveBench-compatible subset if there's interest.