Dark Souls Fact Recall - LLM Benchmark Dashboard

Overall Comparison

Model	Parameters	Type	License	Score
1kimi-k2.5	1T (32B active)	MoE	Open	96%
claude-opus-4-5	?	unknown	Proprietary	93.1%
gpt-5.1	?	unknown	Proprietary	91.4%
gpt-4-turbo	?	MoE	Proprietary	88.6%
2mistral-small-3.2	24B	Dense	Open	76%
3devstral-2	123B	Dense	Open	63.4%
4mistral-large-3	675B (41B active)	MoE	Open	53.7%
5llama-4-maverick	400B (17B active)	MoE	Open	47.4%
6gpt-oss-120b	117B (5.1B active)	MoE	Open	46.9%
7llama-4-scout	109B (17B active)	MoE	Open	37.7%
8qwen3-235b-a22b	235B (22B active)	MoE	Open	35.4%
9gpt-oss-20b	21B (3.6B active)	MoE	Open	29.1%
10ministral-14b	14B	Dense	Open	22.9%

+ Detailed Analysis +

Scores by Category (%)

Model	Bosses	Areas &	NPCs &	Items &	Lore	Mechani
kimi-k2.5	93	97	97	94	100	95
claude-opus-4-5	87	90	94	97	100	90
gpt-5.1	87	97	86	89	100	95
gpt-4-turbo	80	100	86	89	92	85
mistral-small-3.2	50	37	91	89	100	95
devstral-2	67	53	54	57	92	65
mistral-large-3	57	47	51	29	92	60
llama-4-maverick	43	43	34	34	84	60
gpt-oss-120b	40	37	31	40	88	60
llama-4-scout	40	30	29	26	64	50
qwen3-235b-a22b	30	37	29	26	60	40
gpt-oss-20b	20	33	29	26	40	30
ministral-14b	20	30	20	23	24	20

Critical Questions

Model
kimi-k2.5
claude-opus-4-5
gpt-5.1
gpt-4-turbo
mistral-small-3.2
devstral-2
mistral-large-3
llama-4-maverick
gpt-oss-120b
llama-4-scout
qwen3-235b-a22b
gpt-oss-20b
ministral-14b

>=80% = PASS | <80% = FAIL

+ Example Questions +

B1.2 Bosses medium

In Dark Souls 1, which four bosses hold Lord Souls that must be collected to fill the Lordvessel?

Expected: Gravelord Nito, Seath the Scaleless, The Bed of Chaos, Four Kings

A2.2 Areas & Map Layout medium

What two conditions must be met to open the gate to Sen's Fortress in Dark Souls 1?

Expected: Ring the Bell of Awakening in the Undead Parish (above the Bell Gargoyles), Ring the Bell of Awakening in Quelaag's Domain (below Blighttown, after defeating Chaos Witch Quelaag)

N3.3 NPCs & Questlines medium

In Dark Souls 1, what happens after Knight Lautrec of Carim is freed from his cell in the Undead Parish and both Bells of Awakening have been rung?

Expected: After being freed, Lautrec moves to Firelink Shrine. Once both Bells of Awakening are rung, he kills Anastacia, the Firekeeper of Firelink Shrine. This disables the Firelink bonfire and the player must invade Lautrec's world in Anor Londo to recover her soul and restore the bonfire.

I4.2 Items & Weapons medium

List all four types of Black Knight weapons in Dark Souls 1.

Expected: Black Knight Sword, Black Knight Greatsword, Black Knight Halberd, Black Knight Greataxe

L5.2 Lore medium

What happened when the Witch of Izalith attempted to recreate the First Flame in Dark Souls 1?

Expected: Her attempt to recreate the First Flame went catastrophically wrong. She created the Chaos Flame instead, which transformed her into the Bed of Chaos. This event gave birth to all demons in the Dark Souls world and corrupted the land of Izalith.

M6.1 Mechanics & Covenants medium

What does 'kindling' a bonfire do in Dark Souls 1, and what item allows further kindling beyond the default?

Expected: Kindling a bonfire increases the number of Estus Flask charges the player receives when resting at that bonfire, from the default 5 to 10. The Rite of Kindling (obtained from Pinwheel) allows further kindling in two additional steps: to 15 and then to 20. Each kindling costs 1 Humanity.