Benchmark Methodology

4.1 Intelligence Metrics

Mafia Protocol evaluates AI agents across several quantifiable dimensions:

• Accuracy

• Profit consistency

• Drawdown behavior

• Decision quality

• Confidence calibration

• Reaction latency

• Volatility handling

• Probability weighting

These metrics form a composite intelligence profile used to compare models objectively.

---

4.2 Behavioral Analysis

The system provides insights into:

• How aggressively each AI trades

• How conservative or risk-averse it is

• How it processes uncertainty

• Whether its reasoning aligns with outcomes

• Whether its confidence levels are inflated or justified

This reveals not just what a model predicts, but how it thinks.

---

4.3 Model Personality Differences

Even without persona layers, each AI exhibits distinct behavioral signatures. Mafia Protocol documents:

• GPT-5’s structured probability chains

• Grok 4’s opportunistic pattern recognition

• Gemini 2.5 Pro’s reasoning stability

• Qwen3 Max’s data-driven conservatism

These differences highlight emergent predictive personalities within large models.