[Leaderboard] Counting Sheep - 86.7% F1 Memory Extraction

[mbmarsirius]

Leaderboard Submission

System: Counting Sheep
Organization: Marsirius AI Labs

Results (HaluMem-Medium, Memory Extraction)

Metric	Score
F1	86.7%
Precision	76.5%
Recall	100.0%

Comparison

Rank	System	F1
🥇 1	Counting Sheep	86.7%
2	MemOS	79.7%
3	Mem0-Graph	57.9%

About

Counting Sheep uses cognitive memory architecture inspired by human brain behavior - sleep-based consolidation, LLM-powered extraction, hierarchical memory.

Contact: mb@marsirius.ai

We can provide evaluation logs for verification.

---

Thank you for this excellent benchmark!

[hush-cd]

Thank you very much for your interest in HaluMem and for submitting the results of Counting Sheep. We appreciate the detailed description of your system and the effort you’ve put into the evaluation.

At the moment, the provided results cover only a single task and metric (F1 / Precision / Recall on HaluMem-Medium – Memory Extraction). For inclusion in the official leaderboard, we currently require more comprehensive evaluation results across the benchmark tasks.

Specifically, in addition to Memory Extraction, we would need the evaluation results (with corresponding metrics) for:


• Memory Updating
• Memory Question Answering
  

This is to ensure a fair and consistent comparison across systems, since the leaderboard is intended to reflect overall performance across the full HaluMem benchmark rather than a single subtask.

Once the complete set of required evaluation results is available, we would be happy to review the submission and consider adding it to the leaderboard. Evaluation logs for verification are also very welcome.

Thanks again for your contribution and for your kind words about the benchmark. We look forward to your updated results!