Terms of Use

Last updated 29 October 2026

1. Who we are

MemJudge (memjudge.org) is a free, non-commercial, open-source benchmark for AI memory systems. It is run by Bit Robotics Ltd, a company registered in England and Wales (company number 11680576), registered office 13 Freeland Park, Wareham Road, Lytchett House, Poole, Dorset, BH16 6FA. Bit Robotics also makes Recalld, a memory system MemJudge tests; every result says whether the person who ran it is affiliated with the system it tests. Contact: admin@memjudge.org.

2. What these terms cover

These terms cover the memjudge.org website, the hosted runner you use through the Run page, and the results we publish. The MemJudge software itself is licensed separately under Apache-2.0; you can run it on your own computer without these terms. How we handle personal data is described in the Privacy notice.

3. Using the site

You may read, link to and reuse published results under their licences (section 8). Do not attack, overload or try to get around the security or limits of the site or the runner.

4. Hosted runs

• Who can run. You must be 18 or over and agree to these terms when you start a run.
• Sign-in. You sign in with GitHub. Your GitHub account id is used only to apply the limits (one run at a time, three a day); it is stored only as a keyed hash and never published.
• Your keys. You give API keys for a memory service and for the judge. They reach us over HTTPS, are stored only encrypted, are used only for that run, are deleted when a job reads them (any left unused are deleted within 3 days) and are never logged. Resuming a run asks for them again.
• Code we do not control. During a hosted run, your keys are used by third-party code: the memory service's SDK and the open-source packages that the connector and MemJudge depend on. We review connectors, pin exact package versions and limit where a run can connect, but we cannot guarantee this code is free of errors or malicious changes. A compromised package could read your keys and misuse them, for example through the vendor's own API.
• Use keys made for the run. For each run you must create new keys that you use for nothing else, set a spending limit on them where the vendor offers one, and revoke them when you are done with the run. If you think a key was misused, revoke it at once and tell us at admin@memjudge.org.
• Your costs. The memory service and the judge bill your own accounts, and you are responsible for those charges. We show the judge's estimated upper cost before you start; we do not estimate the memory service's cost and are not responsible for charges on your accounts.
• Your vendor's terms. You must have the right to run the benchmark on your memory service account and to have its result published. Some services' terms limit benchmarking or publishing results. We do not offer hosted runs of services whose terms we know forbid it (currently Mem0 Platform and Zep Cloud). For the others, checking your service's terms is your responsibility, and by starting a run you confirm you have.
• Honest answers. Answer the affiliation question truthfully, and do not use accounts opened under false details.
• Availability. The runner is free and provided as it is. A run may stop or fail; a stopped run can be resumed within 48 hours, at most 3 times. We may refuse or stop runs to protect the service or comply with the law.

5. Publication

A completed hosted run is published automatically and publicly: its scores, every question, the context your memory system returned, the judge's verdict and your affiliation answer. It is signed so anyone can check it. Do not start a run you do not want published.

By starting a run you give Bit Robotics Ltd a free, worldwide and permanent permission to publish its result, including the parts that come from your accounts (the context your memory system returned and the judge's replies), and to license MemJudge's own parts as section 8 describes. You confirm you are allowed to give it. We may correct, label or remove results under our Corrections policy.

6. Your responsibility

If someone, such as a memory-service vendor, makes a claim against Bit Robotics Ltd because you broke these terms or your memory service's terms, you are responsible for the reasonable losses and costs this directly causes us. This does not apply to anything that is our fault.

7. No warranty, and our liability

MemJudge is free and provided "as is". We work to make results accurate and reproducible and to keep your keys safe, but we do not guarantee either. To the extent the law allows, we are not liable for indirect or consequential loss, lost profit or data, charges on your vendor or judge accounts, or loss from the theft or misuse of keys you gave for a run, including misuse by third-party code the run uses. Our total liability to you for anything to do with MemJudge is limited to £100. Nothing here limits liability that the law does not allow to be limited, such as for fraud, or for death or personal injury caused by negligence.

8. Licences

MemJudge's code is Apache-2.0. Scores, metrics and verdicts are CC BY 4.0. Dataset text inside results keeps its own licence: LoCoMo is CC BY-NC 4.0 (non-commercial use only) and LongMemEval is MIT. "MemJudge" and its logo are trademarks of Bit Robotics Ltd (trademark policy). Other product names belong to their owners and are used only to identify the systems tested; MemJudge is not affiliated with or endorsed by them.

9. Changes and law

We may update these terms; the date at the top shows the current version. A change applies to runs started after it, and to your use of the site from then on. These terms are governed by the law of England and Wales, and you can bring legal proceedings in the courts of England and Wales. If you live in Scotland or Northern Ireland, you can also bring them in your local courts.