Skip to content
MemJudge
LeaderboardCompareRunHow scoring worksGitHub

Privacy notice

Last updated 28 October 2026

MemJudge collects as little personal data as it can. You can read the whole site without signing in, and it uses no analytics, adverts or tracking cookies.

1. Who is responsible

Bit Robotics Ltd (company number 11680576, registered office 13 Freeland Park, Wareham Road, Lytchett House, Poole, Dorset, BH16 6FA) is responsible for your data. Contact: admin@memjudge.org.

2. What we handle, and why

  • Visiting the site. Cloudflare, which hosts the site, processes your IP address and request details to deliver the pages and protect them from attacks. Your browser keeps your theme choice and, on the Run page, your sign-in for that tab and the id of your last run. These stay on your device and only make the features you use work, so we do not ask for cookie consent.
  • Signing in. GitHub shares your public profile when you sign in. We use only your numeric account id, and we store only a keyed hash of it: never the id itself, your username, name or email.
  • Starting a run. We keep a run record: that hash, the run's settings and progress, your affiliation answer, the date of the terms you agreed to, and timestamps. The runner's request logs record your IP address. Your API keys are encrypted for the run's jobs, deleted when a job reads them (unused ones within 3 days) and never logged. The jobs log their progress with your keys removed; only you see those lines, on the run's form.
  • Published results. A result holds no personal data except your affiliation answer (yes or no, and which system), and it is not linked to your GitHub account.
  • Emails. If you email us, we keep the correspondence to deal with it.

Our lawful basis is legitimate interests: running a free public benchmark, applying its fair-use limits, keeping it secure and handling reports. We do not sell data or use it for marketing.

3. Who else handles it

  • GitHub, for sign-in.
  • Google Cloud, which runs the runner in the United States (Iowa).
  • Cloudflare, which serves the website and the published result files.

These providers may process data in the United States. Transfers rely on the UK–US data bridge where the provider is certified under it, and otherwise on the UK's standard contract terms. Your memory service and judge provider (Anthropic or OpenRouter) receive the benchmark data under your own accounts and their own privacy terms; that data is dataset text, not personal data about you.

4. How long we keep it

  • Run records: deleted automatically 90 days after the run starts.
  • Daily-limit counters, which hold the same hash: deleted after 30 days.
  • The runner's request logs and the run jobs' logs: 30 days.
  • Cloudflare's logs: as Cloudflare's own policies set.
  • Published results: for as long as the leaderboard runs, unless removed under the Corrections policy.
  • Emails: as long as we need them to deal with what you asked.

5. Your rights

You can ask to see a copy of your data, to correct or delete it, and to restrict or object to how we use it. Email admin@memjudge.org. We store only a hash of your GitHub id, so tell us your GitHub username and we will find your records from it.

6. Complaints

If you are unhappy with how we handle your data, email admin@memjudge.org first. We acknowledge a complaint within 30 days and tell you what we did about it. You can also complain to the Information Commissioner's Office at ico.org.uk or on 0303 123 1113.

7. Children and changes

The hosted runner is for people aged 18 or over. We may update this notice; the date at the top shows the current version.

MemJudge is a free, non-commercial, vendor-neutral benchmark for AI memory systems. The code is Apache-2.0 and scores are published under CC BY 4.0; dataset text in the results keeps its own licence (LoCoMo is CC BY-NC 4.0, non-commercial). It is built by Bit Robotics Ltd, which makes Recalld, so every run says whether its submitter is affiliated with the system it tests.

Every result here is signed. How signing works and how to check a result yourself.

Terms of Use · Privacy notice · Corrections policy · admin@memjudge.org

Live: site 4c60e4d · results ffc8d78