Online Assessment Integrity and Anti-Cheating: A Progressive Model for a Consumer Math App

Threat model table

Attack	Who does it	How detectable	Cost to defend
Looking up the answer (search, textbook)	Any age/tier	Response time far below the fastest-plausible human solve time; near-instant correctness after visible idle/tab-blur	Low — server-side response-time floor per item, tab-visibility events
Parent/sibling solves it for the child	Young children mostly	Style mismatch vs. the account’s own skill trend line	Low-medium — trend-based flag only, never punitive at this age
Solver app / calculator on a second device	Older children, teens, adults	Response-time floor; a solver returns near-instantly regardless of difficulty while a human’s time scales with it	Low-medium — same floor mechanism, calibrated per item type
Second device answers while primary device is the “timer”	Teens, competitive tier	Hard without device attestation; mitigated structurally by keeping timing server-side so a second device gains no measurable edge	Medium — architectural, not a bolt-on check
Sharing answers between friends	Any age, class contexts	Answer-similarity/collusion statistics (omega, GBT, K-index); meaningful only once the item bank is large	Medium-high — needs a real bank plus statistical machinery
Automated scripts/bots (API replay, headless browser)	Technical users, leaderboard farmers	Bot-management signals (behavioral fingerprint, proof-of-work, TLS/JA3), rate limiting, Turnstile/Privacy Pass tokens	Low-medium — off-the-shelf infrastructure
Account sharing (one login, many people)	Families, competitive tier	Concurrent-session detection, WebAuthn credential mismatch, skill discontinuity	Medium — needs session/device tracking
Deliberately failing easy content to farm rank (“sandbagging”)	Competitive/leaderboard users	Variance anomaly vs. own history	Medium — needs a maintained skill/rating baseline (already required by topic 18)

Findings

1. Server-authoritative scoring and why client timing cannot be trusted

A browser is fully inspectable and modifiable by its own user: DevTools can pause execution, rewrite variables, replay edited network requests, and override Date.now()/performance.now(). This is the same threat model that made “client-authoritative” multiplayer architectures (client reports its own score/time, server just believes it) obsolete. The server must independently timestamp question-served and answer-received, and independently verify correctness — the client only renders and collects. Nothing else scales to a leaderboard where speed carries points, since a client-reported duration is exactly what most rewards manipulation.

2. Item bank strategy: size, parameterization, randomization, exposure control

CAT research gives a directly applicable playbook. Item exposure — the share of test-takers who see a given item — tends toward 1 for the most informative items in a naive adaptive algorithm, which is itself a security problem: a repeatedly-shown item becomes shareable [10]. Three established mitigations: the Sympson-Hetter method (draw a random number, compare to a per-item exposure parameter before administering even the best-fit item); randomesque/stratified selection (pick randomly among the top 5-10 most-informative items, not always the single best); and shadow testing (van der Linden — build a full hypothetical optimal test at each step for globally, not just locally, optimal choices) [10]. Underneath all three is a large item pool, grown cheaply via parameterized/algorithmic item generation (a template like a + b = ? with randomized operands per difficulty band) rather than hand-authored items — explicitly the practical way pools are grown economically per the CAT literature [10].

3. Statistical detection: response-time outliers

Van der Linden’s lognormal response-time model treats a person’s item response times as governed by a person-level “speed” parameter alongside item-level time-intensity and discrimination parameters, structurally parallel to how two-parameter logistic IRT treats correctness [11]. Fitted, it supports classical and Bayesian posterior-predictive checks for aberrance — a response markedly faster or slower than predicted — already applied to detect aberrant behavior on computerized adaptive tests [11]. For Math Challenge, the practical version needs none of the full model at first: an empirical floor (“no verified human solves this item class under X ms”) is a legitimate first line of defense, escalating to the fuller model only at tiers where stakes justify the investment.

4. Statistical detection: answer-similarity and collusion indices

Answer-copying/collusion detection is an established psychometric subfield, confirmed via ERIC: Wollack’s omega (Ω) index (refined by Maeda & Zhang 2017; Sunbul & Yormaz 2018), the generalized binomial test (GBT) compared against omega for power/Type I error (Zopluoglu & Davenport, 2012), the K-index (Holland) versus Kullback-Leibler divergence (Belov & Armstrong, 2010; Ucar & Dogan, 2021), a response-time-based KL measure (Man et al., 2018), and a Variable Match Index (Belov, 2011) [12]. All share a structure: they flag when two test-takers give the same wrong answer more often than chance predicts given their individual ability — an unusually high rate of identical incorrect responses is the signature. This is meaningful only once an item bank is large enough that two people converging on the same item by chance is rare.

5. Lockdown browsers and remote proctoring — and why not to use them on children

Camera-based remote proctoring (Proctorio, ExamSoft, Honorlock, Respondus) surged during COVID-19 and left a documented trail of harm and legal pushback:

• Effectiveness is doubtful. University of Twente research is reported (via secondary summary) as finding Proctorio’s cheating-detection sensitivity “very close to zero” [4].
• A US federal court found a specific practice unconstitutional. Ogletree v. Cleveland State University concerned a mandated webcam “room scan”; the case is documented as a Fourth Amendment ruling against the practice by a federal district court in Ohio (2022) [3].
• Illinois BIPA litigation. Students alleged Proctorio collected biometric information/identifiers without required consent; Proctorio denied the allegations. BIPA carries statutory damages of $1,000 (negligent) / $5,000 (intentional) per violation, which is why proctoring vendors face recurring litigation of this kind [13].
• The Dutch case often cited runs the other way. Rechtbank Amsterdam, ECLI:NL:RBAMS:2020:2917 (11 June 2020): UvA’s Central and Faculty Student Councils sought an injunction against Proctorio; the court rejected all claims, finding the processing lawful under GDPR Art. 6(1)(e), “necessary” given COVID-19 closures, “proportionate” (automated screening, limited human review, 30-day deletion), and “adequately safeguarded” (EU storage, processor agreement, DPIA on file), noting Proctorio was “less invasive than continuous live video surveillance” [5]. It is a genuine court win for proctoring — conditioned on a narrow, documented safeguard set (short retention, DPIA, adult students, pandemic necessity) that describes nothing about a consumer app for four-year-olds.
• A parallel discrimination finding. VU Amsterdam separately faced a Dutch Institute for Human Rights complaint alleging Proctorio discriminated against a Black student; the institute found no discrimination in that case, but the complaint sits within the same broader pattern of algorithmic-bias concern around proctoring (unverified beyond snippet).

Conclusion for Math Challenge: camera/microphone proctoring of children has no place in this product at any tier. The documented harms apply with more force to minors than to the university adults these cases involved, and none of the Amsterdam court’s mitigating conditions (pandemic necessity, adult consent capacity, institutional DPIA) exist here.

6. Device attestation on the web in 2026

• WebAuthn/passkeys: standard browser API, works in any modern browser including installed PWAs; binds a public-key credential to a device, and a server can store multiple credential IDs per account — WebAuthn itself doesn’t auto-flag “new device,” that bookkeeping is server-side [16].
• Private Access Tokens (Apple) / Privacy Pass (IETF): a three-party scheme (issuer, client, server) proving a request comes from a legitimate device without revealing identity, via a challenge-response flow and RSA blind-signature validation; built into iOS 16+/macOS Ventura+ to reduce CAPTCHAs, and folded into Cloudflare Turnstile — but no confirmed first-class support outside Apple platforms, so treat as a bonus signal on Safari/iOS traffic, not a general mechanism [14][15][18].
• Google Play Integrity API: exclusively for native Android apps distributed via Google Play, verifying app-binary authenticity, install source, and device genuineness — does not apply to a PWA; Google’s own web guidance points elsewhere (Privacy Sandbox) [17].
• Apple App Attest: analogous native-iOS attestation, tied to App Store distribution the same way Play Integrity is tied to Play Store — unavailable to a PWA not distributed through the App Store.

Practical read for a PWA-first product: WebAuthn is the one device-binding primitive genuinely available everywhere; Play Integrity/App Attest are structurally unavailable without native wrapper apps; Private Access Tokens are a real but Apple-weighted bonus already folded into Turnstile.

7. Bot detection and rate limiting

Cloudflare’s bot-management stack combines an ML engine scoring every request 1-99 from request/header/session features, a heuristics engine matching known-malicious fingerprints, and JavaScript-based detection of headless browsers, refined by a session cookie (__cf_bm) that smooths scores to cut false positives [18]. Turnstile is the consumer-facing version: small non-interactive JS challenges (proof-of-work, proof-of-space, web-API probing, browser-quirk detection) instead of a visual puzzle, already treating Privacy Pass tokens as one input [15][18]. Rate limiting on submission/scoring endpoints is the simpler complementary layer: capping submissions per account/IP/window catches high-volume scripted abuse regardless of whether any single request looks human.

8. How named competitive platforms handle cheating at scale

• Codeforces: rates performance relative to opponents of known rating within a contest rather than fixed per-problem points, reducing the incentive to game a raw point value (see topic 18 research, docs/research/2026-07-31-mc-18-leaderboards-competition.md) [1].
• Chess.com: detects suspicious play from “over 100 gameplay factors,” explicitly not relying on a single accuracy score (“Accuracy is not cheat detection”); automated systems handle ~85% of closures; Jan-Mar 2025 saw ~28,000 appeals reviewed of 314,000 closures at a 0.2% grant rate; mandatory two-camera Proctor software is required only for cash-prize events; methodology was externally vetted by Harvard statistician Natesh S. Pillai (2016) and endorsed by US Chess (2020) [2].
• Duolingo: fights leaderboard/XP-farming (auto-completing bots, repetitive-content farming, timing exploits) by monitoring unusual XP spikes and unrealistically fast lesson completion, backed by bans and community reporting — the response-time-outlier principle applied informally at product scale [19].
• Kaggle: rules are known industry-wide to restrict private sharing outside team boundaries, cap team size/merger windows, and prohibit multi-accounting, but this section’s specifics are general industry knowledge, not a confirmed citation (documentation was not directly accessible during this research).
• Kahoot: no citable primary-source content on cheating/bot tooling (e.g., third-party “flooder” scripts) was retrievable; this remains an open gap, not a sourced claim.

Design implications

A concrete, six-tier progressive ladder. Each tier only adds controls on top of the previous tier’s server-authoritative foundation — nothing is removed going up, nothing above tier 0 is ever pushed down onto a younger child’s tier.

1. Tier 0 — Kinder (4-6): server-authoritative timing/scoring only, invisibly. Server independently timestamps question-served/answer-received and checks correctness; client never controls either value. No visible anti-cheat UI, no lockout, no cheating messaging — a parent solving alongside their child is the intended use case, not a threat.
2. Tier 0 — response-time floor, logging-only. A per-item-type minimum plausible solve time is recorded and logged if breached, never blocking or scoring zero. Pure telemetry to calibrate later tiers; surfaces only to a future parent/guardian dashboard, never to the child.
3. Tier 1 — Early elementary (7-9): silent variance monitoring. Server tracks each learner’s own accuracy/speed trend per skill; a sudden large deviation triggers only a soft signal (slightly more cautious adaptive difficulty) — never a lockout, warning, or visible penalty.
4. Tier 1 — rate limiting on submission endpoints. Basic per-account/IP submission caps (shared infrastructure) protect the backend from scripted abuse from this tier up.
5. Tier 2 — Late elementary/middle school (10-13): response-time floor becomes an active, gentle signal. Breaching the floor triggers a friendly UI moment (“that was fast — want to double check?”) rather than a silent log; repeated breaches lower mastery-estimate confidence, never void points. Still no lock, no proctoring, no parental alarm.
6. Tier 2 — item bank randomization begins to matter. Parameterized item generation (randomized operands per difficulty band) becomes the default delivery mechanism, since this is the age range where a sibling or classmate first gains an incentive to hand down an exact problem set.
7. Tier 3 — High school/general teen (14-17): Turnstile + WebAuthn device binding on leaderboard actions only. Turnstile protects leaderboard-affecting submission endpoints (invisible by default); WebAuthn device credentials are introduced for account recognition, not as a login requirement — flagging “a third new device this week” as one input, never a sole gate.
8. Tier 3 — response-time and answer-similarity statistics activate, leaderboard scope only. Once the item bank is large enough (Finding 4), the server runs a simple response-time-outlier check (Finding 3) and, where accounts share enough common items, an answer-similarity check modeled on the published indices — scoped to leaderboard activity, not ordinary practice.
9. Tier 4 — Advanced/pre-competitive (16+, opted into ranked play): full item-exposure control. Sympson-Hetter-style exposure limiting (Finding 2) caps how often even the best-fit next item is shown to similar-ability users, guarding the “everyone at this rank gets the same next problem” sharing vector that matters once real stakes exist.
10. Tier 4 — session/account-sharing heuristics become active, not just logged. Concurrent-session detection and skill-discontinuity flags now actively raise rating volatility/RD (the way a suspicious new account would), rather than only appearing in a dashboard.
11. Tier 5 — Competitive/scholarship-eligible top tier (opt-in, explicit stakes, guardian consent where a minor is involved): the full statistical suite, still zero cameras. The published psychometric machinery (omega/GBT-style collusion detection, the fuller lognormal response-time model) earns its cost here, since the bank is large and stakes are high. Even at this ceiling the answer is more statistics, never a webcam, lockdown browser, or biometric capture — Finding 5’s record gives no scenario where camera/biometric proctoring of a minor, or of an adult with no institutional mandate, is defensible.
12. Server-side vs. client-side, at every tier, without exception. Client-side, always: rendering the problem, collecting the answer, local UI feedback. Server-side, always, from tier 0: the timestamp pair, the correctness check, the score, and (tier 3 up) every statistical signal in Findings 3-4 and 7. Timing/correctness authority never moves to the client at any tier — the ladder is progressive in stakes, not in client trust, which is never granted.
13. What we will deliberately never do to a child, at any tier. No webcam or microphone capture. No biometric data collection (face, voice, keystroke dynamics, gaze tracking). No lockdown browser. No remote human proctor. No score penalty or account action visible to a child under Tier 3 — below Tier 3, signals are calibration telemetry and, at most, a parent/guardian dashboard item. No punitive framing (“you were caught cheating”) anywhere — the worst visible outcome at any tier is a lower-confidence mastery estimate or a friendly prompt, matching the decided design: anti-cheating stays nearly invisible for young children and tightens only with rising stakes.

Open questions for the project owner

1. At what age/tier, if any, should a parental dashboard surface anomaly signals (Tiers 1-2) — and should it ever be visible to the child, even indirectly?
2. Will Math Challenge run any event with real-world stakes (scholarship, cash prize, school-recognized competition) justifying Tier 5’s full statistical suite, or does “competitive tier” mean leaderboard bragging rights only?
3. Should WebAuthn/passkey adoption ever be required, or always optional, given it is the one universal device-binding primitive but adds friction for a young child’s account?
4. For legitimate family account sharing (parent and child on one login), how should Tier 4+ session heuristics avoid misflagging normal family device-switching as suspicious?
5. Is there appetite to publish a trust/safety statement that Math Challenge will never use webcam/biometric proctoring, as a differentiator from Proctorio-style products and a trust signal to parents?