The hallmarks, mapped to the brief
The hackathon asks for harnesses that adapt, remember, and operate reliably over the long haul, built on MongoDB Atlas, Vector Search and agentic memory tooling, with OpenRouter for the model. Each row names the feature, where to see it, and the narrated demo beat that shows it.
The evolver rewrites rules, context policy, guardrails and tool access from traces and feedback. Config only, never code.
Where: Workspace → Evolve, then the per-operation diff on the Evolution result screen.
Preferences, facts and episodes live in the memories collection with embeddings. Every run, trace, eval and patch is persisted, so the harness has a queryable history.
Where: Workspace → Retrieved memories; Versions → version tree.
Six deterministic evals per user replay on every candidate. A gate blocks regressions and locked-guardrail removals. Any version rolls back in one click.
Where: Evolution result → Evals tab and blocked operations; Versions → Rollback.
Six collections: harnesses, traces, memories, evalCases, evalRuns, patches. Every box after the run writes to Atlas.
Where: Mode badge in the header; README has the collection and index definitions.
$vectorSearch over memory embeddings, filtered by user and memory kind. The harness's context policy sets kinds, topK and minScore, and is itself an evolvable knob.
Where: Workspace → Context policy knob and Retrieved memories with similarity scores.
The agent writes its own memories: every piece of feedback becomes an episode with an embedding, and every promotion becomes a fact. Retrieval is policy-driven, not a fixed prompt.
Where: Workspace → Feedback (writes an episode); Versions (promotion memory).
The agent run and the evolver call OpenRouter through the OpenAI SDK, with a fast model for runs and a stronger one for evolution. Without a key the app runs on a config-aware mock and is labelled Demo data.
Where: Mode badge in the header; model name on every trace.
Gate rule 5: a human clicks Promote. Partial promotion, reject and rollback are all one click, and blocked operations are shown in red with the rule that blocked them.
Where: Evolution result → Promote selected; Versions → Rollback.
Three seeded users, three domains: a PM's status update, a support lead's customer reply, and a care coordinator's discharge summary with a locked PHI guardrail.
Where: Dashboard → user cards; switch users in the header.
Architecture
- Run task
- Load harness
- $vectorSearch per context policy
- OpenRouter call + mock tools
- Guardrails
- traces
- Evolver proposes patch
- Replay evals: current vs candidate
- Gate
- Human promotes new version
Every box after the run writes to Atlas: harnesses, traces, memories (vector index on embedding, filters on userId and kind), evalCases, evalRuns, patches. Stack: Next.js App Router, TypeScript, Tailwind, shadcn/ui, Zod, the MongoDB Node driver, and OpenRouter through the OpenAI SDK.