Clearframe: rights-cleared video datasets
Side project: there is no client, I built it to study or to experiment.
Personal project, built over a weekend: an AI agent that assembles video datasets for model training, where a deterministic rights rule, not the model, decides which clips get in. Every manifest is signed and anyone can verify it.
Case study
- My role
- All of it, alone: product, agent and evals, backend, frontend and AWS infrastructure.
- Duration
- One weekend, October 2026
The problem
Video models are trained on footage that must be licensed for training, in the right territory, on the right date. A language model can read a request and find relevant clips, but it is not a legal control: it can be talked into ignoring a license, and nobody downstream can tell what it actually checked.
What I did and why
- The rule in code, at the only write path: the model proposes clips through tools, and a clip enters the dataset only if a pure function approves its license. The model cannot choose the territory or date that function checks.
- Evals as a gate: 21 cases with hand-written ground truth, prompt injections included, run on every pull request that changes the agent, plus an ablation without the rule to measure what it is worth.
- Manifests that prove their clearance: each clip carries the license it was approved on and the catalog version, and manifests are locked in S3 with Object Lock, next to a run record in DynamoDB.
- A notary that signs only what the rule allows: a separate Lambda, out of the pipeline's reach, re-checks each manifest against its own catalog and signs it with a KMS key. Anyone can verify it in the browser, from the command line or with the published keys.
- A live interface: the page follows the run as it streams, with each clip's verdict, the agent's timeline, its cost and the history of past runs.
- AWS in Terraform, minimal permissions: Lambda behind CloudFront, Cognito sign-in, a daily quota per user, and CI on GitHub OIDC that can touch only this stack's resources.
Results
- 0rights violations in 63 eval runs
- 3 of 8injected runs that beat the prompt-only rule
- 100%precision and recall across 21 eval cases
- $0.0018model cost per request
- 7,2 smedian run time
- 266automated tests
Stack highlights
- Claude (Anthropic)
- AI SDK
- Typescript
- Next.js
- Zod
- AWS Lambda
- AWS KMS
- Terraform
- Amazon Bedrock
- Github Actions
Working on something similar? Let's talk.
Get in touchCosa ho esplorato.
Un agente che usa i tool senza avere l'ultima parola: Claude propone le clip, una funzione pura controlla la licenza di ognuna per il territorio e la data della richiesta, e il dataset nasce solo da ciò che lei approva. Intorno: eval con ground truth scritta a mano che fanno da filtro su ogni pull request, un notaio separato che ricontrolla e firma ogni manifest con una chiave KMS, una pagina che verifica la firma nel browser, e tutto lo stack AWS in Terraform, con la CI su GitHub OIDC.
Cosa mi porto via.
Misurare prima di decidere: le eval hanno mostrato che una regola scritta solo nel prompt lasciava passare un'iniezione in 3 run su 8, e hanno trovato difetti che una review non avrebbe visto, come una ricerca che bruciava il budget di token su clip inutilizzabili.
