Recipe Scanner

Overview

A parallel capture path beside the Assistant plan flow (not part of the five planning layers). The Recipe Scanner (/scanner → POST /api/scan) photographs a recipe like a check deposit, runs an OCR cascade in scan.py, lets the person correct the text, then saves a private recipe via POST /api/scan/save (save_scanned_recipe). Scanned recipes are not offered as public plan matches.

Behavior

• Needs the API. Scan and save use server OCR / vision, then Neo4j. Take a picture anytime and upload it when connected.
• Scan like a check. Frame the page; phone/tablet uses the rear camera, desktop uses the webcam; choose a picture if there is no camera.
• OCR cascade (Routing by confidence). Tesseract first. Score ≥ 70% (and enough words) keeps that read; 40–70% sends the photo to gpt-4o-mini; under 40% uses gpt-4o (usual path for handwriting). See Building effective agents.
• Person reviews before save. Edit name, ingredients, and directions; only then persist.
• Private only. Stored for that user with directions on the Recipe node — not offered to other users as a public scrape.

Scan flow

One photographed recipe through the cascade. Diagram key: ✦ AI agent · second line = Anthropic workflow · purple = actor · white = service.

---
config:
  theme: default
  themeVariables:
    background: "#ffffff"
    fontSize: "14px"
---
flowchart TB
    person[Person]
    camera[Camera or photo]
    api["POST api scan"]
    tess[Tesseract OCR]
    route{"Confidence"}
    mini["✦ gpt-4o-mini
Routing"]
    full["✦ gpt-4o
Routing"]
    review[Person corrects text]
    neo[Private Neo4j recipe]

    person --> camera --> api --> tess --> route
    route -->|70 percent plus| review
    route -->|40 to 70| mini --> review
    route -->|under 40| full --> review
    review -->|POST api scan save| neo

    classDef agent fill:#e8f0ee,stroke:#0f3d36,stroke-width:2px,color:#0d0d0d
    classDef service fill:#fafaf8,stroke:#c8c8c2,stroke-width:1px,color:#0d0d0d
    classDef actor fill:#f3eef8,stroke:#6b4f9a,stroke-width:2px,color:#0d0d0d
    class mini,full agent
    class camera,api,tess,route,review,neo service
    class person actor

Code: backend/app/scan.py, frontend/pages/scanner.tsx. Not a recipe → NotRecipe / refused before save.