Ellie AI

Ellie AI

The State-of-the-Art AI Video Analysis Agent

Upload any video, ask anything. Gemini 2.5 Flash + Whisper + React 19 + tRPC 11. Neural Noir design.

Why this exists · Capabilities · Architecture · Stack · Run Locally · Operations

Marketing landing page (built with Lovable, in the separate private repo Alexi5000/ellie-techtide): ellie-techtide.lovable.app

Why this exists

What it does. Ellie accepts a video upload (MP4, WebM, MOV, or AVI up to 100 MB), sends it to Gemini 2.5 Flash through a server-side provider gateway to produce timestamped summaries, scenes, frames, audio events, and emotions, persists those results with Drizzle and MySQL, and answers follow-up questions in a chat panel using the stored analysis as context. Voice input is transcribed with a Whisper-compatible endpoint. The browser talks to the server only through typed tRPC 11 procedures.

Who it serves. People who need to understand recorded media without scrubbing timelines by hand, and maintainers who want a small, typed React 19 + Express + tRPC reference application with environment validation, health and readiness probes, and Docker packaging already in place.

Why Alex built it. Alex Cinovoj, Founder & CTO of TechTide AI, created the repository on 2025-04-26. The first product under this name was "Ellie Voice Receptionist", an AI voice assistant for law firms (commit 1229a5d1, 2025-07-16). On 2026-02-18 the project was rebuilt as a video analysis agent (commit 306a265f, "Complete rebuild: Ellie AI Video Analysis Agent v2.0", moving from Express REST and Socket.io to tRPC 11, React 19, Vite 7, Gemini 2.5 Flash, and Whisper). The next day it was made a public, anonymous application with ephemeral browser-side chat state, rate limiting, and strict input validation (commit 12e32471, 2026-02-19). Pull request #1 (2026-05-17) hardened it for production review with readiness endpoints, environment validation, and Docker packaging. The design intent recorded in this README since that rewrite is deliberately Neural Noir: dark, cinematic, fast, and agentic without hiding the engineering underneath.

Real-world use case (2025-2026 evidence). Proposed use case. A content or training team uploads a recorded webinar, lets Ellie produce a timestamped outline, and asks "where does the presenter cover pricing?" to jump straight to the right segment. The repository contains the routes, schema, and validation for this flow and a Vitest suite that exercises input validation and the anonymous upload capability model; it does not contain recorded provider responses or golden-video fixtures, so a live Gemini run is required to confirm answer quality. Gemini 2.5 Flash became generally available in June 2025 with multimodal input and configurable thinking budgets.[^1] tRPC v11, the typed RPC layer between the browser and the Express server, shipped as stable in March 2025.[^2] The client runs on the React 19.2 line released in October 2025[^3] and is bundled with Vite 7, released in June 2025.[^4]

Capabilities

Ellie’s README is product-forward, but the repository is not a mockup. The core routes implement upload, multimodal analysis, chat, transcription, session helpers, and operational readiness endpoints.

Architecture

Ellie currently uses a heavy-client, thin-orchestration backend model. The browser owns the immediate product feel and ephemeral conversation memory. The backend owns the safe boundary for uploads, storage, provider calls, persistence, rate limits, and readiness checks. This keeps the agent surface responsive while preserving a clean path to queue workers, durable conversations, and multi-tenant orchestration later.

graph LR
    User["User"] --> Client["React 19 + Vite\nNeural Noir client"]
    Client --> TRPC["tRPC 11\ntyped calls"]
    TRPC --> Express["Express + Node\nthin orchestration"]
    Express --> Storage["S3-compatible\nmedia storage"]
    Express --> DB["Drizzle + MySQL\nvideo + analysis records"]
    Express --> Gemini["Gemini 2.5 Flash\nmultimodal analysis + chat"]
    Express --> Whisper["Whisper\nspeech-to-text"]
    Express --> Ops["Health + readiness\nenv validation"]

Stack

Ellie is TypeScript across the product boundary. The stack is modern, but intentionally familiar: React and Vite for the browser, Express and tRPC for the API, Drizzle and MySQL for persistence, and Forge-compatible provider helpers for Gemini and Whisper.

Run Locally

The committed pnpm-lock.yaml is the canonical dependency graph; root npm installs are unsupported because they omit the required wouter patch. The default path is a single pnpm workflow from the repository root. Configure environment values first, install with the lockfile, then run the TypeScript server in development mode.

gh repo clone Alexi5000/Ellie
cd Ellie
pnpm install --frozen-lockfile
cp .env.example .env
pnpm validate:env
pnpm dev

Environment

Ellie keeps secrets server-side. The browser calls typed routes; the backend talks to storage, the database, Gemini, and Whisper through environment-managed credentials.

Repository Map

The repository is small enough to navigate directly and structured enough to survive production hardening. The important split is simple: client/ is the experience, server/ is the orchestration boundary, shared/ is the contract surface, and drizzle/ is the data plane.

Operations

Ellie is prepared for deployment review, not just local screenshots. The repository includes a root Dockerfile, Compose file, CI-compatible scripts, environment validation, and liveness/readiness endpoints. Some GitHub workflow checks depend on repository-level secrets and permissions, so local validation remains the authoritative green gate until those secrets are configured.

Product Roadmap

Ellie’s current shape is intentionally lean: strong browser experience, typed backend orchestration, persisted analysis, and real provider calls. The next step is deeper production orchestration.

Documentation

The root README is the front door. The rest of the documentation exists for maintainers who need setup, release, deployment, and testing detail without bloating the product narrative.