Computer Vision · Local-First Software
Local-First Computer Vision: Indexing Photo Archives Without Cloud Surveillance
Design Thesis: Searching a 100,000-image archive by natural language (“civil engineering site inspection in Huambo during rainy season”) no longer requires uploading confidential media to a hyperscale cloud vendor. With 8-bit quantized vision-language encoders and embedded HNSW vector indexes, sub-180ms semantic retrieval runs entirely on commodity desktop and workstation hardware.
Contents
1. The privacy and bandwidth dilemma of cloud photo libraries
Law firms, engineering consultancies, medical clinics, journalists, and privacy-conscious individuals accumulate hundreds of gigabytes of photographic evidence and media archives. Finding a specific image six months later is nearly impossible when files are named `IMG_4829.JPG` inside nested date folders.
Consumer cloud photo services solve search by uploading every image to central servers, running proprietary object-detection and facial-clustering pipelines, and storing the unencrypted media indefinitely. For organisations subject to professional secrecy or operating on metered, high-latency internet connections, that trade-off fails on two counts:
- Confidentiality exposure: Uploading site-inspection photographs, identity documents, or client assets to third-party consumer clouds violates data-sovereignty policies and exposes the archive to unilateral account lockouts or policy scans.
- Upload bandwidth bottlenecks: Synchronising 250 GB of high-resolution RAW and JPEG files over an asymmetric corporate link in Luanda or regional offices saturates uplink capacity for weeks.
2. Quantized dual-encoder vision-language architectures
When we architected TICNES Photo Vault, we designed the indexing pipeline to run locally on the user’s machine — whether or not a discrete GPU is present. The core of semantic image search is a contrastive vision-language model (similar to CLIP or SigLIP) consisting of two aligned encoders:
- An Image Encoder that maps a decoded image tensor into a normalized 512- or 768-dimensional unit vector.
- A Multilingual Text Encoder (supporting both Portuguese and English queries) that maps a natural-language search phrase into the exact same vector space.
Because the image encoder only needs to run once per photo during background ingestion, whereas the text encoder runs interactively at query time, we apply asymmetric optimization:
| Component | Execution Frequency | Runtime Format | Typical Latency (CPU / NPU) |
|---|---|---|---|
| Image Encoder | Once per new file (background queue) | ONNX INT8 quantized vision transformer | `~110ms` per image on 8-core laptop CPU |
| Synthetic Captioner | Once per new file (idle worker) | Lightweight distilled VLM (optional) | `~380ms` per image (background thread) |
| Multilingual Text Encoder | Per user search keystroke / submit | ONNX FP16/INT8 text transformer | `< 18ms` per query |
| HNSW Vector Search | Per user search query | Memory-mapped HNSW + SQLite FTS5 | `< 12ms` across 100,000 vectors |
3. Combining semantic vectors, OCR and EXIF metadata
Pure vector similarity handles conceptual queries well (“sunset over cargo port”, “whiteboard diagram from architecture workshop”), but users frequently combine visual concepts with exact constraints: “receipt from July 2025 above 50,000 Kz” or “photos taken with Canon EOS in Benguela”.
To support compound queries without sending data to the cloud, our ingestion worker extracts three parallel representations for every file:
- Dense Visual Embedding: Stored in a local Hierarchical Navigable Small World (HNSW) graph index for cosine similarity search.
- On-Device Optical Character Recognition (OCR): Text visible inside the image (such as street signs, document headers, invoice totals, or slide titles) is extracted locally and indexed into a SQLite FTS5 full-text table with trigram tokenization.
- Structured EXIF & Filesystem Metadata: Capture timestamp, GPS coordinates (reverse-geocoded against an offline boundary table of municipalities and provinces), camera model, lens focal length, and user-assigned tags.
At query time, a lightweight local query parser separates temporal and geographic filters from the visual description, applies a fast SQLite bitmap pre-filter, and scores the remaining candidates using a weighted combination of cosine similarity and FTS5 lexical match.
4. Content-addressed incremental indexing and deduplication
Real-world photo folders are messy: users rename directories, copy folders across external drives, and save duplicate images from messaging apps. If renaming a folder forced the vision model to re-index 40,000 images from scratch, the software would be unusable.
We solve this by addressing every media asset by its cryptographic BLAKE3 content hash rather than its file path. When the filesystem watcher detects a moved or renamed file, it computes the BLAKE3 digest in under a millisecond, sees that the embedding and EXIF record already exist in the local database, and simply updates the path pointer. Exact duplicates and perceptual near-duplicates (computed via 64-bit difference hashes, `dHash`) are grouped automatically so search results stay clean.
5. Zero-knowledge encrypted backup optionality
Local-first does not mean sacrificing off-site backup. When users or teams choose to replicate their Photo Vault index and encrypted thumbnails to remote object storage, encryption happens prior to network transmission using AES-GCM-256 with keys derived on-device via `Argon2id`. The storage server sees only opaque ciphertext blocks; neither TICNES AI nor the storage provider can view the images, read the tags, or execute searches against the archive.