deduplication

Fast, secure, efficient backup program

Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

Deduplicating archiver with compression and authenticated encryption.

Prometheus Alertmanager

A C library for parsing/normalizing street addresses around the world. Powered by statistical NLP and open geo data.

rustic - fast, encrypted, and deduplicated backups powered by Rust

Extremely fast tool to remove duplicates and other lint from your filesystem

Scalable data pre processing and curation toolkit for LLMs

Context intelligence layer for LLM agents: persistent memory with write-time dedup, sensitivity tagging, conflict detection, and hierarchical decay. ~12ms. No LLM calls. MIT.

A Multi Agent Memory MCP That Connect Agents Across Systems and Machines

Web Scraping, Document Deduplication & GPT-2 Fine-tuning with a newly created scam dataset.