parquet

QuestDB is a high performance, open-source, time-series database

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Official Rust implementation of Apache Arrow

Create full-fledged APIs for slowly moving datasets without writing a single line of code.

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Parseable is an open source, unified infrastructure observability platform built in Rust on a data lake architecture. It tracks logs, metrics, traces, and events across apps, agents, and systems, reducing storage costs by up to 90% through columnar telemetry compression.

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

One SQL interface for 60+ tools (e.g., GitHub, Notion, Airtable). Plug into any LLM through MCP.

Comprehensive Vector Data Tooling. The universal interface for all vector database, datasets and RAG platforms. Easily export, import, backup, re-embed (using any model) or access your vector data from any vector databases or repository.

Fast search engine on object storage, with full text search, vectors, and SQL, natively on Parquet.

Vector search using only Parquet and DataFusion

Lightweight representation engineering dataflow operations for agent developers.

Harvest-first Rust toolkit for open data metadata: sync 300+ portals (CKAN, DCAT, SPARQL, data.json, Socrata, OpenDataSoft, ArcGIS, CSW, STAC ) into PostgreSQL, add optional local embeddings and semantic search. Ceres Open Data Index (2M+ datasets) on Hugging Face.