big-data

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

Apache IoTDB

Data-Centric Pipelines and Data Versioning

Apache Hive

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

Apache Calcite

Apache Ignite

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

🚄 FASTJSON2 is a Java JSON library with excellent performance.

Data Science Roadmap from A to Z

Extensible SQL Lexer and Parser for Rust

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

A @ClickHouse fork that supports high-performance vector search and full-text search.

My journey to learn and grow in the domain of Machine Learning and Artificial Intelligence by performing the #100DaysofMLCode Challenge. Now supported by bright developers adding their learnings :+1:

Easily convert common crawl to a dataset of caption and document. Image/text Audio/text Video/text, ...

A high-performance RDMA distributed file system for fast LLM Inference and GPU Training.