data-engineering

Apache Superset is a Data Visualization and Data Exploration Platform

Learn how to develop, deploy and iterate on production-grade ML applications.

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

Event Driven Orchestration & Scheduling Platform for Mission Critical Applications

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

Turns Data and AI algorithms into production-ready web applications in no time.

Workflow Engine for Kubernetes

An orchestration platform for the development, production, and observation of data assets.

The Data Engineering Cookbook

Always know what to expect from your data.

Incremental engine for long horizon agents 🌟 Star if you like it!

Graph-Native Infrastructure for Context and Accountable AI Systems

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

🧙 Build, run, and manage data pipelines for integrating and transforming data.

Fancy stream processing made operationally mundane

Open Source Feature Flags, Experimentation, and Product Analytics

The Open Source Feature Store for AI/ML

LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows