apache-airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

Elyra extends JupyterLab with an AI centric approach.

AI agent tooling for data engineering workflows.

⚡ Control Apache Airflow with natural language via MCP. Chat with your workflows using Claude, GPT, or any LLM — no REST API calls needed. Supports Airflow 2.x (43 tools) & 3.0+ (45+ tools).

AI Data Engineering Healthcare Platform: PySpark Medallion Lakehouse, Airflow DAGs, 5 ML Models, Local LLM RAG, and FastAPI Clinical Architecture.

📊 ETL pipeline for YouTube competitor analytics. Orchestrated with Airflow, Docker, and AWS. Features sentiment analysis and a Power BI dashboard.