pyspark

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

State of the Art Natural Language Processing

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

A Comprehensive Framework for Building End-to-End Recommendation Systems with State-of-the-Art Models

A free, self-paced 24-week AI engineering course: Python, machine learning, LLMs, RAG, fine-tuning, agents and MCP, Azure and Vertex and Bedrock, and Databricks. 43 runnable notebooks, one continuous case study. MIT licensed, no signup. By Zorost Intelligence AI Lab.

AI Data Engineering Healthcare Platform: PySpark Medallion Lakehouse, Airflow DAGs, 5 ML Models, Local LLM RAG, and FastAPI Clinical Architecture.

The Practical Machine Learning Course focuses on equipping learners with the essential skills and tools needed to apply machine learning techniques in real-world scenarios.