scraper

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Write web scrapers in Ruby using a clean, AI-assisted DSL. Kimurai uses AI to figure out where the data lives, then caches the selectors and scrapes with pure Ruby. Get the intelligence of an LLM without the per-request latency or token costs.

Open source web infrastructure for AI. Scrape, crawl, and automate the web, clean markdown, browser sessions, ready for your agents.

ScraperAI is an open-source, AI-powered tool designed to simplify web scraping for users of all skill levels.

Resume_Builder_AIHawk is a powerful Python tool that allows you to automatically customize your resume based on a job URL, ensuring it perfectly aligns with the job requirements and skills. With an interactive command-line interface, this tool makes it easy to navigate through options and select from various pre-defined styles

📚 This is an adapted version of Jina AI's Reader for local deployment using Docker. Convert any URL to an LLM-friendly input with a simple prefix http://127.0.0.1:3000/https://website-to-scrape.com/

Turn Webpage to LLM friendly input text. Similar to Firecrawl and Jina Reader API. Makes RAG, AI web scraping, image & webpage links extraction easy.

Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.

wxpath - declarative web crawling with XPath; a Web Query Language (WQL)

ScrapeGPT is a RAG-based Telegram bot designed to scrape and analyze websites, then answer questions based on the scraped content. The bot utilizes Retrieval Augmented Generation and webscraping to return natural language answers to the user's queries.

Reddit_Commentator_AIHawk is a Python project showcasing the power of artificial intelligence in social media interaction. This tool demonstrates AI's capability to generate contextually relevant Reddit comments using GPT models and LangChain. It automates the process of analyzing trending posts and creating engaging responses.