cpu

Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

Snap any video URL or audio file into plaintext. No GPU. No cloud. One command.

Easy-Translate is a script for translating large text files with a SINGLE COMMAND. Easy-Translate is designed to be as easy as possible for beginners and as seamlesscustomizable and as possible for advanced users.

Gradio based tool to run opensource LLM models directly from Huggingface

A LLaMA2-7b chatbot with memory running on CPU, and optimized using smooth quantization, 4-bit quantization or Intel® Extension For PyTorch with bfloat16.

Development and deployment on AWS of a question-answer LLM model using Llama2 with 7B parameters and RAG with LangChain