Local inference
Ollama ↗
A local runtime for downloading, serving and testing a broad set of open models.
Open Source Radar
GitHub Radar
A reviewed watchlist for repositories that matter to the open-model ecosystem. Star counts and repository update times make it easy to compare the projects readers are watching.
Model framework
Transformers ↗
Hugging Face’s model-definition library for training and inference across text, vision and audio.
AI interface
Open WebUI ↗
A self-hosted interface for working with Ollama and OpenAI-compatible model APIs.
Agent framework
LangChain ↗
A widely used platform for building applications and agent workflows around language models.
Generative media
ComfyUI ↗
A node-based graphical interface and backend for modular diffusion-model workflows.
Local inference
llama.cpp ↗
A compact C/C++ implementation for efficient local LLM inference across many devices.
Inference serving
vLLM ↗
A high-throughput, memory-efficient serving engine for large language and multimodal models.
RAG & agents
RAGFlow ↗
A retrieval-augmented generation engine that combines document retrieval with agent capabilities.
Fine-tuning
LLaMA-Factory ↗
A unified toolkit for fine-tuning and evaluating a broad range of language and vision-language models.
Agent framework
AutoGen ↗
Microsoft’s framework for programming multi-agent applications and tool-driven workflows.
RAG & data
LlamaIndex ↗
A framework for document agents, retrieval workflows and connecting LLMs to data.
Vector database
Milvus ↗
A cloud-native vector database for scalable approximate-nearest-neighbour search.
Training & inference
DeepSpeed ↗
A distributed deep-learning optimization library for efficient training and inference.
Observability
Langfuse ↗
An open AI-engineering platform for LLM evaluation, observability and prompt management.
Inference serving
SGLang ↗
A high-performance serving framework for large language and multimodal models.
Vector database
Chroma ↗
Open search infrastructure for building AI applications around embeddings and retrieval.
Agent framework
Semantic Kernel ↗
Microsoft’s SDK for integrating LLMs, memory and tool use into applications.
Apple silicon
MLX ↗
Apple’s array framework for machine learning research and efficient on-device model work.
RAG & agents
Haystack ↗
A modular orchestration framework for production-ready retrieval and agent pipelines.
Fine-tuning
PEFT ↗
Hugging Face’s parameter-efficient fine-tuning library for adapting foundation models.
Inference serving
TensorRT-LLM ↗
NVIDIA’s optimized runtime and APIs for efficient LLM inference on its GPUs.
Training & inference
LitGPT ↗
A collection of high-performance LLM recipes for pretraining, fine-tuning and deployment.
Inference serving
OpenLLM ↗
A server for exposing open-source models through OpenAI-compatible cloud APIs.
Inference serving
Text Generation Inference ↗
Hugging Face’s production text-generation server for deploying large language models.
Inference serving
Triton Inference Server ↗
NVIDIA’s optimized cloud and edge serving platform for a range of model frameworks.
Evaluation
OpenCompass ↗
An LLM evaluation platform spanning models and more than one hundred datasets.