AI-operations

Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

A high-throughput and memory-efficient inference and serving engine for LLMs

Platform to programmatically author, schedule, and monitor workflows.

The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.

Workflow Engine for Kubernetes

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

Machine Learning Pipelines for Kubeflow

The Open Source Feature Store for AI/ML

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

A curated list of awesome projects and resources related to Argo (a CNCF graduated project)

Turns Data and AI algorithms into production-ready web applications in no time.

Always know what to expect from your data.

⚡️SwanLab - an open-source, modern-design AI training tracking and visualization tool. Supports Cloud / Self-hosted use. Integrated with PyTorch / Transformers / verl / LLaMA Factory / ms-swift / Ultralytics / MMEngine / Keras etc.

Label Studio is a multi-type data labeling and annotation tool with standardized output format

The absolute trainer to light up AI agents.

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

🐢 Open-Source Evaluation & Testing library for LLM Agents

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

Build, Manage and Deploy AI/ML Systems

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

Automated Machine Learning on Kubernetes

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database​.

Dynamic, resilient AI orchestration. Coordinate data, models, and compute as you build AI workflows.

Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD, evaluation, and observability.

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.

Learn how to develop, deploy and iterate on production-grade ML applications.

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.

Curated list of the best truly open-source AI projects, models, tools, and infrastructure. Daily updated.

Open-source inference server and production cluster for all the models your agent needs.

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

An orchestration platform for the development, production, and observation of data assets.

Maestro: Netflix’s Workflow Orchestrator

Community maintained hardware plugin for vLLM on Huawei Ascend

Python SDK for healthcare AI — typed, validated FHIR tools for agents, real-time EHR connectivity, production deployment ✨ 🏥

Distributed AI Model Training and LLM Fine-Tuning on Kubernetes

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

A comprehensive Python package template to kickstart and standardize your MLOps initiatives and data pipelines.

Lineage metadata API, artifacts streams, sandbox, API, and spaces for Polyaxon

Deploy any AI model, agent, database, RAG, and pipeline locally or remotely in minutes

An awesome & curated list of best LLMOps tools for developers

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.

🔥 Real-time NVIDIA GPU dashboard

LUPINE is a GPU over IP bridge allowing GPUs on remote machines to be attached to CPU-only machines.

🏕️ Reproducible development environment for humans and agents

Engine for AI/ML/Data tracking, visualization, explainability, drift detection, and dashboards for Polyaxon.

🤖 𝗟𝗲𝗮𝗿𝗻 for 𝗳𝗿𝗲𝗲 how to 𝗯𝘂𝗶𝗹𝗱 an end-to-end 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗿𝗲𝗮𝗱𝘆 𝗟𝗟𝗠 & 𝗥𝗔𝗚 𝘀𝘆𝘀𝘁𝗲𝗺 using 𝗟𝗟𝗠𝗢𝗽𝘀 best practices: ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 12 𝘩𝘢𝘯𝘥𝘴-𝘰𝘯 𝘭𝘦𝘴𝘴𝘰𝘯𝘴

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™ TPUv6e/v7/Trainium2/3 | 开源持续推理基准研究平台 — Kimi K3-Code、MiniMax M3、DeepSeekv4、GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72，即将推出™ TPUv6e/v7/Trainium2/3

AI Infrastructure Engineer Learning Track - Production ML infrastructure curriculum (2-4 years experience)

AI Infra / AI Orchestration / AI Control Plane

The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps best practices

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

A high-performance ML model serving framework, offers dynamic batching and CPU/GPU pipelines to fully exploit your compute machine

List of Deep Learning Cloud Providers

Notebooks, code samples, sample apps, and other resources that demonstrate how to use, develop and manage machine learning and generative AI workflows using Google Cloud Vertex AI.

Start building and deploying Python packages and Docker images for MLOps tasks.

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems techniques.

✨ Build a machine learning model from a prompt

Streamlining reinforcement learning with RLOps. State-of-the-art RL algorithms and tools, with 10x faster training through evolutionary hyperparameter optimization.

Open Source Data Annotation & Labeling Tools

End-to-end, code-first tutorials for building production-grade GenAI agents. From prototype to enterprise deployment.

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

🔥 An autonomous AI agent that runs your deep learning experiments 24/7 while you sleep. Zero-cost monitoring, Leader-Worker architecture, constant-size memory.

LightlyStudio - The Unified Data Platform for Multimodal ML

skops is a Python library helping you share your scikit-learn based models and put them in production

CAIPE is an open-source AI platform for building, governing, and operating AI agents and agentic workflows for platform engineering and beyond.

Argilla is a collaboration tool for AI engineers and domain experts to build high-quality datasets

Efficient Deep Learning Systems course materials

Google Cloud Platform Vertex AI end-to-end workflows for machine learning operations

This repo provides a customizable stack for starting new ML projects on Databricks that follow production best-practices out of the box.

Hyperparameter optimization and feature selection for scikit-learn using evolutionary algorithms. A modern alternative to GridSearchCV and RandomizedSearchCV.

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models

Machine Learning Engineering Open Book

Free MLOps course from DataTalks.Club. Register here 👇🏼 to get notified about the next cohort

Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters

Deliver safe & effective language models

A curated list of Artificial Intelligence Top Tools

A curated list of MLSecOps tools and resources for securing machine learning and AI systems - adversarial ML defense, LLM security, AI red teaming, model scanning, supply-chain protection, and MLOps pipeline security.

Aim 💫 — An easy-to-use & supercharged open-source experiment tracker.

Resources of our survey paper "Optimizing Edge AI: A Comprehensive Survey on Data, Model, and System Strategies"

ZKE（Z Kubernetes Engine）：AI 原生的 Kubernetes 云操作环境，桌面式多集群控制台加受控 AIOps Agent，基于 Server + Agent 与 QUIC/mTLS，适用于私有云、混合云及边缘环境

OpenMLDB is an open-source machine learning database that provides a feature platform computing consistent features for training and inference.

Learn how to create, develop, and maintain a state-of-the-art MLOps code base

A collection of scientific methods, processes, algorithms, and systems to build stories & models.

cube studio开源云原生一站式机器学习/深度学习/大模型AI平台，mlops算法链路全流程，算力租赁平台，notebook在线开发，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务VGPU虚拟化，边缘计算，标注平台自动化标注，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模型多机推理，私有知识库，AI模型市场，支持国产cpu/gpu/npu 昇腾生态，支持RDMA，支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano等分布式

:sunglasses: A curated list of awesome MLOps tools

👕 Open-source course on architecting, building and deploying a real-time personalized recommender for H&M fashion articles.

🏬 modelstore is a Python library that allows you to version, export, and save a machine learning model to your filesystem or a cloud storage provider.

Synapsa Commons: free, hands-on AI courses that run anywhere (Colab, Kaggle, Binder, Codespaces, Jupyter). Build EU AI Act conformity evidence, validate models like a risk committee, price predictive-maintenance alarms, measure document extraction, and make a simulated humanoid walk. Every lesson autograded. From the team building Synapsa.

🤖 MLE-Agent: Your intelligent companion for seamless AI engineering and research. 🔍 Integrate with arxiv and paper with code to provide better code/research plans 🧰 OpenAI, Anthropic, Gemini, Ollama, etc supported. :fireworks: Code RAG

Collective Knowledge (CK), Collective Mind (CM/CMX) and MLPerf automations: community-driven projects to learn how to run AI, ML, and other emerging workloads more efficiently and cost-effectively across diverse models, datasets, software, and hardware using MLPerf methodology and benchmarks.

Monitor the stability of a Pandas or Spark dataframe ⚙︎

☁️ Build multimodal AI applications with cloud-native stack

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台，算法全链路流程，多租户，算力租赁平台，token中转，拖拉拽任务流pipeline编排，多机多卡分布式训练，超参搜索，推理服务，VGPU虚拟化，云边端协同，边缘计算，自动化标注平台，deepseek等大模型sft微调/奖励模型/强化学习训练，vllm/ollama/mindie大模型多机推理，私有知识库llmops智能体，AI模型市场，支持国产异构算力调度,昇腾/寒武纪/海光/摩尔/沐曦等，支持ib/roce/RDMA，信创支持

Safety-first guardrails for AI-driven cloud and Kubernetes operations

PyTorch Lightning + Hydra. A very user-friendly template for ML experimentation. ⚡🔥⚡

MONAI Deploy App SDK offers a framework and associated tools to design, develop and verify AI-driven applications in the healthcare imaging domain.

Hopsworks - Data-Intensive AI platform with a Feature Store

Superduper: End-to-end framework for building custom AI applications and agents.

NucliaDB, The AI Search database for RAG

BharatMLStack is an open-source, end-to-end machine learning infrastructure stack built at Meesho to support real-time and batch ML workloads at Bharat scale

Azure MLOps (v2) solution accelerators. Enterprise ready templates to deploy your machine learning models on the Azure Platform.

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

Deprecated historical repo. Superlinked now develops SIE, a self-hosted inference engine for embeddings, reranking, OCR, extraction, and document processing.

A curated list of open source tools used in analytics platforms and data engineering ecosystem

Examples for Cerebrium Serverless GPUs

MLOps for DevOps Engineers - A hands-on, project-based guide to Machine Learning Operations

Open-source, low-code AutoML platform for Python. PyCaret 4.0: sklearn-native engine + React control plane.

nannyml: post-deployment data science in python

Ollama for classical ML models. AOT compiler that turns XGBoost, LightGBM, scikit-learn, CatBoost & ONNX models into native C99 inference code. One command to load, one command to serve. 336x faster than Python inference.

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

AIShield Watchtower: Dive Deep into AI's Secrets! 🔍 Open-source tool by AIShield for AI model insights & vulnerability scans. Secure your AI supply chain today! ⚙️🛡️

Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.

Distributed Machine Learning Patterns from Manning Publications by Yuan Tang https://bit.ly/2RKv8Zo

The Virtual Feature Store. Turn your existing data infrastructure into a feature store.

Next Generation Experimental Tracking for Machine Learning Operations

ML/AI meta-model, used in MLRun/Iguazio/Nuclio, see qgate-sln-<MLRun | solution>

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

🧠 Motorhead is a memory and information retrieval server for LLMs.

PostgreSQL vector database extension for building AI applications

Cookiecutter template for FastAPI projects using: Machine Learning, uv, Github Actions and Pytests

MONAI Deploy aims to become the de-facto standard for developing, packaging, testing, deploying and running medical AI applications in clinical production.

The Fuzzy Labs guide to the universe of open source MLOps

Learn how to design, develop, deploy and iterate on production-grade ML applications.

A curated list of references for MLOps

Making data lake work for time series

mRMR (minimum-Redundancy-Maximum-Relevance) for automatic feature selection at scale.

🤖Self-Modifying Framework from the Future 🔮 World's First AMS

Feathr – A scalable, unified data and AI engineering platform for enterprise

An open-source ML pipeline development platform

부스트캠프 AI Tech - Product Serving 자료

MLOps examples

Deploy a ML inference service on a budget in less than 10 lines of code.

Boosting your Web Services of Deep Learning Applications.

This is an example of a Containerized Flask Application that can deploy to many target environments including: AWS, GCP and Azure.

Train and Deploy an ML REST API to predict crypto prices, in 10 steps

🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.

Coarse-grained lineage and tracing for machine learning pipelines.

🌀 𝗧𝗵𝗲 𝗙𝘂𝗹𝗹 𝗦𝘁𝗮𝗰𝗸 𝟳-𝗦𝘁𝗲𝗽𝘀 𝗠𝗟𝗢𝗽𝘀 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 | 𝗟𝗲𝗮𝗿𝗻 𝗠𝗟𝗘 & 𝗠𝗟𝗢𝗽𝘀 for free by designing, building and deploying an end-to-end ML batch system ~ 𝘴𝘰𝘶𝘳𝘤𝘦 𝘤𝘰𝘥𝘦 + 2.5 𝘩𝘰𝘶𝘳𝘴 𝘰𝘧 𝘳𝘦𝘢𝘥𝘪𝘯𝘨 & 𝘷𝘪𝘥𝘦𝘰 𝘮𝘢𝘵𝘦𝘳𝘪𝘢𝘭𝘴

Serverless Machine Learning Course for building AI-enabled Prediction Services from models and features

CDF SIG MLOps

Aqueduct is no longer being maintained. Aqueduct allows you to run LLM and ML workloads on any cloud infrastructure.

deployKF builds machine learning platforms on Kubernetes. We combine the best of Kubeflow, Airflow†, and MLflow† into a complete platform.

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

Complete deep learning project developed in Full Stack Deep Learning, 2022 edition. Generated automatically from https://github.com/full-stack-deep-learning/fsdl-text-recognizer-2022

A Full Stack ML (Machine Learning) Roadmap involves learning the necessary skills and technologies to become proficient in all aspects of machine learning, including data collection and preprocessing, model development, deployment, and maintenance.

🍫 Example code for a basic ML Platform based on Pulumi, FastAPI, DVC, MLFlow and more

The open source, end-to-end computer vision platform. Label, build, train, tune, deploy and automate in a unified platform that runs on any cloud and on-premises.

Slides, scripts and materials for the Machine Learning in Finance Course at NYU Tandon, 2022

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

A booklet on machine learning systems design with exercises. NOT the repo for the book "Designing Machine Learning Systems", which is `dmls-book`

CodeProject.AI Server is a self contained service that software developers can include in, and distribute with, their applications in order to augment their apps with the power of AI.

Smoothly Manage Multiple LLMs (OpenAI, Anthropic, Azure) and Image Models (Dall-E, SDXL), Speed Up Responses, and Ensure Non-Stop Reliability.

A Survey of AI startups

A Data Engineering & Machine Learning Knowledge Hub

This repository is a curated collection of links to various courses and resources about Artificial Intelligence (AI)

Just enough Kubernetes for you to fly

:closed_book: Clarity in the current fast-paced mess of Open Source innovation

The collaboration workspace for Machine Learning

Construct a modern data stack and orchestration the workflows to create high quality data for analytics and ML applications.

ModelFox makes it easy to train, deploy, and monitor machine learning models.

This repository offers a goldmine of materials for students of computer vision, natural language processing, and machine learning operations.

Notes for Machine Learning Engineering for Production (MLOps) Specialization course by DeepLearning.AI & Andrew Ng