Repolode
Submit
microsoft

microsoft/genaiops-promptflow-template

A template and guidance for operating LLM-infused apps built with Prompt Flow on Azure AI Studio or Azure Machine Learning, covering experimentation, evaluation and deployment.

View on GitHub
Stars
369
Forks
277
Open issues
42
Last commit
2 months ago

What is it?

This project is an LLMOps template for teams building LLM-infused applications with Prompt Flow. It targets the inner loop of experimentation and evaluation, and the outer loop of deployment and inference, and it can run flows locally or on Azure.

The template supports Python function flows, class flows and YAML DAG flows, and it includes a pure Python evaluation path via the promptflow-evals package. Multiple flows and datasets can live in one repository, with variant and hyperparameter experiments, comparison reports in CSV and HTML, and A/B deployment on AzureML managed compute.

CI/CD orchestration is available through GitHub Actions, Azure DevOps and Jenkins. Deployment targets include Kubernetes, Azure Web Apps and AzureML managed compute, and a separate DataOps folder holds data pipelines that register datasets as Azure ML data assets.

Key features

  • 01Hosts multiple Prompt Flow flows in a single repository with a per-flow lifecycle.
  • 02Runs variant and hyperparameter experiments and generates CSV and HTML reports.
  • 03Supports A/B deployment on AzureML managed compute.
  • 04Integrates with GitHub Actions, Azure DevOps and Jenkins pipelines.
  • 05Deploys flows to Kubernetes, Azure Web Apps and AzureML managed compute.

Use cases

  • Comparing prompt variants and model hyperparameters on a shared evaluation dataset.
  • Moving an LLM app from local experimentation to a managed endpoint on Azure.
  • Setting up CI/CD gates that test and deploy prompt flows automatically.
  • Managing several LLM use cases such as classification, RAG chat and code generation in one repo.

Quick start

  1. 1

    Clone the repository from GitHub

    git clone https://github.com/microsoft/llmops-promptflow-template.git
  2. 2

    Create a .env file at the top level and add your connection details, such as an AzureOpenAI connection named aoai

Things to know

  • Flows in the examples expect an AzureOpenAI connection, so you need an Azure OpenAI API key and endpoint.
  • The README's local setup covers only AzureOpenAI as a provider, and it is focused on Azure AI Studio and Azure Machine Learning.
  • The license is not clearly stated (NOASSERTION), so check the license terms before reuse.

In collections

Testing frameworks & test automation tools

Topics

#llmops#prompt-engineering#azure#mlops#evaluation#llm#ci-cd#aistudio#azuremachinelearning#cloud

FAQ

What is genaiops-promptflow-template?
A template and guidance for operating LLM-infused apps built with Prompt Flow on Azure AI Studio or Azure Machine Learning, covering experimentation, evaluation and deployment. This project is an LLMOps template for teams building LLM-infused applications with Prompt Flow. It targets the inner loop of experimentation and evaluation, and the outer loop of deployment and inference, and it can run flows locally or on Azure.
Is genaiops-promptflow-template free to use? What license does it have?
genaiops-promptflow-template is open source under the Other license. Check the license terms before using it in commercial products.
Is genaiops-promptflow-template still maintained?
Updated within 6 months. The last commit to genaiops-promptflow-template was on Sep 15, 2026.
What can I use genaiops-promptflow-template for?
Comparing prompt variants and model hyperparameters on a shared evaluation dataset. Moving an LLM app from local experimentation to a managed endpoint on Azure. Setting up CI/CD gates that test and deploy prompt flows automatically. Managing several LLM use cases such as classification, RAG chat and code generation in one repo.
How do I install genaiops-promptflow-template?
Clone the repository from GitHub: git clone https://github.com/microsoft/llmops-promptflow-template.git Create a .env file at the top level and add your connection details, such as an AzureOpenAI connection named aoai
Where can I get genaiops-promptflow-template?
The source code and installation instructions are on GitHub: https://github.com/microsoft/genaiops-promptflow-template

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects