How to deploy azure devops in a production and non-production tenant

Of course. Deploying Azure DevOps (now called Azure DevOps Services for the cloud offering) across production and non-production environments is a common and critical scenario. The key is to manage isolation, security, and the flow of work and code between environments.

Here is a comprehensive guide covering the two primary strategies and best practices.

Core Concept: Tenants vs. Organizations

First, let’s clarify the terminology you used:

• Azure AD Tenant: A dedicated and trusted instance of Azure Active Directory. An organization typically has one tenant.
• Azure DevOps Organization: A container for your projects, pipelines, users, and groups. It exists within the Azure DevOps service and is associated with an Azure AD tenant.

You likely don’t want two separate Azure AD Tenants for prod and non-prod (as this creates identity management complexity). Instead, you want two separate Azure DevOps Organizations within the same tenant for isolation.

Recommended Structure:

• Azure AD Tenant: yourcompany.com (Single source of truth for identities)
• Non-Prod Azure DevOps Org: https://dev.azure.com/yourcompany-nonprod
• Prod Azure DevOps Org: https://dev.azure.com/yourcompany-prod

Strategy 1: The Isolated Organization Model (Most Common & Secure)

This model creates two completely separate Azure DevOps organizations. This provides the highest level of isolation and is the recommended approach for most enterprises.

Step 1: Setup and Isolation

1. Create Two Organizations:
   • Go to https://aex.dev.azure.com/.
   • Create your non-production organization (e.g., yourcompany-nonprod).
   • Create your production organization (e.g., yourcompany-prod).
   • Both will be connected to your single company Azure AD tenant.
2. Configure User and Group Access:
   • In each organization, go to Organization Settings > Users.
   • Add users and groups from your Azure AD.
   • Principle of Least Privilege: Grant users access only to the orgs they need.
     • Developers: Access to non-prod, maybe read-only to prod.
     • Release Managers/Ops: Access to both, with higher permissions in prod.
3. Create Projects:
   • Recreate the necessary projects (e.g., eCommerce-App) in both the non-prod and prod organizations.

Step 2: Source Code Management (The “How” of Code Promotion)

You have two main patterns for code:

1. A) Mono-Repo Pattern:

• You have a single, large repository in the non-prod org.
• Promotion Mechanism: Use Azure DevOps Pipelines to build the code from the non-prod repo and then release the artifact (e.g., a Docker image, a .zip file) to the production organization’s pipelines.
• How: The prod pipeline is triggered to deploy when a new artifact is available. The prod org never has a direct git connection to the non-prod org.

1. B) Multi-Repo (Fork) Pattern:

• The production organization contains the “golden” repository.
• Developers fork this repository into a project in the non-production organization (or into their own forks) to do their work.
• Promotion Mechanism: They create Pull Requests from their fork (in non-prod) back to the main repository (in prod). This is similar to the open-source GitHub model.
• How: The prod org’s main repo has branch policies that trigger build and validation pipelines upon PR completion.

Step 3: Pipeline and Artifact Management (The “How” of Deployment)

This is the most crucial part. You never let a pipeline in non-prod deploy directly to production infrastructure.

1. Build Pipeline (Non-Prod Org):
   • Located in the non-prod organization.
   • Triggers on commits to the main/develop branch.
   • Jobs: Compiles code, runs unit tests, and most importantly, publishes an artifact.
   • Publish Artifact to: You have two excellent options:
     • Azure DevOps Artifacts Feed (Recommended): Create a feed in your non-prod org. The build pipeline publishes the package (e.g., npm, NuGet, Universal Package) here.
     • Azure Blob Storage: A simple “drop” location for the final build outputs.
2. Release Pipeline (Prod Org):
   • Located in the production organization. This is key for security.
   • Trigger: Listens for a new artifact version from the non-prod Artifact Feed or storage location.
   • Jobs:
     • Download Artifact: Pulls the approved artifact from the non-prod source.
     • Deploy Stages: Has stages like QA, Staging, Production. It deploys the same, immutable artifact that was built and tested in non-prod. This guarantees what you tested is what gets deployed.

Step 4: Connecting the Orgs Securely

The prod org’s pipeline needs permission to download artifacts from the non-prod org.

1. Create a Service Principal (SPN) in Azure AD: This will be the identity the prod pipeline uses.
2. Grant the SPN Access:
   • In the non-prod Azure DevOps org, go to the Artifact Feed settings.
   • Add the SPN as a reader/user with appropriate permissions to read and list packages.
3. Create a Service Connection in Prod:
   • In the prod Azure DevOps org, create a new Service Connection (type: Azure Resource Manager or Generic).
   • Configure it to authenticate using the SPN’s Client ID and Secret.
4. Use the Connection in Pipeline:
   • In your YAML or Classic release pipeline, use this service connection in the task that downloads the artifact (e.g., DownloadPackage or AzureFileCopy task).

Strategy 2: The Single Organization & Project Model (Simpler, Less Isolation)

This model uses a single Azure DevOps organization and uses project-level isolation and branching strategies to separate environments.

• Structure:
  • One Org: https://dev.azure.com/yourcompany
  • One Project: eCommerce-App (or separate projects: eCommerce-App-Proj, eCommerce-App-Dev)
• Isolation: Achieved through:
  • Permissions: Strict user permissions on branches, release pipelines, and variable groups.
  • Branches: main branch for production, develop for non-production.
  • Variable Groups: Separate, secured variable groups for prod and non-prod settings (e.g., connection strings, secrets).
• Pipelines: A single pipeline can be designed with multiple stages (Dev -> QA -> Prod) with manual approval gates before production.
• When to Use: For smaller teams or less critical applications where strict isolation is not a regulatory requirement. It’s simpler but has a higher risk of accidental misconfiguration.

Best Practices for Both Strategies

1. Infrastructure as Code (IaC): Use Terraform or Azure Bicep/ARM templates to define your environments. Store these templates in Azure Repos and deploy them via pipelines to ensure consistency between non-prod and prod.
2. Secrets Management: NEVER store secrets in pipeline code. Use:
   • Azure Key Vault: The gold standard. Store secrets in a non-prod Key Vault and a prod Key Vault. Pipelines retrieve secrets at runtime.
   • Azure DevOps Variable Groups: Link them to Key Vault for the best of both worlds.
3. Mandatory Pull Requests and Branch Policies: Enforce code reviews and automated builds on the main branch in both orgs to protect quality.
4. Pipeline Templates: If using a single org, or even across orgs (by importing them), use YAML templates to share common pipeline steps and ensure consistency.
5. Monitoring and Logging: Integrate your pipelines with Azure Monitor and Application Insights to track deployments and their health.

Summary Table

For a production-grade setup, Strategy 1 (Isolated Organization Model) is overwhelmingly the recommended and safest approach. It provides the clear separation of duties and security boundaries necessary for managing critical environments.

xxx

how to deploy Azure devops across a prod tenant with a prod Entra ID instance and a non-prod tenant with non-prod Entra ID instance

Of course. This scenario—deploying Azure DevOps across two completely isolated tenants (prod and non-prod) with separate Entra ID (Azure AD) instances—is a strict requirement in highly regulated industries or for organizations with mandatory tenant-level separation.

This architecture is more complex due to the identity and network isolation. The core challenge is enabling secure, automated workflows between two tenants that have no inherent trust relationship.

Here is a detailed guide on how to design and implement this.

Core Architecture Principle

The guiding principle is: The Production Azure DevOps organization must be isolated within the Production Tenant. It should have no direct write access back to the Non-Prod tenant. Code and artifacts are promoted from Non-Prod to Prod in a one-way, controlled manner.

Recommended Structure:

• Non-Prod Tenant (company-nonprod.com)
  • Entra ID: nonprod.onmicrosoft.com
  • Azure DevOps Org: https://dev.azure.com/company-nonprod
  • Subscription: Non-Prod Subscription(s)
  • Key Vault: Non-Prod Key Vault
• Production Tenant (company-prod.com)
  • Entra ID: prod.onmicrosoft.com
  • Azure DevOps Org: https://dev.azure.com/company-prod
  • Subscription: Production Subscription(s)
  • Key Vault: Prod Key Vault

Step 1: Setup the Azure DevOps Organizations

1. Non-Prod Azure DevOps Org:
   • Log into the Azure Portal for your Non-Prod Tenant.
   • Ensure you are in the correct directory.
   • Navigate to Azure DevOps organizations and create https://dev.azure.com/company-nonprod.
   • This org will be backed by the Non-Prod Entra ID. All developers will be members here.
2. Production Azure DevOps Org:
   • Log into the Azure Portal for your Production Tenant.
   • Switch your directory to the Prod Entra ID.
   • Create the production organization: https://dev.azure.com/company-prod.
   • This org will be backed by the Prod Entra ID. Access will be highly restricted (e.g., only to release managers and automation identities).

Step 2: Establish a Secure “Bridge” Between Tenants

This is the most critical step. You need a secure service principal in the Prod tenant that has limited, specific permissions to access resources in the Non-Prod tenant (specifically, the artifact feed).

The best way to do this is via a Service Principal (App Registration) that exists in the Non-Prod Tenant and is granted access to the required resource. The Prod pipeline will then use a client secret to authenticate as that identity.

1. A) Create the Bridge Identity in the Non-Prod Tenant

1. In the Non-Prod Azure Portal, go to Entra ID > App registrations.
2. Create a new registration (e.g., azdo-prod-bridge). Note down its Application (Client) ID and Directory (Tenant) ID.
3. Generate a Client Secret (under Certificates & secrets). Note this down securely—you will need it for the Prod pipeline.
4. Grant this App Access to the Artifact Feed:
   • In your Non-Prod Azure DevOps org, go to your Artifact Feed.
   • Go to Feed Settings > Permissions.
   • Click Add Users/Groups.
   • Search for the App by its name (azdo-prod-bridge). It should appear.
   • Assign it the Reader role. This allows the Prod pipeline to read and download packages from this feed.

1. B) Configure the Production Pipeline to Use the Bridge Identity

1. In your Production Azure DevOps project, go to Project Settings > Service connections.
2. Create a new service connection of type Generic.
3. Fill in the details:
   • Server URL: The URL to your Non-Prod Azure DevOps Artifact Feed (e.g., https://pkgs.dev.azure.com/company-nonprod/_packaging/MyFeed/nuget/v3/index.json). For other types (like Universal Packages), the URL will differ.
   • Username/Email: The Application (Client) ID from step A.2.
   • Password/Token: The Client Secret from step A.3.
4. Name the connection (e.g., nonprod-artifacts-feed) and create it.

Now, your Prod pipeline can use this Generic service connection in a NuGetAuthenticate or DownloadPackage task to securely pull artifacts from the Non-Prod tenant.

Step 3: Design the Workflow (Code & Artifact Flow)

The pattern is a one-way promotion: Build once in Non-Prod, promote the immutable artifact to Prod.

In the Non-Prod Tenant (company-nonprod):

1. Code: Developers work on code in repositories in the Non-Prod org.
2. CI Pipeline: A build pipeline triggers on commits to main. It:
   • Compiles code.
   • Runs unit/integration tests.
   • Publishes a build artifact (e.g., a Docker image to a Container Registry in the Non-Prod subscription, or a NuGet package to the Azure Artifacts feed in the Non-Prod org).
3. Artifact Ready: The final, versioned artifact (e.g., myapp:1.0.45) is now stored in the Non-Prod ecosystem.

In the Production Tenant (company-prod):

1. CD Pipeline: A release pipeline in the Prod org is triggered manually or by a schedule. It:
   • Uses the nonprod-artifacts-feed service connection to download the specific, approved artifact version from the Non-Prod feed.
   • Deploys that exact artifact to the production infrastructure (which is in the Prod subscription).
2. IaC: Infrastructure changes are managed similarly. Terraform or Bicep code is developed and tested in Non-Prod. The compiled IaC artifacts (e.g., Terraform plan files, Bicep templates) are published as artifacts and then consumed and applied by the Prod pipeline.

Visual Workflow

Code

Key Considerations and Best Practices

1. Identity is the Perimeter: The Service Principal (azdo-prod-bridge) is a critical security boundary. Protect its Client Secret in Azure Key Vault within the Prod tenant and reference it in your pipeline using a variable group linked to Key Vault. Do not store the secret in plain text.
2. Network Security: If your artifacts are in a private network (e.g., an internal Azure Container Registry), you will need additional networking like Azure Virtual Network Peering (which requires global virtual network connectivity) or a network solution like Azure VPN Gateway/ExpressRoute. This adds significant complexity.
3. Alternative: Download via Secure File Copy: Instead of direct feed integration, your Non-Prod pipeline could place the final artifact in a secure, temporary blob storage container in the Non-Prod tenant. The Prod pipeline could then use the same service principal concept with AzureFileCopy to pull it. The pattern is the same.
4. Auditing and Governance:
   • Enable Auditing in both Azure DevOps organizations.
   • Use Azure Policy in both tenants to enforce compliance (e.g., ensuring all production resources are tagged correctly).
   • Clearly document the promotion process and access controls.
5. Manual Approval Gates: The Prod release pipeline should have mandatory manual approval checks before deploying to any production environment. This gives a human the final “go/no-go” decision.

This setup provides the strongest possible isolation while still enabling modern, automated application delivery. The initial configuration is complex but results in a very secure and auditable pipeline.

xxx

what are the best practice personas and user stories for a Azure Devops Secure Development Environment (SDE)_?

Of course. Designing a Secure Development Environment (SDE) in Azure DevOps requires a clear understanding of who interacts with the system and what their secure workflows should be. Using personas and user stories is an excellent way to formalize these requirements.

Here are the best practice personas and their corresponding user stories for an Azure DevOps SDE.

Core Personas for an Azure DevOps SDE

1. The Developer (Emma): The primary user who writes code and needs to do so quickly and securely.
2. The Security Engineer (Sam): Defines security policies, manages secrets, and ensures compliance.
3. The Release Manager/Ops Engineer (David): Manages the deployment process and production infrastructure.
4. The External Auditor (Priya): Reviews the environment for compliance with internal and external regulations.

Persona 1: The Developer (Emma)

Goal: To develop features quickly without introducing security vulnerabilities and without being blocked by cumbersome processes.

User Stories for Emma:

• As a Developer, I want to run pre-commit security scans on my code so that I can catch vulnerabilities before they are even pushed to the remote repository.
  • Feature: Pre-commit hooks integrated with tools like git-secrets or Talisman.
• As a Developer, I want my Pull Requests to automatically be scanned for secrets and vulnerabilities so that I get immediate feedback during code review.
  • Feature: Azure Pipelines with SAST (e.g., Snyk, SonarQube, Microsoft Security DevOps) or secret scanning (e.g., GitGuardian, TruffleHog) as a PR policy.
• As a Developer, I want to easily access non-production secrets for local debugging without ever seeing the actual values, so I can be productive without compromising security.
  • Feature: Integration with Azure Key Vault via azure/identity libraries or managed identity for local development.
• As a Developer, I want to be able to quickly spin up a isolated, ephemeral development environment that mirrors production, so I can test my changes accurately without complex setup.
  • Feature: Infrastructure as Code (IaC) pipelines using Bicep/Terraform to deploy dev environments on-demand.
• As a Developer, I want to understand the software bill of materials (SBOM) and license compliance of my dependencies, so I can avoid introducing legal or security risks.
  • Feature: Pipeline tasks that generate SPDX SBOMs and scan dependencies for vulnerabilities and licenses (e.g., Azure DevOps SBOM task, OWASP Dependency-Check).

Persona 2: The Security Engineer (Sam)

Goal: To enforce security policy as code, protect secrets, and ensure the entire development lifecycle is compliant without being a bottleneck.

User Stories for Sam:

• As a Security Engineer, I want to define security policies as code that are automatically enforced across all repositories, so that compliance is consistent and not manual.
  • Feature: Azure Policy for DevOps or custom pipeline templates that include mandatory security scanning steps.
• As a Security Engineer, I want to manage all secrets in a central, secure vault and grant pipelines temporary access to them, so that secrets are never stored in code or pipeline variables.
  • Feature: Azure Key Vault integrated with Azure Pipelines variable groups.
• As a Security Engineer, I want a dashboard view of all critical vulnerabilities across all projects and pipelines, so I can prioritize and track remediation efforts.
  • Feature: Azure Defender for DevOps (formerly Advanced Threat Protection) dashboard, Microsoft Defender for Cloud, or third-party security dashboard.
• As a Security Engineer, I want to ensure that only approved, secure base images are used for container development, so I can reduce the attack surface.
  • Feature: Pipeline task to scan container images for CVEs and enforce policies that reject builds using non-compliant base images (e.g., from Azure Container Registry).
• As a Security Engineer, I want to enforce branch protection policies like mandatory reviewers and successful builds before merging, so that broken or insecure code cannot enter the main branch.
  • Feature: Azure Repos Branch Policies.

Persona 3: The Release Manager/Ops Engineer (David)

Goal: To ensure deployments are reliable, repeatable, and secure, with clear audit trails and rollback capabilities.

User Stories for David:

• As a Release Manager, I want to deploy immutable artifacts that were security-scanned in a previous stage, so I know exactly what is being deployed and that it has been vetted.
  • Feature: Multi-stage YAML pipeline that promotes a single, versioned artifact from dev to prod.
• As a Release Manager, I want manual approval gates and checks before deploying to production, so a human can verify the change and business readiness.
  • Feature: Manual Approval checks in Azure Pipelines release stages.
• As a Release Manager, I want every deployment to production to be automatically recorded and auditable, including who approved it and what exact artifact was deployed.
  • Feature: Native Azure DevOps deployment history and audit logs streamed to a Log Analytics workspace.
• As a Release Manager, I want to define infrastructure changes as code and deploy them through the same pipeline as application code, so that environment drift is eliminated and all changes are reviewed.
  • Feature: IaC pipelines (Bicep, Terraform) with plan/apply stages and state management.
• As a Release Manager, I want to ensure production pipelines use identity-based connections that have least-privilege access, so there are no secrets to manage or rotate for deployments.
  • Feature: Service Connections using Managed Identity (e.g., Azure Resource Manager service connection with a system-assigned managed identity).

Persona 4: The External Auditor (Priya)

Goal: To independently verify that the development environment and processes comply with required standards (e.g., SOC2, ISO27001, GDPR).

User Stories for Priya:

• As an Auditor, I want to access a read-only, immutable log of all user and pipeline activities across Azure DevOps, so I can verify who did what and when.
  • Feature: Azure DevOps Audit Stream connected to Azure Monitor Logs or a SIEM.
• As an Auditor, I want to see evidence that security scanning is mandatory and that no vulnerabilities above a “low” severity threshold can be deployed to production.
  • Feature: Pipeline logs and reports showing security tasks passing/failing. Evidence of branch policies blocking PRs.
• As an Auditor, I want to review access control lists (ACLs) to confirm that the principle of least privilege is followed, and that no developers have access to production secrets or environments.
  • Feature: Azure DevOps Security management pages and Azure RBAC reports.
• As an Auditor, I want to verify that all infrastructure is defined as code and that no manual changes are made to production environments outside of the pipeline.
  • Feature: Azure Activity Logs showing that all changes were made by the pipeline’s service principal, not individual users.

Summary of Enabling Azure DevOps Features

By defining these personas and user stories, you can systematically build and validate an Azure DevOps SDE that is not only secure but also developer-friendly and compliant.

xxx

What are the best practice user stories and personas for Azure devops environment across a prod tenant with a prod Entra ID instance and a non-prod tenant with non-prod Entra ID instance

Of course. Designing user stories and personas for a multi-tenant, production-isolated Azure DevOps environment requires a sharp focus on security boundaries, controlled promotion, and auditable workflows.

Here are the best-practice personas and user stories for this specific, high-compliance scenario.

Core Guiding Principle

The overarching theme for all user stories is Unidirectional Flow and Strict Isolation. Code and artifacts flow from Non-Prod to Prod. Identities from the Prod tenant have highly restricted, read-only access to specific resources in Non-Prod. Identities from Non-Prod should have zero access to Prod.

Personas for a Multi-Tenant Azure DevOps Setup

1. The Developer (Emma): Resides in the Non-Prod tenant. Her world is development and testing.
2. The Security Engineer (Sam): Exists in both tenants but with different responsibilities. Defines the guardrails.
3. The Release Manager (David): Primarily operates in the Prod tenant. The gatekeeper of production.
4. The Auditor (Priya): Needs read-only access to both tenants to verify compliance and separation.

Persona 1: The Developer (Emma) – Non-Prod Tenant

Goal: To develop and test features quickly within the secure confines of the non-production environment.

User Stories for Emma:

• As a Developer in the Non-Prod tenant, I want to work in a fully isolated Azure DevOps project, so that my experimental work cannot accidentally impact production resources.
• As a Developer, I want my pull requests to be automatically validated by security scans and build pipelines that run in the Non-Prod tenant, so I get fast feedback without crossing tenant boundaries.
• As a Developer, I want to access test secrets and configuration from a Non-Prod Key Vault using my Non-Prod identity, so I can run and debug my code locally and in non-prod CI pipelines securely.
• As a Developer, I want to be able to trigger an automated deployment of my code to a non-production environment using IaC from my Non-Prod tenant, so I can validate my changes in a integrated environment.
• As a Developer, I want to see clear documentation on how to request promotion of a specific, validated artifact version to production, so I know the process is controlled and managed by the Release Manager.

Persona 2: The Security Engineer (Sam) – Cross-Tenant

Goal: To establish and maintain the secure “bridge” between tenants, enforce policies, and protect secrets.

User Stories for Sam:

• As a Security Engineer, I want to create a dedicated Service Principal (App Registration) in the Non-Prod tenant with read-only access to the artifact feed, so the Prod tenant pipeline can securely pull artifacts without using personal credentials.
• As a Security Engineer, I want to store the Client Secret for the bridge Service Principal in the Prod tenant’s Key Vault, so it is never exposed in the Non-Prod tenant and is managed by Prod’s stricter security controls.
• As a Security Engineer, I want to define Azure Policies that enforce mandatory security scanning steps in both the Non-Prod and Prod pipeline templates, so compliance is automated and consistent across both tenants.
• As a Security Engineer, I want to configure the audit log streams from both Azure DevOps organizations and both Azure AD tenants to feed into a central SIEM, so I can monitor all cross-tenant activity.
• As a Security Engineer, I want to ensure that no user from the Non-Prod tenant has any access or permissions within the Prod tenant, enforcing the principle of least privilege and tenant isolation.

Persona 3: The Release Manager (David) – Prod Tenant

Goal: To manage the controlled, auditable, and approved promotion of artifacts into the production environment.

User Stories for David:

• As a Release Manager in the Prod tenant, I want to manage a release pipeline that is isolated in the Prod Azure DevOps organization, so the production deployment process is protected from changes in the Non-Prod tenant.
• As a Release Manager, I want my Prod pipeline to authenticate to the Non-Prod artifact feed using the secure service connection (configured with the bridge SPN’s secret from Prod Key Vault), so I can download the specific artifact version approved for release.
• As a Release Manager, I want manual approval gates and checks within the Prod pipeline that require my explicit approval before deploying to any production stage, so I can validate change requests and ensure business readiness.
• As a Release Manager, I want to deploy infrastructure and applications using service connections that leverage a Prod-managed Managed Identity with least-privilege RBAC on the Prod subscription, so there are no secrets to manage for Azure deployments.
• As a Release Manager, I want a complete and immutable audit trail for every production deployment, showing who approved it, what artifact was deployed (with its hash), and the outcome, so I can prove compliance and facilitate troubleshooting.

Persona 4: The Auditor (Priya) – Cross-Tenant

Goal: To independently verify that the tenant isolation is effective and that all promotion processes are secure and compliant.

User Stories for Priya:

• As an Auditor, I want read-only access to the audit logs of both Entra ID tenants, so I can verify that there are no unauthorized cross-tenant logins or access attempts.
• As an Auditor, I want to review the permissions of the bridge Service Principal in the Non-Prod tenant to confirm it has only the read-only permissions required to access the artifact feed and nothing more.
• As an Auditor, I want to inspect the Prod release pipeline’s history to confirm that every deployment used an artifact sourced from the Non-Prod feed and not from any local or unauthorized source.
• As an Auditor, I want to see evidence that the Client Secret for the bridge Service Principal is stored and accessed only from the Prod tenant’s Key Vault, proving that the Prod tenant maintains control over the authentication secret.
• As an Auditor, I want to run a report on user membership in the Prod Entra ID to confirm that no developers from the Non-Prod tenant are present, validating the isolation of production access.

Summary Table: Personas and Their Tenants

By defining these personas and user stories, you move beyond technical configuration and build a governance model that ensures your multi-tenant Azure DevOps setup is not only functional but also secure, compliant, and auditable by design.

xx

Best practice to separate IaC DevOps from Applications development DevOps in a Azure environment with a prod tenant with a prod Entra ID and a non-prod tenant with non-prod Entra ID

Of course. Separating Infrastructure as Code (IaC) DevOps from Application DevOps in a multi-tenant environment is a critical best practice for security, compliance, and manageability. This setup enforces a clear separation of duties, where application developers cannot directly modify infrastructure, especially in production.

Here is a detailed breakdown of the best practice approach.

Core Principle: The “Paved Road” or “Platform” Model

The goal is to provide Application Teams with a self-service, automated, and secure way to request infrastructure defined as code, while the actual deployment and management of that infrastructure are controlled by a central Platform/IaC Team. The Application Team owns the specification of the infrastructure, while the Platform Team owns the implementation and security of the deployment process.

Recommended Multi-Tenant Structure

This structure builds upon your existing tenant isolation:

• Non-Prod Tenant (company-nonprod.com)
  • IaC DevOps Project: azdo-platform-nonprod (Hosts IaC modules, pipelines for deploying non-prod infrastructure)
  • App DevOps Project: azdo-app-teamA-nonprod (Hosts application code and app-specific IaC parameters)
• Production Tenant (company-prod.com)
  • IaC DevOps Project: azdo-platform-prod (Hosts pipelines for deploying production infrastructure only)
  • App DevOps Project: azdo-app-teamA-prod (Hosts production application code and parameters; highly restricted access)

Key Idea: The IaC pipelines in the Prod Tenant are the only entities with permission to deploy infrastructure to the Production subscription.

Best Practice Implementation Strategy

1. Code Organization and Repositories

• Platform Team (IaC) Repos:
  • Located in the azdo-platform-nonprod project.
  • Contain reusable, parameterized IaC Modules (e.g., Bicep modules, Terraform modules for AKS, SQL DB, Storage Accounts).
  • These modules enforce naming conventions, tags, security policies (e.g., NSG rules), and compliance settings by default.
• Application Team Repos:
  • Located in their respective azdo-app-teamX-nonprod project.
  • Contain:
    1. Application source code.
    2. Application Parameters: Environment-specific configuration files (e.g., dev.params.bicep, prd.main.tfvars) that call the approved IaC modules from the Platform team’s repos. They specify what infrastructure they need (e.g., appServicePlanSize: “P1v2”) but not how it’s built.

2. The Infrastructure Pipeline Design (The “How”)

This is the most critical pattern. The flow is unidirectional: Parameters from Non-Prod -> Validated in Non-Prod -> Promoted to Prod -> Deployed by Prod IaC Pipeline.

Step 1: Application Team Creates a PR

• A developer adds a new infrastructure requirement (e.g., a Redis cache) to their application’s parameter file (e.g., prd.main.tfvars) in their Non-Prod project.
• A PR triggers a validation pipeline in the Non-Prod project. This pipeline:
  • Lints the parameter file.
  • Plan/Preview the infrastructure change using the Platform Team’s modules (e.g., terraform plan).
  • Posts the plan result as a PR comment so the Platform Team and developers can review the proposed changes.

Step 2: Merge and Build Artifacts

• Once the PR is approved and merged, a pipeline in the App Team’s Non-Prod project runs.
• Its primary job is to package the parameter files (e.g., into a .zip or a versioned Universal Package) and publish them to an Artifact Feed in the Non-Prod tenant.

Step 3: The Controlled Promotion to Production

• The Release Manager or a member of the Platform Team triggers a pipeline in the azdo-platform-prod project in the Production Tenant.
• This pipeline:
  1. Uses a Secure Bridge to download the specific parameter artifact from the Non-Prod tenant’s feed (as described in previous answers).
  2. Uses its own, highly privileged Service Connection (with a Managed Identity from the Prod Tenant) to execute terraform apply or bicep deploy based on those parameters.
  3. This identity has RBAC roles (e.g., Contributor) on the Production subscription.

Diagram

Code

3. Access Control and Permissions

• Application Developers (Non-Prod Tenant):
  • Have Read access to the Platform Team’s IaC module repos.
  • Have Contributor access to their own application and parameter repos.
  • Have Zero Access to the Production Tenant or its Azure DevOps organization.
  • Have Zero Access to deploy directly to any Azure subscription.
• Platform Team (Both Tenants):
  • Have Contributor access to their IaC modules in the Non-Prod tenant.
  • Have Contributor access to the azdo-platform-prod project in the Prod tenant to manage the deployment pipelines.
  • Have User Access Administrator or similar rights on the Production subscription to grant necessary permissions to deployed resources (e.g., Managed Identities).
• Release Managers (Prod Tenant):
  • Have permissions to execute the pipeline in azdo-platform-prod but not necessarily to change its code.

4. Key Benefits of This Separation

1. Security & Compliance: The “blast radius” of a mistake is contained. An application developer cannot break the entire production network. All changes are peer-reviewed and applied through a controlled, audited process.
2. Separation of Duties: Developers define requirements; the Platform Team defines and controls the secure implementation. This is a fundamental security control.
3. Consistency and Governance: All infrastructure is built from approved, standardized modules that enforce organizational policies, reducing configuration drift and security gaps.
4. Auditability: Every production infrastructure change is traced back to a code change in an application repo, a PR review, and an execution by a specific pipeline identity in the Prod tenant.
5. Developer Empowerment: Developers still get the agility of self-service infrastructure without needing deep operational expertise or dangerous permissions.

This approach requires more upfront design but results in a vastly more secure, maintainable, and scalable cloud environment, perfectly suited for organizations with strict compliance requirements enforced by tenant isolation.

xx

can we have a disgram to show the above setup?

Of course. This diagram visualizes the end-to-end workflow for separating IaC and Application DevOps across isolated production and non-production tenants, as described in the previous answer.

Diagram

Code

How to Interpret the Diagram and Process Flow:

1. The Two Isolated Tents (Blue & Purple):
   • The blue area represents the Non-Production Tenant. This is where development and testing happen.
   • The purple area represents the Production Tenant. This is where controlled, audited deployments happen.
2. Step 1: Development & Pull Request (Non-Prod):
   • An Application Developer working in the App Team Project adds a new infrastructure requirement (e.g., a Redis cache) to their application’s parameter file in their Application Repo.
   • They create a Pull Request (PR). This triggers a validation pipeline (not shown for simplicity) that uses the modules from the Platform Team Project to run a terraform plan or bicep what-if and post the results for review.
3. Step 2: Build & Publish Artifact (Non-Prod):
   • Once the PR is approved and merged, the App CI Pipeline runs.
   • It builds the application code and, crucially, packages the infrastructure parameter files.
   • It publishes this package as a versioned artifact (e.g., v1.1.0) to the Artifact Feed.
4. Step 3: The Secure Bridge Between Tenants:
   • This is the critical link. A Service Principal (App Registration) exists in the Non-Prod Entra ID with read access to the Artifact Feed.
   • The Prod IaC CD Pipeline uses this SPN’s credentials (stored in the Prod tenant’s Key Vault) to authenticate and pull the specific artifact package from the Non-Prod feed. This is a secure, one-way transfer.
5. Step 4: Deployment to Production (Prod):
   • The Prod IaC CD Pipeline, which lives in the isolated production environment, is triggered. It contains the steps to deploy infrastructure.
   • It uses the downloaded parameters from the artifact.
   • It executes its deployment commands (terraform apply / bicep deploy) using a Managed Identity from the Prod Tenant. This identity has strict, least-privilege permissions (e.g., Contributor) on the Production Subscription.
   • This ensures the deployment is done with production-controlled identities and security.

Key Benefits Visualized:

• Separation of Duties: The App Developer requests infrastructure via code in their repo, but the Platform Team’s pipeline in the prod tenant is the only thing that can deploy it.
• Unidirectional Flow: Artifacts flow from Non-Prod to Prod. There is no access back from Prod into Non-Prod.
• Tenant Isolation: The Production Azure DevOps organization and Subscription are protected within the production security boundary. No developers have direct access.
• Auditability: Every production deployment is triggered by a specific, versioned artifact that was created from a merged code change in the non-prod tenant.