Download File

Solution Design

Contents

1       Executive Summary. 9

2       Introduction. 10

2.1         Scope. 10

2.2         Out of Scope. 10

2.3         Intended Audience. 11

3       Design Approach. 12

3.1         Requirements. 12

3.1.1          Compliance Requirements. 12

3.1.2          Functional Requirements. 12

3.1.3          Non-functional Requirements. 13

3.2         Architecture Principles. 14

3.3         Design Decisions. 15

4       Conceptual Solution Architecture. 20

4.1.1          Capability Model Reference Architecture. 22

4.1.2          Conceptual Application Architecture. 23

4.2         Target State – Solution Overview.. 24

4.2.1          Capability Model View.. 25

4.2.2          Standard Tooling Identification. 26

5       Azure Solution Design. 27

5.1         Billing Model 27

5.2         Tenant Hierarchy. 28

5.3         Azure Regions. 29

5.4         Platform Automation and DevOps. 30

5.4.1          Version Control 30

5.5         Identity and Access Management. 31

5.5.1          Authentication. 31

5.5.2          Multi-factor Authentication. 31

5.5.3          Privilege Management. 32

5.5.4          Directory Services. 32

5.5.5          Azure identity services tooling. 33

5.6         Public Key Infrastructure. 33

5.6.1          Internal Certification Authorities. 33

5.6.2          External Certification Authorities. 34

5.7         Networking. 34

5.7.1          Networking Principles. 35

5.7.2          Azure network service tooling. 35

5.8         Secrets Management. 38

5.8.1          Azure secrets management tooling. 39

5.8.2          Data Encryption Keys. 40

5.9         Storage. 40

5.9.1          Data-at-rest Encryption. 41

5.9.2          Data-in-transit Encryption. 41

5.9.3          Azure storage services tooling. 41

5.10      Compute. 42

5.10.1        Azure compute service tooling. 43

5.11      Containers and Container Orchestration. 44

5.11.1        Kubernetes. 45

5.11.2        Helm.. 46

5.11.3        Azure container tooling. 47

5.12      Remote Access. 48

5.12.1        Azure remote access tooling. 49

6       Resiliency. 50

6.1         Availability. 50

6.2         Backups. 50

6.3         Disaster Recovery. 51

6.4         Azure resiliency tooling. 51

6.4.1          Recovery services vault. 51

7       Security. 52

7.1         Compliance Framework Alignment. 52

7.1.1          Cloud security posture management. 52

7.2         Security Log Aggregation. 53

7.3         Security Event and Incident Management. 53

7.4         Web Application Firewall 54

7.5         Cloud Workload Protection. 55

7.6         Cloud-native Application Protection. 55

7.7         Virtual Machine Endpoint Protection. 56

7.8         Vulnerability Management. 56

8       Operations. 58

8.1         Logs and Metrics. 58

8.2         Alerting. 58

8.3         Update Management. 58

9       Monitoring Customer Experience. 60

Appendix A      References. 61

A.1        Design References. 61

Tables

Table 1 – Project Details. 2

Table 2 – RLX Contact Details. 2

Table 3 – RLX Version Control Details. 2

Table 4 – Compliance Requirements. 12

Table 5 – Functional Requirements. 13

Table 6 – Non-functional Requirements. 14

Table 7 – Architecture principles. 15

Table 8 – Solution Design Decisions. 19

Table 9 – Conceptual Design Principles. 21

Table 10 – Solution Standard Tooling. 26

Table 11 – Solution Mandated Tooling. 26

Table 12 – Networking Principles. 35

Table 13 – Azure Kubernetes Configuration Options. 45

Figures

Figure 1 – Azure Landing Zone conceptual diagram.. 20

Figure 2 – RLX Public Cloud Capability Model 22

Figure 3 – Application Conceptual Architecture. 23

Figure 4 – High-Level Architecture. 24

Figure 5 – Defined Capability Model Tool Mapping. 25

Figure 6 – Azure Hub-and-Spoke Topology. 34

Figure 7 – Key Vault capabilities. 39

Figure 8 – Contrast of Virtualisation Architectures Container Storage. 44

Design Decisions

Design Decision 1 – Subscription Sourcing Model 27

Design Decision 2 – Number of tenants in use. 28

Design Decision 3 – Structure of management groups. 28

Design Decision 4 – Use of subscriptions. 28

Design Decision 5 – Default language. 28

Design Decision 6 – Primary region for Identity. 29

Design Decision 7 – Primary Region for Production. 29

Design Decision 8 – Primary Region for Disaster Recovery. 29

Design Decision 9 – Primary Region for Nonproduction. 29

Design Decision 10 – Availability Zone Usage. 29

Design Decision 11 – Availability Zone Assignment. 29

Design Decision 12 – Infrastructure-as-Code. 30

Design Decision 13 – Platform Automation Tooling. 30

Design Decision 14 – CI/CD Agents. 30

Design Decision 15 – Version Identification. 30

Design Decision 16 – Identity Source. 31

Design Decision 17 – Authentication. 31

Design Decision 18 – Multi-factor Authentication. 32

Design Decision 19 – Authorisation. 32

Design Decision 20 – Separation of Duties. 32

Design Decision 21 – Directory Services Tooling. 33

Design Decision 22 – Certification Authorities (Internal). 34

Design Decision 23 – Certification Authorities (External). 34

Design Decision 24 – Network Topology. 37

Design Decision 25 – Route Management. 38

Design Decision 26 – Firewall Tooling. 38

Design Decision 27 – Firewall SKU.. 38

Design Decision 28 – HTTPS Inspection. 38

Design Decision 29 – Secrets Tooling. 40

Design Decision 30 – Volume Encryption Keys. 40

Design Decision 31 – Default Storage Replication. 40

Design Decision 32 – Default Disk Performance Tier. 40

Design Decision 33 – Storage Encryption Keys. 41

Design Decision 34 – Linux (RPM) Server Operating System.. 42

Design Decision 35 – Linux (Debian) Server Operating System.. 42

Design Decision 36 – Windows Server Operating System.. 43

Design Decision 37 – Windows Client Operating System.. 43

Design Decision 38 – Virtual Machine Image Management. 43

Design Decision 39 – Container Registries. 45

Design Decision 40 – Kubernetes Kernel Version. 45

Design Decision 41 – Kubernetes Network Stack. 45

Design Decision 42 – Kubernetes Network Controls. 46

Design Decision 43 – Kubernetes Deployment Architecture. 46

Design Decision 44 – Kubernetes Access. 46

Design Decision 45 – Container Deployment Configuration Management. 47

Design Decision 46 – Remote Access Tooling. 48

Design Decision 47 – Restricted Environment (CDE) Access. 48

Design Decision 48 – Privileged Access Environment. 48

Design Decision 49 – Availability Targets. 50

Design Decision 50 – Availability Architecture. 50

Design Decision 51 – Backup Tooling. 51

Design Decision 52 – Compliance Framework(s) Alignment. 52

Design Decision 53 – Cloud Security Posture Management Tooling. 53

Design Decision 54 – Security Log Aggregation. 53

Design Decision 55 – Security Incident and Event Management Tooling. 54

Design Decision 56 – Web Application Firewall (Internet) Tooling. 54

Design Decision 57 – Web Application Firewall (Internal) Tooling. 55

Design Decision 58 – Cloud Workload Protection Tooling. 55

Design Decision 59 – Cloud-native Application Protection Tooling. 55

Design Decision 60 – Virtual Machine Endpoint Protection Tooling. 56

Design Decision 61 – Monitoring Tooling. 58

Design Decision 62 – Operational Log Aggregation. 58

Design Decision 63 – Update management for Azure Virtual Desktop Session Hosts. 59

Design Decision 64 – Update management for Virtual Machines. 59

Design Decision 65 – Update management for Container images. 59

Design Decision 66 – Update management for Azure Kubernetes Services. 59

1                       Executive Summary

This document details the solution approach to building, deploying, and operating a modern, secure Microsoft Azure platform for applicable business services, including internal and external facing functions and services.

Leveraging RLX’s experience across multiple public cloud platforms, Microsoft’s Cloud Adoption and Well-Architected Frameworks, national and regional governments, and secure-by-design core values, this document details the architecture framework and design requirements for this Azure tenant that will be used to:

1. Enable highly scalable cloud-native workloads to be deployed and managed.
2. Provide a secure platform for a mission-critical public service application.
3. Provide assurance on intent and direction for cloud-native architecture.

There are several key outcomes of this and related documents, namely:

• Establishment of the Azure hierarchy of management groups, subscriptions, and resource groups.
• A highly available, scalable network to serve current and future demand.
• An integrated identity and access management solution, tied to a ‘least-privilege’ approach and supporting both first-party and third-party platform consumers.
• Clearly defined security and operational guardrails on the use of the platform.
• Adherence, visibility, and governance of applicable regulatory and compliance-aligned frameworks.
• Holistic reporting and monitoring across the platform.
• DevSecOps and platform automation at an enterprise level.
• Definition of capabilities to support disaster recovery and high availability requirements.

Business benefits delivered through the adoption of this architecture and related initiatives are:

1. Well-known, near-real-time security posture from initiation through to operations.
2. Defined performance and availability benchmarks.
3. Rationalisation and/or lower complexity in
   1. Reduction in services and/or functions
   2. Innovation or agility in technology adoption and consumption
   3. Scalability, responsiveness, and flexibility

2                       Introduction

2.1               Purpose

The purpose of this High-Level Design (HLD) is the creation of a Microsoft Azure Enterprise-Scale Landing Zone (ESLZ). The HLD describes the required Azure components and related services to operate an Azure ESLZ, as well as providing an overview of the specific tailoring for CUST’s business, functional, and non-functional technical requirements.

The overall architecture for the Azure environment, including this Landing Zone, is intended to follow a pattern-based approach that looks to apply enterprise-wide principles to Azure platform design and implementation, together with best practices from the Azure Cloud Adoption and Well-Architected Frameworks.

The architecture patterns applied to this Landing Zone have been selected with the following outcomes:

• A secure, sustainable digital infrastructure using Infrastructure-as-Code (IaC), immutable infrastructure, and separation of duties through DevSecOps.
• A modern, future-forward seamless digital platform that facilities innovation using a “SaaS over PaaS over IaaS” philosophy for workloads and elasticity.
• Providing a solid technology-based foundational platform for future growth into Azure for CUST ’s internally and externally facing business applications.
• Efficient, cost-effective, and scalable tooling, leveraging secure-by-design principles and cloud-native technologies.
• Ensuring continuity of service for all platform consumers by embedding the availability and resilience of the Azure platform.
• Ensuring continuity of projects and development, using highly available Software- and Platform-as-a-Service offerings, that are consumption-based and benefit from out-of-the-box security and resiliency.

The HLD also provides a common understanding of the decisions, their rationale, and tooling used to operate the ESLZ and related services to all stakeholders.

2.2               Scope

The components listed below are in scope of this HLD for the Azure ESLZ:

• Enterprise Agreement enrolment and Entra ID tenant(s)
• Entra ID Directory-based identity and access management
• Azure management groups and subscription hierarchies
• Network topology, integration, and delegation
• Reporting, monitoring, and operational capabilities
• Platform-wide security configuration, governance, and compliance
• Platform-wide automation and DevSecOps capabilities

2.3               Out of Scope

Any item not identified in Section 2.2 will be considered out of scope for the purpose of this document.

3                       Design Approach

3.1               Critical drivers

3.2               Solution principles

3.3               Requirements

3.3.1          Compliance Requirements

The data and workloads hosted through the platform are required to comply with several security and regulatory frameworks. These frameworks, and high-level alignment requirements are listed below.

Table 4 – Compliance Requirements

3.3.2          Functional Requirements

This section outlines the functional requirements identified for the Azure platform. Functional requirements are the capabilities related to the performance or operation required of a given system, solution, or capability.

Table 5 – Functional Requirements’

3.3.3          Non-functional Requirements

This section outlines the non-functional requirements identified for the Azure platform. Non-functional requirements are the inherent characteristics unrelated to perform of a given system, solution, or capability.

Table 6 – Non-functional Requirements’

3.4               Architectural approach

3.4.1          Microsoft frameworks

Microsoft offers several frameworks to support build and operations of cloud environments. These are:

• Cloud Adoption Framework (CAF)
• Well-Architected Framework (WAF)

The CAF is specifically designed to support new organisations and enterprises prepare for cloud-based workloads and ways of working. The WAF is designed to support the refactoring or build of application on cloud, to best maximise the value of cloud. Both of these frameworks are leveraged in the build of the Enterprise-Scale Landing Zone.

The primary purpose of the Azure Enterprise-Scale Landing Zone (ESLZ) is to ensure that when a new workload is deployed to Azure, such as a new secure research project, the supporting foundation is already in place which reduces onboarding time and provides compliance in line with organisational security and governance requirements.

RLX will create a new Azure ESLZ in an existing Microsoft Entra ID tenancy. This involves prescriptive guidance coupled with best practice for the Azure control plane. Azure Landing Zones are predefined and preconfigured environments which provide baselines for scalability, security, governance, networking, and identity. Azure Landing Zones will be built to align with CUST’s operational requirements.

The Azure ESLZs to be provisioned contains the following architectural components:

• Scalable and cost-optimised hub-and-spoke network topology using logically segmented subnets, on-premises connectivity and controlled routing behaviours
• Pre-defined governance, monitoring, compliance, and security controls
• DevOps platform to support automated infrastructure deployments and maintenance updates
• Integration with enterprise identity platform to control environment access and manage user lifecycle
• Centralised billing with the ability to support chargeback scenarios

3.4.2          Architecture principles

The following principles have been applied to this Design and related Designs:

Table 7 – Architecture principles

3.5               Assumptions

3.6               Design Decisions

The below table captures the results of each Design Decision. For the detail in each Decision, refer to the specific Decision in the design.

Table 8 – Solution Design Decisions

4                       Solution Overview

The Azure Platform is the foundational infrastructure capability provider, from which one can design, build, deploy, and manage the full spectrum of applications required to support all business technology initiatives.

The below image shows the proposed concepts being the solution. Each subscription is focused on a specific capability; shared – such as identity – or dedicated. This meets the Enterprise-Scale Landing Zone (ESLZ) recommendation.

The Enterprise-Scale architecture provides prescriptive architecture guidance coupled with Azure best practices and follows design principles across the critical design areas for an organisation’s Azure environment and landing zones. It is an architecture approach and reference implementation that enables an effective adoption and implementation of landing zones on Azure that are easily built and scalable.

Figure 1 – Azure Landing Zone conceptual diagram

The Conceptual Architecture is closely aligned with the Microsoft Cloud Adoption Framework’s approach to an Enterprise-Scale Landing Zone and focuses on the following five design principals.

Table 9 – Conceptual Design Principles

4.1               Capability Model Reference Architecture

The below diagram demonstrates RLX’s reference architecture and capability model approach for cloud architecture. As part of the design process, each desired capability is mapped to one or more technologies. Existing technologies which are leveraged for an environment are captured, reviewed, and determined if appropriate for use or if needing to be replaced with a tool which meets functional and non-functional requirements.

Figure 2 – RLX Public Cloud Capability Model

4.2               Conceptual Application Architecture

The following diagram illustrates how the Conceptual Architecture can be implemented to support applications at scale. Where the Identity, Connectivity, and Management components are “shared” across one or more workloads.

This enables each individual components can scale individually, based on pre-determined or dynamic conditions and with different mechanisms. In this way, cost/value benefit can be obtained as some shared services have a baseline capacity supporting a given number of services at any one time.

Figure 3 – Application Conceptual Architecture

4.3               Target State – Solution Overview

The target state hosting platform is the Azure Cloud. Azure IaaS and PaaS services will be used to provide the platform level support services. The Azure platform will be built from several key technologies and services, as described further within this Solution Design.

Figure 4 – High-Level Architecture

4.3.1          Capability Model View

The below Reference Architecture Capability Model identifies each capability expected to be consumed within the Azure platform.

Figure 5 – Defined Capability Model Tool Mapping

4.3.2          Standard Tooling Identification

The below table identifies the standard tooling set applied within the Azure platform, enabling key ESLZ capabilities.

Table 10 – Solution Standard Tooling

4.3.2.1        Mandated Tooling

The following table describes the tooling which is required to support one or more application stacks within the environment.

Table 11 – Solution Mandated Tooling

5                       Solution Overview

The Microsoft Cloud Adoption Framework (CAF) for Azure is a series of guidance articles, patterns, and design approaches to help organisations to create and implement business and technology strategies for the cloud. It provides best practices, documentation, and tools. Cloud architects, IT professionals, and business decision makers use this information to achieve their cloud adoption goals.

The CAF’s Enterprise-Scale Landing Zone (ESLZ) architecture represents the strategic design path and target technical state for an organization’s Azure environment. It continues to evolve alongside the Azure platform. Azure landing zones are the output of a multi-subscription Azure environment that accounts for scale, security, governance, networking, and identity.

This Solution Design leverages the standard Microsoft CAF ESLZ architecture by extending the pattern, based on RLX’s experience and prioritisation of secure-by-design principles.

5.1               Billing Model

Microsoft consumption elements, namely Microsoft 365 licensing and Azure subscriptions, can be consumed through two (2) approaches:

• An Enterprise Agreement with Microsoft, or
• A “Cloud Services Provider” (CSP) engagement with a third-party.

An Enterprise Agreement is a direct negotiation engagement with Microsoft, where funds or a budget are allocated to commercial agreement which is drawn down or consumed over time. This does have a minimum time commitment of three (3) years but can consolidate costs for all Microsoft and Azure services.

A CSP is a third-party to both Microsoft and the end-consumer, that can provision services to the end-consumer without a formal budget allocation against an Enterprise Agreement. CSPs allow for end-consumers to manage cost truly flexibly, rather than be locked into a commercial agreement with Microsoft that may require change over time.

Design Decision 1 – Subscription Sourcing Model

5.2               Tenant Hierarchy

A ‘tenant’ in terms of Microsoft is a directory for identities. Microsoft licenses and Azure subscriptions are assigned to a specific tenant. This serves as the central point for all billing elements specific to that tenant.

An organisation may operate one (1) or more tenants, depending on the applicable use case, such as an enterprise with multiple business names to isolate identities and costs for those businesses.

Design Decision 2 – Number of tenants in use

Design Decision 3 – Structure of management groups

Design Decision 4 – Use of subscriptions

Design Decision 5 – Default language

5.3               Azure Regions

The Azure Platform physical architecture will leverage Azure datacentres which are in the Australia East and Australia Southeast regions. Australia Southeast is located within the metropolitan Melbourne area, and Australia East is located within the Sydney metropolitan area.

Design Decision 6 – Primary region for Identity

Design Decision 7 – Primary Region for Production

Design Decision 8 – Primary Region for Disaster Recovery

Design Decision 9 – Primary Region for Nonproduction

Design Decision 10 – Availability Zone Usage

Design Decision 11 – Availability Zone Assignment

5.4               Platform Automation and DevOps

Azure DevOps is currently recommended by Microsoft as one of the preferred Version Control Systems. Azure DevOps is a fully managed SaaS platform that orchestrates pipeline and agents.

GitHub Enterprise is an alternative solution as GitHub Enterprise is equivalent to GitHub’s public service but is designed for private use by large-scale enterprise and small-to-medium software development teams where they wish to host their repositories behind a corporate firewall.

Design Decision 12 – Infrastructure-as-Code

Design Decision 13 – Platform Automation Tooling

Design Decision 14 – CI/CD Agents

5.4.1          Version Control

Use of version control is valuable in identifying configuration and change in configuration over time. There are several modern methods for tracking this configuration., such as Semantic Versioning or Date-based Versioning.

Design Decision 15 – Version Identification

5.5               Identity and Access Management

5.5.1          Authentication

Authentication refers to the process of verifying the identity of a user or a system. The Azure platform solution will validate unique identifiers for users and systems through the Entra ID identity system. Entra ID will enable the use of a common set of policies, practices, and protocols to manage the identities and trust of users and systems.

Additional authentication controls include:

• Strict password policies – e.g., strength and prevention of re-use – will apply to Entra ID accounts.
• Requests for new accounts to follow a request-based process for auditability. Entra ID roles for account creation will be limited to an appropriate Entra ID group to prevent unauthorized creation of accounts.
• As a preventative measure against catastrophic administrator lockout from the Entra ID, emergency access accounts will be provisioned.

Design Decision 16 – Identity Source

Design Decision 17 – Authentication

5.5.2          Multi-factor Authentication

Entra ID will be configured to require at least two verification methods to confirm the identity of users. Policies will be in place to ensure that Entra ID based multi-factor authentication is enabled and enforced by default for all logins, including the Emergency Access Accounts (which should be stored in a physical secure location physically remote from the office location) and System Identities (used for system authentication).

Design Decision 18 – Multi-factor Authentication

5.5.3          Privilege Management

Entra ID provides identity governance capabilities through Entra ID Entitlement Management. Longitudinal access management is implemented via well-defined access packages and extending this pattern to delegate time-bound privileged access to resources, approvals, and access reviews, as well as facilitating access to external users where required.

Privileged Access Management (PAM) is required to control access to highly privileged credentials and prevent unauthorised access to the Azure platform. Entra ID Privileged Identity Management (PIM) will also be used to restrict access to privileged Azure resources. PIM provides users just-in-time privileged access to Azure and Entra ID resources, and oversight of what users are performing with their privileged access.

Design Decision 19 – Authorisation

Design Decision 20 – Separation of Duties

5.5.4          Directory Services

Whilst Entra ID is a cloud-native solution, compatible with modern applications, there are several areas which Entra ID is not optimal as a solution. Entra ID Domain Services is an Azure service capable of extending Entra ID to fit those use cases, such as Kerberos authentication.

Design Decision 21 – Directory Services Tooling

5.5.5          Azure identity services tooling

5.5.5.1        Entra ID

Entra ID is a cloud-first, cloud-native end-to-end identity and access management solution. It supports adoption and use of identities, conditional access policies governing how identities can be consumed, privilege management for controlled roles and/or group memberships. It also supports integration of applications that support Single-Sign On protocols, such as SAML v2, OpenID Connect, or OAuth 2.0.

5.5.5.2        Entra ID Domain Services

Entra ID Domain Services (AADDS) is an extension of Entra ID, to support traditional directory services for virtual machines and applications. An instance of AADDS provides the logical AD endpoints for connectivity, where the underlying infrastructure is entirely managed by Microsoft Azure.

5.6               Public Key Infrastructure

Public Key Infrastructure (PKI) is a term covering certificates, such as those used to enable HTTPS on websites, and the services that issue or sign certificates known as Certification Authorities (CAs). A known, defined PKI environment for internal and external certificate issuance is a key part of establishing trust on secured connections for data-in-transit security. Data-in-transit security is mostly performed through Transport Layer Security (TLS), which necessitates the adoption and implementation of trusted certificates.

5.6.1          Internal Certification Authorities

Internal CAs are required to be able to establish trust on an internal network, for services which require to expose or consume Transport Layer Security (TLS) protocols. This is an essential component for TLS inspection for firewall traffic.

Design Decision 22 – Certification Authorities (Internal)

5.6.2          External Certification Authorities

External CAs are required to be used to allow Internet-facing services to be trusted by end-users. The lifecycle management of issued certificates will be managed by a platform-hosted solution and integrated with a CA provider for zero- or low-cost certificates.

Design Decision 23 – Certification Authorities (External)

5.7               Networking

Virtual Network(s) provide a key building block for establishing virtual private networks that isolate network communication within the Azure environment. By default, when an Azure resource is created and connected to Azure Virtual Network, they are allowed to route to any subnet within the virtual network, and outbound access to the Internet is provided by Azure’s Internet connection. This default configuration will be altered using User Defined Routes (UDRs) thus forcing Azure resource to route through virtual firewalls deployed in a central virtual network.

Figure 6 – Azure Hub-and-Spoke Topology

The network is designed with regional hubs and spoke based configuration to provide centralised management of core network concerns such as routing, firewalls, and DNS within each region.

Key design points include:

• Deployment across Australia East / Southeast paired regions for HA/DR.
• ExpressRoute private WAN connections to remote networks for performance and network security.
• Private network endpoints for Azure native services, to avoid sending sensitive internal traffic via the Internet.

The network aligns to a standard hub-and-spoke based architecture to enable centralised traffic management through the hub networks, and isolation of applications / workloads within the spoke networks. The Hub networks form the major connectivity transit zone for traffic within an environment. The current network architecture includes two hubs, one for production-based traffic and workloads as well as non-production network to enable less strict change control for development and testing purposes without impacting the production infrastructure.

5.7.1          Networking Principles

The following principles will be used to drive the selection of IP ranges and subnets for use within the environment.

Table 12 – Networking Principles

5.7.2          Azure network service tooling

5.7.2.1        Azure virtual networks

Virtual Networks provide the Layer 3 capability to issue IPv4 and/or IPv6 addresses to Infrastructure-as-a-Service resources, such as Virtual Machines. Virtual Networks can host one or more IP CIDR ranges, allowing a significant number of IP resources to be consumed.

Subnets are the capability within virtual networks sub-divide and micro-segment networks, to ensure that only relevant services are hosted in the same subnet.

5.7.2.2        Azure firewall

Azure Firewall is a PaaS solution provided by Microsoft which performs Layer 4 and Layer 7 security policy management for network traffic, akin to traditional or Next-Generation Firewalls. Azure Firewall does require the use of Public IPs for management, to communicate with the Azure management API; these will be procured through Azure via an IP prefix assignment.

5.7.2.3        Public IP prefixes and addresses

Public IP addresses are unique addresses assigned to devices and resources connected to the internet. A public IP prefix is a range of consecutive public IP addresses allocated to a tenant. Instead of assigning individual IP addresses, a prefix is used to issue IP addresses due to the many advantages.

Using a prefix allows scalability and efficient allocation of IP addresses, a range can be assigned to a single block, reducing fragmentation, and optimizing address utilisation.  Streamlined networking is also facilitated with the use of prefixes, the prefix can be referenced instead of individual IP addresses separately for greater network control and reduced change-over-time in configuration.

5.7.2.4        Route tables

By default, network traffic in Azure Virtual Networks is routed directly between subnets and IP addresses. To support Zero Trust principles and maintain security, custom routing logic is required to be introduced into each virtual network’s subnets.

Route tables will be used to statically set traffic paths as dynamic cross-region route management is not required.

5.7.2.5        DNS zones

Azure DNS zones act as DNS zone files, providing zone hosting with a wide range of DNS record capabilities and integration. Private DNS Zones are intended for use within a private network, not a public or Internet-facing network. Azure provides the capability to integrate with virtual networks, allowing for both resolution of DNS records as well as automatically registering resources within the virtual network to the DNS zone.

5.7.2.6        Network security groups

Network Security Groups (NSGs) act as layer 4 network access control lists, but not as powerful as a next-generation firewall inspecting layer 7 traffic. NSGs are the last line of network defence in Azure Virtual Networks, able to be applied to both subnets and network interfaces.

5.7.2.7        Application security groups

Application security groups (ASGs) are used in conjunction with network interfaces and NSGs. ASGs allow for groups of resources to be created, so that NSG rules only need to identify the ASG as the source or destination, removing the need for IP addresses to be entered. Network Interfaces can be attached to one or more ASGs, meaning an interface could be subject to multiple rules.

5.7.2.8        Load balancers

Load Balancers (LBs) in Azure come in several types. Layer 4 LBs for handling TCP and UDP and either as public or private; or, the Application Gateway for Layer 7 routing of HTTP traffic.

5.7.2.9        ExpressRoute circuits

ExpressRoute circuits are a highly available, low latency, high bandwidth wide-area network (WAN) connection option to be able to integrate Azure networks into an existing or private network. Consuming an Azure Virtual Network Gateway, along with a connection through a subset of network providers, ExpressRoute is a viable alternative to VPN tunnels or mesh networks where latency, privacy, and security are concerns.

5.7.2.10     Virtual and local network gateways

Virtual Network Gateways (VNGs) are a network routing appliance, able to operate either as a VPN server – terminating IPSec and SSL-VPN tunnels – or as an ExpressRoute circuit peer. VNGs use the BGP network protocol to be able to manage complexity and routing at scale across tunnels and circuits.

Local Network Gateways (LNGs) are used only in conjunction with VNGs when deploying IPSec tunnels, for static routing. If an environment at the other end of an IPSec tunnel can’t handle BGP for dynamic routing, an LNG needs to be deployed to instruct the VNG what networks are available on the other end of the tunnel and via which destination address.

Design Decision 24 – Network Topology

Design Decision 25 – Route Management

Design Decision 26 – Firewall Tooling (East/West)

Design Decision 27 – Firewall Tooling (North/South)

Design Decision 28 – Firewall SKU

Design Decision 29 – HTTPS Inspection

5.8               Secrets Management

Azure Key Vault will be used to store encryption keys, secrets such as passphrases, and certificates used for HTTPS, as well as managing their lifecycle. Key Vault then is the source of truth for all certificates, secrets, and keys throughout the platform. Applications may deploy one or more Key Vaults as needed to secure keys and secrets.

Figure 7 – Key Vault capabilities

In above diagram, a VM is leveraging a certificate stored in Key Vault for providing HTTPS on a website. This allows for the certificate to be managed and updated without needing administrative access to the underlying VM. In addition, the web and VM logs are being pushed to a storage account file share, which is encrypted at the Azure management layer using a key stored in the same Key Vault.

This is an example of the capabilities that Key Vault can bring to help separate the elements which improve security within a given Azure cloud environment.

5.8.1          Azure secrets management tooling

Azure Key Vault supports keys, secrets, and certificates. Keys meaning RSA or EC keypairs; secrets meaning any value such as an API token; and certificates meaning PKCS12-formatted keypair and signed certificate bundles. Other services, such as Azure Storage, depend on Azure Key Vault for use in managed encryption, opposed to Microsoft-managed encryption.

An advantage of leveraging Key Vault is the ability to not just store secret values, such as keys, but also generate keys as well. With this, keys generated and stored in Key Vault cannot be exported in clear-text, reducing the ability of administrators to store highly sensitive credentials outside of controlled areas. This is as-designed for Key Vault to prevent key exposure. Instead, administrators may request a “key signing” operation instead, where it is required. For private keys that are required to be imported into Key Vault, this should instead be governed by an appropriate process.

Where possible, Key Vaults will be deployed with all defined access within the IaC definition; however, in the event of issues or troubleshooting, some access policies may need to be manually updated. These can then be ‘reset’ by re-running the deployment pipeline.

Design Decision 30 – Secrets Tooling

5.8.2          Data Encryption Keys

Design Decision 31 – Volume Encryption Keys

5.9               Storage

The Azure platform will require the use of storage for storing and retaining data which supports the platform functionality, including virtual machine backups and logging. Specifically, VM disks, storage accounts and Log Analytics workspaces. To enable clear isolation and ownership boundaries, services will be provisioned and logically allocated to a specific application or service requirement. This approach also ensures scalability, availability and security can be configured based on the targeted needs of each service.

Design Decision 32 – Default Storage Replication

Design Decision 33 – Default Disk Performance Tier

5.9.1          Data-at-rest Encryption

Data at rest (DaR) encryption refers to the encryption of data as it persists in storage. DaR encryption is a critical security control as it addresses security risks related to direct physical access to storage media, as the underlying data is not recoverable and cannot be changed without the configured encryption key. This makes it an important layer in the defence in depth strategy of Microsoft data centres. In addition, there are often compliance and governance reasons to deploy DaR encryption.

There are two layers of data at rest security:

• Encryption managed by the Azure platform provider (Microsoft), and,
• Encryption managed by the Customer.

Out-of-the-box encryption as part of the Azure platform is managed by Microsoft. The storage account service encrypts data at-rest with the AES-256 encryption cipher.

All forms of storage will be encrypted at rest as well as in transit. The specific encryption protocols and resource configuration will be defined in the Detailed Design and enforced via Cloud Security Posture Management (CSPM) tooling.

Design Decision 34 – Storage Encryption Keys

5.9.2          Data-in-transit Encryption

Data-in-transit (DiT) encryption refers to the encryption of data as it transits from one endpoint to another, usually across a network bearer. Broadly speaking DiT encryption is handled transparently by the Azure platform using TLS 1.2 or later both on external and internal Azure interfaces.

All the cloud services and protocols used for encryption are implemented should be using Australian Signals Directorate approved cryptographic algorithms. All PaaS services encrypt data in-transit with the most secure, supported Transport Layer Security protocol, either version 1.2 or 1.3.

5.9.3          Azure storage services tooling

5.9.3.1        Storage accounts

Storage accounts are a PaaS service offering, providing blob (block-level), file (SMB), table, and queue storage capabilities. Storage accounts can also be configured for replication tiers, with single region, multi-region, and/or multi-zonal support options.

5.9.3.2        Disks

Disks are an IaaS capability for mapping persistent storage to virtual machines to be treated as attached storage and is available in several tiers depending on performance requirements.

5.10          Compute

The compute aspects of Azure platform are based on two primary compute types, PaaS and Virtual Machines. The choice of technology is based on the availability of a suitable native technology, with a preference for PaaS over IaaS where possible.

Where possible, the preference is to utilise scale sets over specific Virtual Machine creation and management to enable additional scaling opportunities as well as to reduce the operation management and complexity of managing specific instances. To ensure consistency within Virtual Machine-based compute, Virtual Machines images will be built via automation tools, such as Hashicorp Packer.

The design decisions that follow provide a recommended approach on known, good, trusted Virtual Machine operating systems.

Design Decision 35 – Linux (RPM) Server Operating System

Design Decision 36 – Linux (Debian) Server Operating System

Design Decision 37 – Windows Server Operating System

Design Decision 38 – Windows Client Operating System

Design Decision 39 – Virtual Machine Image Management

5.10.1     Azure compute service tooling

5.10.1.1     Azure compute gallery

Azure Compute Galleries are the service which manages Standard Operating Environment images for virtual machines. Consumers create VMs using specific named images and versions. Image publishers create VMs, perform configuration, then publish the image to the gallery for re-use.

5.10.1.2     Azure virtual machines

The traditional IaaS capability to run a virtual machine, Azure supports both Windows and Linux VMs. There is an active marketplace of VMs, supporting vendor-provided images such as next-generation firewalls, or “complete” images such as a combined web/app/database all-in-one server.

5.10.1.3     Azure virtual machine scale-sets

VM Scale Sets are image-based VMs which scale-out and scale-in dynamically based on pre-defined conditions. Scale-sets can only run from an image and are not traditionally accessible like standard virtual machines. Scale-sets are best used for highly parallel workloads, such as data processing or web servers under heavy load.

5.10.1.4     Azure availability sets

Availability sets are a mechanism to separate virtual machines within Azure’s underlying physical infrastructure and management activities. When using more than 1 virtual machine, an availability set is used to identify fault domains, which VMs are then allocated into. In this way, application availability can be achieved, even when there is an issue with the underlying Azure infrastructure.

5.11          Containers and Container Orchestration

Containers are a modern standard for virtualisation of application hosting, which represents a second-generation approach to virtualisation. Under a well-known approach with Virtual Machines, applications were deployed to comparatively large instances, and exposed a “full” operating system for support. This allowed for a high degree of customisation and variability with identical applications, resulting in a varied support and performance baseline.

Contrasted to container-based applications, where an application is preconfigured with the same specification and performance benchmark. Rather than exposing a “full” operating system, a container ‘contains’ only the minimal binaries required to operate, enabling for a much greater volume of containers to be deployed.

Containerisation allows for the same application to be run in multiple different environments, agnostic to the underlying operating system of the host.

Figure 8 – Contrast of Virtualisation Architectures Container Storage[1]

Containers are like Virtual Machine images. They are compiled or built and need to be stored in preparation for deployment. Azure offers a native service, which supports containers.

Design Decision 40 – Container Registries

5.11.1     Kubernetes

Kubernetes is an extension of the container model, managing the underlying hosts for containers at scale. Leveraging a cluster approach, Kubernetes offers capabilities which are essential for managing containers and applications at scale. At a high level, these capabilities include:

• Self-healing: restarting, replacing, and lifecycling containers which do not meet conditions satisfactory for the application to be “available”.
• Load-balancing: native inherent capabilities to deploy multiple instances and have them operate in parallel or sequentially.

Microsoft Azure’s iteration of Kubernetes is Azure Kubernetes Service (AKS). Microsoft offers several options for operating AKS within environments. These are described in the table below.

Table 13 – Azure Kubernetes Configuration Options

Microsoft updates the available AKS version frequently, in line with the overall Kubernetes kernel version release cycle.

Design Decision 41 – Kubernetes Kernel Version

5.11.1.1     Network integration paradigm

Kubenet is a model closer to true PaaS, where Microsoft manages the underlying networking of the cluster and components. Azure CNI is a model where networking is drawn from the deployed Virtual Network.

Design Decision 42 – Kubernetes Network Stack

5.11.1.2     Network controls tooling

Kubernetes pods, the instance of a container that runs within a deployment, have open network access by default. Azure provides two (2) methods for implementing network controls on pods: Azure Network Policy Manager or Calico.

Azure NPM is a relatively new capability which is specific to Azure’s deployment but is not yet generally available on all cluster configurations. Calico is an open-source capability which can be applied to Kubernetes agnostic to hosting environment.

Design Decision 43 – Kubernetes Network Controls

5.11.1.3     Management API visibility

Azure supports two (2) visibility architectures for AKS clusters: public and private. Public clusters are clusters which have been deployed so that the management API interface may be reachable over the direct Internet. Private clusters are deployed such that the management API interface can only be reached through the private network space where the cluster resides.

Design Decision 44 – Kubernetes Deployment Architecture

5.11.1.4     Access Management

Azure supports two (2) methods for governing access into AKS environments: native and Entra ID. Native access management means managing credentials within the AKS cluster itself for all administrative functions. Entra ID-based access management allows for the use of Entra ID identities and delegated access and privilege management with Entra ID functions.

Design Decision 45 – Kubernetes Access

5.11.2     Helm

When deploying containers to Kubernetes clusters, there are a wide range of tools supporting deployments. The native Kubernetes command-line interface (CLI) tool, kubectl, can be used to apply configuration settings individually or using a configuration file.

As scale, this is not ideal, as multiple containers might be deployed with similar settings which results in repeated configuration creation and managing conflicts. This is where Helm comes into play. Helm is a packaging tool specifically for containers on Kubernetes. Alternatives to Helm include using Kubernetes manifest files (that is, not creating a “bundled” package definition) or another IaC tool such as Terraform.

Individual Kubernetes manifest files are individually created YAML files, where each specific resource type is defined in isolation from each other, even if used for the same deployment. For example, a web container would need a minimum of a service (to listen on port 80 or 443), and a deployment (to specific how many containers and what image). In a Helm chart, this would already be available and pre-defined – as a web app is the most common use-case – and just the template values passed through as parameters. Individual YAML files would have to be created, and then names and references copied to each other to work, factoring in sequence order, and changes in one would not flow through to the other.

Terraform is another IaC language that supports containers, however, is exceptionally difficult to manage containers with due to Terraform’s state management – that is, if Terraform does not push or import a change, Terraform can’t handle the change. With the underlying Kubernetes API, there are instances where the Kubernetes API will need to apply changes or alterations due to autoscaling, node management, and other cluster operations. Before a subsequent change, Terraform would need to import these changes. This would have to be a manual process, as the Kubernetes cluster appends values to the deployment which are not relevant to managing the configuration.

Considering all the above, Helm is by far the “best” option for managing container deployments at scale across clusters, as it is purpose-built for Kubernetes. It integrates and works with the Kubernetes API and only manages the configuration that the Helm chart maintainer is defining as in-scope.

Design Decision 46 – Container Deployment Configuration Management

5.11.3     Azure container tooling

5.11.3.1     Azure container registry

Azure container registry is the service used to store built Docker images, Helm charts, and Bicep modules. Images, charts, and modules can be pushed to the registry for re-use in deployments or pipelines in the environment.

5.11.3.2     Azure Kubernetes services

Azure Kubernetes Services is the service that operates Kubernetes within Azure; there are other services that provide Docker hosting. Azure operates the underlying enabling services, such as etcd, and the nodes deployed to operate Kubernetes can be used to host container-based workloads, such as Docker images or PaaS services such as Azure Functions.

5.12          Remote Access

Several mechanisms are available to provide secure remote access into an environment. In a cloud-only environment, the method for determining the most appropriate access approach is determined on whether there is a persistent need to access privately hosted systems. In most organisations, this is achieved using a network layer tool, which establishes a connection to grant access to applications and systems.

For secure environments, or where there is a need to isolate environment traffic from a broader network, this can be achieved through a virtual desktop solution, such as Remote Desktop, or Citrix. For this secure environment, the recommended approach is to leverage Azure Virtual Desktop.

Design Decision 47 – Remote Access Tooling

Design Decision 48 – Restricted Environment Access

5.12.1     Azure remote access tooling

5.12.1.1     Azure virtual desktop

Azure Virtual Desktop (AVD) is an evolution of Windows Remote Desktop Services, where the configuration and infrastructure elements are hosted in Azure, and where session hosts permit a standard Windows GUI interface for a desktop. AVD can be exposed directly to the Internet or configured to only allow connections from specific private IP ranges.

6                       Resiliency

Resiliency comprises multiple subject areas, not just backups. Resilient design of application workloads is essential to businesses and their customers, to ensure that they can transact and perform work.

In the context to this Solution Design, there are three (3) areas relevant to resiliency:

• Availability: the management of time where the application is available to end-users.
• Backups: the methods and copy types of persistent data volumes; and,
• Disaster Recovery: the methods and architecture by which applications can operate in critical situations.

Business Continuity is not a subject covered in this Solution Design, as the ability of the business to perform operations is determined by business processes and objectives. Disaster Recovery is the technology component represented within the broader Business Continuity Plan (BCP).

6.1               Availability

The following availability standards are well-known measurements. The given outage periods have been provided for a per-month schedule.

• 5% (2-and-a-half nines), meaning no more than 3.65 hours of unscheduled outage.
• 9% (3-nines), meaning no more than 43.83 minutes of unscheduled outage.
• 95% (3-and-a-half-nines), meaning no more than 21.92 minutes of unscheduled outage.
• 99% (4-nines), meaning no more than 4.38 minutes of unscheduled outage.

Where possible, these standards will be aligned to. Some tooling; however, has availability dictated by Microsoft due to being Platform-as-a-Service tooling, such as Entra ID.

Design Decision 50 – Availability Targets

Design Decision 51 – Availability Architecture

6.2               Backups

Within the Azure platform, backups will be required for all data elements which are not defined as infrastructure-as-code, such as databases and virtual machines.

Azure Backup will be used to enable periodic backups of virtual machines, these backups will be managed as a backup plan and applied to all virtual machines that are individually deployed.

Design Decision 52 – Backup Tooling

6.3               Disaster Recovery

Existing technology-provided Disaster Recovery principles are as follows:

• Recovery Point Objective (RPO) – 1 hour
  • RPO is the maximum time tolerable for data-loss.
• Recovery Time Objective (RTO) – 24 hours
  • RTO is the maximum time tolerable to return an application to operation.

The DR approach has been shaped by these availability objectives, the technical requirements of the applications, and the cross-region capabilities of the Azure persistence services.

A significant advantage to environment leveraging Infrastructure as Code (IaC) means application configurations backups and DR is greatly enhanced offering of a consistent and repeatable process to rebuild the entire environment in a timely and automated manner. It facilitates the quick rebuild of entire environments in varying regions depending on the scenario and can be validated and tested in regular intervals.

6.4               Azure resiliency tooling

6.4.1          Recovery services vault

Azure Recovery Services Vaults are a combined backup and replication tool for several Azure services. Backups are facilitated for virtual machines, storage accounts, and some compatible database services; replication is performed for virtual machines intra- or inter-region.

7                       Security

Security encompasses a broad range of topics, especially when considering the wide range of capabilities and tools that can cover activities and tasks related to security. For this Solution Design, the scope of ‘Security’ is limited to the following topics or domains:

• Compliance frameworks alignment
• Security log aggregation,
• Security event and incident management
• Web application firewall
• Cloud security posture management (CSPM)
• Cloud workload protection (CWPP)
• Cloud-native application protection (CNAPP)
• Virtual machine endpoint protection (EPP)

Other items, such as authentication mechanisms or best-practice configuration settings like TLS, are not covered under this section. Instead, the Standard defined for Security Configuration should be referred to.

7.1               Compliance Framework Alignment

With any environment, the organisation for which that environment is deployed to support is subject to one or more compliance frameworks. As part of building and managing that environment, those frameworks need to be known to support implementation of secure configurations to meet those framework control points.

Design Decision 53 – Compliance Framework(s) Alignment

7.1.1          Cloud security posture management

Cloud Security Posture Management (CSPM) tooling is a new capability within cloud services, to be able to report on configuration state for each individual resource and assess that configuration against a one (1) or more framework benchmarks. Framework benchmarks are created by the vendor of the CSPM tooling, or by the consumer if the tool supports custom assessments.

Whilst Azure supports this out-of-the-box with Azure Policy and Azure Defender for Cloud, the management and reporting capabilities for this are extremely immature. Several off-the-shelf items are available, however, any policy which is not available needs to be configured and build manually using Azure Policy. Azure Policy, in turn, is extremely cumbersome to manage, as each policy setting first needs to be created, then rolled-up to an Azure Policy ‘initiative’, which only then can be reported on.

Design Decision 54 – Cloud Security Posture Management Tooling

7.2               Security Log Aggregation

Security logs form a significant basis to the ability of teams to be able to analyse, categorise, and investigate issues which pertain to security for an environment. A central log store provides a single location for those activities to occur, without needing to add additional log locations.

Azure Log Analytics, a component of Azure Monitor, is a native capability within Azure that logs can be fed into directly. It is also the service underpinning Microsoft’s Sentinel SIEM / SOAR solution.

Design Decision 55 – Security Log Aggregation

7.3               Security Event and Incident Management

With any environment, the ability to analyse logs at scale and provide rule-based alerting is critical for any security scenario. With Microsoft Sentinel – a modern, Azure & cloud-native event, incident, and security automation platform – leveraged over the top of Azure Log Analytics, this provides security teams and executive leadership a strong capability and tool to be able to secure and defend even the most complex cloud environments.

Design Decision 56 – Security Incident and Event Management Tooling

7.4               Web Application Firewall

Web application firewalls (WAF) operate at the session layer of a network (aka Layer 7). These tools allow for a comprehensive understanding of the traffic that is traversing them, able to inspect, detect, and prevent malicious or undesirable activity.

There are Azure native solutions that can support this capability, along with a wide range of non-native solutions as well.

When looking at the broader security suite in operation in a given Azure environment, whilst a single underpinning detection & analysis stack is less complex architecturally and licensing-wise, this does introduce risk in that if a single component of that stack misses an event, the entire chain is likely to miss it.

The modern WAF solution requires the ability to support not just raw web / HTTP traffic, but also protect APIs, assist in discovery of configuration changes, and be managed as infrastructure-as-code to support DevSecOps and CI/CD processes.

At this point in time, a Web Application Firewall deployment is not identified as a requirement for this environment.

7.5               Cloud Workload Protection

Cloud Workload Protection Platforms (CWPP) are security tools which can secure not just traditional workloads, such as virtual machines or servers, but can also secure modern cloud workloads such as containers, PaaS storage and database platforms, and similar.

In a cloud-first or cloud-native environment, not all endpoint protection solutions can secure or supporting PaaS resources. The right CWPP solution is scalable and extensible, supporting a wide range of cloud resources, regardless of ‘as-a-Service’ archetype.

CWPP solutions can be agent-based, agentless, or a hybrid mix of both depending on components and application solution architecture.

Design Decision 59 – Cloud Workload Protection Tooling

7.6               Cloud-native Application Protection

Cloud-native Application Protection Platforms (CNAPP) are an holistic integration architecture of CSPM, CWPP, and Cloud Service Network Security (CSNS) tools -though that last one is far less common. A CNAPP solution should be able to integrate their CWPP and CSPM metrics, analyses, and reporting to provide an organisation-wide view on cloud platforms, resources, configuration, and risk profile.

Design Decision 60 – Cloud-native Application Protection Tooling

7.7               Virtual Machine Endpoint Protection

The most familiar security tool to end-users and IT professionals alike, endpoint protection (EPP) is the all-encompassing term for ‘next-generation’ anti-virus (NGAV) and enhanced-detection-and-response (EDR) tooling. Compared to traditional ‘signature’ based anti-virus solutions, EPP solutions deploy agents to endpoints, to collect and stream activity logs from supported endpoints – usually virtual machines or servers – to build a pattern of operational behaviour to determine threats. This is enhanced through the use of shared threat intelligence which can be updated on the fly in the back-end of the EPP vendor, allowing for rapid iteration and detection of new and emerging threats without the endpoint needing to receive updates or ‘definitions’.

Design Decision 61 – Virtual Machine Endpoint Protection Tooling

7.8               Vulnerability Management

Vulnerability management is a process of identifying, evaluating, prioritising, and mitigating security vulnerabilities in an organisation’s IT infrastructure, systems, and applications. The goal of implementing a vulnerability management program is to assist in proactively prevent security breaches and protect critical assets by addressing vulnerabilities before they can be exploited.

To achieve this, a vulnerability assessment capability is used to support the vulnerability management process. Vulnerability assessment solutions identify and evaluate workload and/or network vulnerabilities by constantly scanning and monitoring the solution attack surface for risks. With the use of cloud-native, public cloud platforms that are dynamic, components of the assessment workflow differ significantly from vulnerability assessments for on-premises datacentres, due to the split-responsibility model with public cloud.

Solutions that support both may overlap, however, solutions built for traditional infrastructure are not as mature for cloud-native offerings. In the inverse, solutions built for cloud-native workloads typically ignore or are not supported for traditional Infrastructure-as-a-Service capabilities.

Design Decision 62 – Vulnerability management (containers)

Design Decision 63 – Vulnerability management (IaaS VMs)

8                       Operations

8.1               Logs and Metrics

Metrics for Azure hosted resources will be collected in Azure Monitor. These will be used to trigger alerts. All logs for services running inside the Azure platform will be shipped to Azure Monitor. This will provide for tamper-proof, near real-time auditing.

Design Decision 64 – Monitoring Tooling

Design Decision 65 – Operational Log Aggregation

8.2               Alerting

Azure Monitor will be utilised for alerts relating to the Azure infrastructure. Alerts will be configured to include a webhook call (or similar) to an ITSM solution, such as ServiceNow. This will minimise the time taken for these alerts to be issued and tracked in ITSM ticketing solutions.

This eliminates Azure Monitor as a single point of failure in the alerting chain. Additionally, it provides more contextual information about each alert and direct links back to the source monitoring system to investigate and resolve the issue.

8.3               Update Management

Update Management refers to the practice and process by which security and software updates are applied to workloads. Within the Azure environment, the following workload types require a configuration for managing updates:

• Azure Virtual Desktop Session Hosts
• Virtual Machines
• Container Images
• Azure Kubernetes Services

There are several tools suitable for use within Azure which support different capabilities for managing updates, each of which have their own limitations and caveats. Several of the current generally available services are due to be decommissioned in 2024, as Azure phases out support of the legacy Azure Log Analytics agent.

Design Decision 66 – Update management for Azure Virtual Desktop Session Hosts

Design Decision 67 – Update management for Virtual Machines

Design Decision 68 – Update management for Container images

Design Decision 69 – Update management for Azure Kubernetes Services

• References
  • Design References

Tenant Hierarchy

Contents

1       Overview.. 4

1.1         Purpose. 4

1.2         In-scope. 4

1.3         Design Requirements. 4

2       Solution Architecture. 5

2.1         Conceptual Architecture. 5

3       Azure Governance Hierarchy. 6

3.1         Management Groups. 6

3.2         Subscriptions. 8

4       Resource Locks. 10

5       Monitoring Customer Experience. 11

Appendix A      Enterprise Scale Landing Zone Components. 12

Appendix B      Conceptual ESLZ Structure. 13

Tables

Table 1 – Project Details. 2

Table 2 – RLX Contact Details. 2

Table 3 – RLX Version Control Details. 2

Table 4 Design Requirements. 4

Table 5 Management Groups. 7

Table 6 Subscription Groups. 8

Table 8 ESLZ Components. 12

Figures

Figure 1 Conceptual Organisation. 5

Figure 2 Management Groups. 6

Figure 3 Subscription Structure. 8

Figure 4 Subscription Groups. 9

Figure 5 Conceptual ESLZ Architecture. 13

9                       Overview

9.1               Purpose

This document provides the detailed design for the Management and Subscription Group deployment as stipulated in the Enterprise Scale Landing Zone Solution Design document.

9.2               In-scope

The following components of the platform are in scope for this document:

• Management Group and Subscription Organisation.
• Naming and Tagging convention.

9.3               Design Requirements

The following are the design requirements inherited from the high-level solution design.

Table 4 Design Requirements

10                 Solution Architecture

The CUST Tenancy is the foundational infrastructure capability provider, from which CUST can design, build, deploy, and manage the full spectrum of applications required to support all business technology initiatives. 

10.1          Conceptual Architecture

The conceptual state hosting platform is the Azure Cloud. Azure IaaS and PaaS services will be used to

provide the platform level support services.
­­

Figure 1 Conceptual Organisation

11                 Azure Governance Hierarchy

11.1          Management Groups

Management Groups are used to provide enterprise grade management at scale, providing a flexible hierarchy for unified policy and access management.

Figure 2 Management Groups

The following table outlines the Management Groups that will be deployed for CUST. 

Table 5 Management Groups

11.2          Subscriptions

Subscriptions provide the primary administrative, security and billing boundaries. Every Azure resource (VM, network, managed database, storage account etc.) belongs to a Subscription. The Management and connectivity subscriptions will be segregated into non-production and production.

Figure 3 Subscription Structure

Table 6 Subscription Groups

Figure 4 Subscription Groups

12                 Resource Locks

Resource locks are an additional Azure control that can be applied to prevent the alteration or deletion of critical resources and infrastructure in the environment. Adding resource locks adds an additional degree of defence to prevent users with malicious intent or from making unintentional mistakes.

There are currently two supported lock types:

1. Read-Only: Allows read-only access to resources but prevents modification and deletion.
2. Delete: Allowed authorised modification but prevents deletion.

Resource locks will be applied at a range of different levels across the enterprise scale landing zone. This includes at subscription, resource group, resource levels, which will be defined in the As-Built documentation at build completion.

13                 Monitoring Customer Experience

Customer Experience is one of RLX’s core values. Meeting and exceeding our customers’ expectations is a goal we take very seriously. Our aim is to be a trusted partner, solving our customer’s cyber security challenges with intelligence led solutions that are outcome focused and operationally informed.

To ensure we are meeting this goal and continuously improving our customers’ experience, RLX has implemented a series of formal and informal measures to regularly measure customer satisfaction. These include identifying key points of contact, regularly scheduled meetings, and a formal Voice of the Customer platform utilising the industry best practice Net Promoter Score (NPS). NPS measurement is applied across all areas of RLX and focuses on two key components of customer satisfaction: transactional (related to a particular project or interaction) and relationship (the overall relationship between RLX and our customers across multiple stakeholders).

As a RLX customer, you may from time to time receive a short survey requesting your feedback on us as an organisation and the services we provide.  Surveys will take no longer than a few minutes to complete.  All results are regularly reviewed by RLX’s leadership, along with any subsequent improvement activities undertaken as a result.  While there is an option to unsubscribe if you prefer, we appreciate your honest feedback and thank you in advance for taking the time to respond.

• Enterprise Scale Landing Zone Components

Table 7 ESLZ Components

• Conceptual ESLZ Structure

Figure 5 Conceptual ESLZ Architecture

Naming Standard

Standard

Contents

1       Overview.. 5

2       Environment Names. 6

2.1         Tenant ID.. 6

2.2         Environments. 6

2.3         Tiers. 6

3       Azure Active Directory. 7

3.1         User accounts. 7

3.2         Privileged user accounts. 7

3.2.1          Administrative privileged accounts. 7

3.2.2          Security privileged accounts. 7

3.3         Emergency access accounts. 8

3.4         Enterprise Applications. 8

3.5         Service Principals. 8

3.6         Groups. 9

3.7         Conditional Access Locations. 9

3.8         Conditional Access Policies. 10

4       Azure Resources. 12

4.1         Reserved Words. 13

4.2         Regions. 13

4.3         Management. 14

4.4         Policy. 14

4.5         Networking. 14

4.6         Compute. 14

4.7         Azure Virtual Desktop. 15

4.8         Modern Apps. 15

4.9         Secrets Management. 15

4.10      Storage. 15

4.11      NetApp Files. 15

5       Monitoring Customer Experience. 16

Tables

Table 1 – Project Details. 2

Table 2 – RLX Contact Details. 2

Table 3 – RLX Version Control Details. 2

Table 4 – RLX Version Control Changelog. 2

Figures

No table of figures entries found.

14                 Overview

This document is intended for use to guide the creation and configuration of resources within Azure and Entra ID for this environment. Specifically, the use of a naming standard or convention to be able to identify the scope at which a resource is used for.

Note that throughout this document, upper-case letters have been used to identify characters used within names. When actually deployed to Azure or Entra ID, all names should be deployed using lower case.

15                 Environment Names

This Azure environment will host several different environments, which will be used to develop, support, and operate application workloads. The below table describes the convention by which each environment will be named.

15.1          Tenant ID

Each tenant is assigned a 3-letter identifier. The tenant identifier for this environment is:

• Australian CUST Corporation (CUST)

15.2          Environments

There are three (3) discrete environments, namely:

These environments are isolated from each other and may be comprised of one or more tiers.

15.3          Tiers

The following table describes the individual tiers, or ‘sub-environments’, within each named Environment.

16                 Azure Active Directory

16.1          User accounts

User accounts are unprivileged accounts which are used for day-to-day activities, such as accessing email or communicating with partner organisations.

User accounts are to be created with the following standard:

givenname.surname@support.CUST.au

Where multiple users share identical given names and surnames, eg two (2) John Smith’s, the following standard will apply:

First user: givenname.surname@support.CUST.au

Second user: givenname.surname1@support.CUST.au

The integer identifier will increment by 1 for each subsequent user of the same given name and surname.

16.2          Privileged user accounts

Privileged accounts are accounts used when performing work in a technical system which has permissions or roles assigned which allow for reading, changing, or otherwise editing configuration settings.

16.2.1     Administrative privileged accounts

Privileged user accounts are to be created with the following standard:

adm-<user account name>@CUST.onmicrosoft.com

The use of the Microsoft tenant name, rather than the custom domain name, is to assist in identification and to reduce risk in the event of issues with the custom domain name.

16.2.2     Security privileged accounts

Privileged user accounts are to be created with the following standard:

sec-<user account name>@support.CUST.au

16.3          Emergency access accounts

Emergency access, or so-called “break-glass” accounts are privileged accounts which are used in the event of an emergency, when ‘normal’ access through user and/or privileged accounts is not feasible.

Rather than define a convention, the following table describes the five (5) emergency access accounts to be created.

Additionally, each of these accounts has a physical multi-factor authentication token attached to each of them.

16.4          Enterprise Applications

Enterprise Applications are Entra ID objects used for managing user access to applications. This could be an integrated single-sign-on application, such as Microsoft Office, or custom applications built for a tenant.

The following table describes the guidance attached to Enterprise Applications:

16.5          Service Principals

Service Principals are Entra ID objects used for configuring application authentication to Entra ID. This could be used for an application to support SSO with an Enterprise Application or a client application used to execute privileged tasks within Azure.

They can also be known as “App Registrations”. Service principals are to be created with the following standard:

<Identifier>-<Application Short Name>-<Party>-<Function>

16.6          Groups

Groups in Entra ID are used to logically group items for access, roles, or similar. Groups can be used to manage access to roles, applications, or resources within Azure.

Groups are to be created with the following standard:

<Identifier>-<Area>-<Subject>-<Role>

16.7          Conditional Access Locations

To avoid confusion, the use of a strict location naming policy is recommended. While something like Australia is obvious, “Melbourne Office” can become confusing where there are multiple matching locations or multiple network providers. The below are recommendations but should be fine-tuned, relevant to the organisation.

Conditional Access locations are to be created with the following standards:

Corporate locations: <Country Code>-<City Code>-<Type>-<Address Summary>

Provider locations: SVCLOC-<Provider>-<Usage>

16.8          Conditional Access Policies

For managing two or three basic policies, organisations can avoid a strict naming policy. However, when you start managing different use case scenarios, whether that be groups of users, applications, or controls, add in multiple administrators and continued growth, it will rapidly become an unmanageable beast. Any naming policy should use the following key attributes:

• A reference number for change management; and,
• The application(s) in scope; and,
• The controls enforced; and,
• Who it applies to; and,
• When it applies

Conditional Access policies are to be created with the following standard:

<Policy Number> – <User Scope> – <Applications> – <Conditions> – <Action> – <Control>

17                 Azure Resources

When being deployed to Azure, each resource type needs to be identifiable, to understand:

• What type of resource it is
• Where the resource is deployed
• What environment or environment tier it is deployed to

This is essential to understand the number of resources are in use and how they are consumed. Names, when combined with tags, are used to produce a comprehensive breakdown of what “assets” are present within the environment.

Azure resources are to be created with the following standard, based on permitted characters:

Parameters for Azure resources are described below.

17.1          Reserved Words

The following table contains the words which are reserved for use and cannot be used to create Azure resources.

17.2          Regions

Azure operates in multiple geographical regions as well as some resources which operate region-agnostically. The following table describes the list of regions in use with this environment and the ‘short form’ name each region will be assigned in names.

17.3          Management

17.4          Policy

17.5          Networking

17.6          Compute

17.7          Azure Virtual Desktop

17.8          Modern Apps

17.9          Secrets Management

17.10    Storage

17.11    NetApp Files

Security Standard

Contents

w     Overview.. 5

w     Cryptographic Standards. 5

o       Asymmetric cryptography. 5

w     Asymmetric cryptographic algorithms. 5

w     Asymmetric cryptographic key exchanges. 6

o       Symmetric cryptography. 6

w     Symmetric cryptographic algorithms. 6

w     Symmetric cryptographic ciphers. 7

o       Hash algorithms. 7

o       Hash-based message authentication codes. 7

o       Protocols. 8

w     Transport Layer Security. 8

w     Secure Shell 9

w     Internet Protocol Security. 9

w     Certification Authorities. 11

o       Root Certification Authorities. 11

o       Intermediate Certification Authorities. 11

w     Update Management. 11

o       Virtual machine image updates. 12

o       Virtual machine update management. 12

w     Example calendar view.. 12

o       Container image updates. 13

w     Monitoring Customer Experience. 14

Appendix A      Document References. 15

Tables

Table 1 – Project Details. 2

Table 2 – RLX Contact Details. 2

Table 3 – RLX Version Control Details. 2

Table 4 – RLX Version Control Changelog. 2

Table 5 – Asymmetric encryption comparison in bit-size. 5

Table 6 – Preference order for asymmetric encryption algorithms. 5

Table 7 – Preference order for asymmetric key exchange algorithms. 6

Table 8 – Preference order for symmetric encryption algorithms. 6

Table 9 – Preference order for symmetric ciphers. 7

Table 10 – Hash algorithm preference order. 7

Table 11 – HMAC preference order. 8

Table 12 – TLS 1.3 ciphersuite preference order. 8

Table 13 – TLS 1.2 ciphersuite preference order. 9

Table 14 – IPSec IKE Phase 1 minimum settings. 10

Table 15 – IPSec IKE Phase 2 minimum settings. 10

Table 18 – Virtual machine image update schedule. 12

Table 19 – Virtual machine update schedule. 12

Figures

No table of figures entries found.

• Overview

This Standard document is intended to act as the guiding document when deploying Cloud resources and configurations. Whilst that is not an exhaustive scope item, this should cover most use cases within the Azure platform.

• Cryptographic Standards
  • Asymmetric cryptography

Asymmetric cryptography, also known as public key cryptography or public/private keypairs, is a form of cryptography where different keys are used to encrypt and decrypt data. The public key can only be used to encrypt data and the private key is used to decrypt data.

This type of cryptography is most frequently encountered when browsing to a website over HTTPS, also known as SSL or TLS, or when using public key authentication for Secure Shell (SSH) and Secure Shell File Transfer (SFTP).

• Asymmetric cryptographic algorithms

There are many implementations of asymmetric cryptography in use, determined based on the method of mathematical computation of the private key. Each of these algorithms can generate keys of different sizes, where smaller keys are both less intensive to generate and easier to permutate. Algorithms are most frequently used to generate X.509 certificates, which serve as digital signatures, for a variety of functions such as HTTPS certificates.

The following table shows a key size comparison between RSA and Elliptic-Curve (ECDSA) algorithms.

Table 5 – Asymmetric encryption comparison in bit-size

The following combination of algorithms and bit sizes are listed in descending preference order, as a recommendation in-line with the Australia Cyber Security Centre’s guidance on asymmetric cryptography.

Table 6 – Preference order for asymmetric encryption algorithms

• Asymmetric cryptographic key exchanges

When two systems communicate, or desire to, through an encrypted channel, these systems must perform a ‘handshake’ where the two systems agree on a shared set of elements – an algorithm, a cipher, and a message authentication code. As part of this handshake, the two systems perform an exchange of public keys to agree on a single key for the duration of the channel.

The following key exchanges are approved for use, listed in descending preference order, as a recommendation in-line with the Australia Cyber Security Centre’s guidance on asymmetric cryptography.

Table 7 – Preference order for asymmetric key exchange algorithms

• Symmetric cryptography

Symmetric cryptography is a form of cryptography where the same key is used to both encrypt and decrypt data.

• Symmetric cryptographic algorithms

There are many implementations of symmetric cryptography in use, determined based on the method of mathematical computation of the private key. Each of these algorithms can generate keys of different sizes, where smaller keys are both less intensive to generate and easier to permutate. The most common symmetric algorithm in use today is AES (Advanced Encryption Standard).

The following combination of algorithms and bit sizes are listed in descending preference order, as a recommendation in-line with the Australia Cyber Security Centre’s guidance on symmetric cryptography.

Table 8 – Preference order for symmetric encryption algorithms

• Symmetric cryptographic ciphers

In this context, ciphers are the method by which data is encrypted either as a block or a stream of data. These are respectively known as block ciphers and stream ciphers.

The following symmetric ciphers are listed in descending preference order, as a recommendation in-line with the Australian Cyber Security Centre’s guidance on symmetric cryptography.

Table 9 – Preference order for symmetric ciphers

• Hash algorithms

A hash function, hash or hashing, is a method or algorithm which transforms a block of data of a given size to a hexadecimal string. Hash functions operate one-way; that is, performing a hash of content cannot be ‘unhashed’. Rather, using a hash algorithm on the same content always results in the same hash output. As an example, the string “fox” might be translated to “ae5512b04cf” through a hash function.

Hash functions are frequently used when storing passwords, along with a pseudo-random component known as a ‘salt’ which enables the resulting ‘hashed’ content to be different, even with the same source content. This also increases the computational power required to perform the hashing process, which in turn increases the security of the end-result.

The following list of hashing functions are listed in descending preference order, as a recommendation in-line with the Australia Cyber Security Centre’s guidance on asymmetric cryptography.

Table 10 – Hash algorithm preference order

• Hash-based message authentication codes

A hash-based message authentication code (HMAC) provides a value to assess the integrity and authenticity of data. MACs are frequently used when downloading files from the Internet to compare the signatures provided by the host to ensure the integrity of the downloaded files is what was posted by the host.

Table 11 – HMAC preference order

• Protocols

With the above asymmetric and symmetric standards, this has downstream impacts on the configuration of protocols, such as TLS or SSH. In addition, due to the significant change in baseline requirements from TLS version 1.2 to version 1.3, this has proscribed several options which were previously available.

As a result, the following standards are recommended.

• Transport Layer Security

Transport Layer Security (TLS) is the successor protocol to Secure Sockets Layer (SSL), which is used to establish secure connections to server-hosted applications. In client/server architecture, servers hold asymmetric keypairs – a public & private key – which is used to identify and trust servers to clients. The most common implementation is for HTTPS certificates for websites.

• TLS 1.3

Due to the changes mentioned above, TLS v1.3 presents a significantly different view in comparison to the TLS v1.2 list. TLS v1.3 cipher suite strings do not specific key exchange methods, as the requirement is to support ‘perfect forward secrecy’ (PFS) which mandates the use of the ECDHE or DHE key exchange methods.

The below list is provided in descending preference order.

Table 12 – TLS 1.3 ciphersuite preference order

• TLS 1.2

This table lists the approved TLS v1.2 cipher suites, comprised of a key exchange method, asymmetric algorithm, symmetric algorithm, and hash function, in descending preference order.

Table 13 – TLS 1.2 ciphersuite preference order

• Secure Shell

Secure Shell (SSH) is a command-line interface tool used for interacting and performing administrative tasks on systems, typically Linux. There are open-source and commercially-licensed software that supports this protocol, the most common being OpenSSH. OpenSSH enabled three (3) asymmetric key types out-of-the-box for SSH: ed25519, ECDSA, and RSA.

When implementing SSH, the following sequence of steps should be performed:

1. Remove the ed25519 configuration
2. Regenerate SSH’s ECDSA private key
3. Regenerate SSH’s RSA private key

This will ensure that each SSH private key is randomly generated and avoid re-use of the same private key across multiple systems.

• Secure Shell File Transfer Protocol

Secure Shell File Transfer Protocol (SFTP) is a method of performing file transfer activities over SSH. This is separate to SCP, which also uses SSH for file transfer activities. SFTP configurations should align to the SSH guidance, use separate keys, and block SSH commands from occurring over SFTP connections.

• Internet Protocol Security

The Internet Key Exchange (IKE) protocol is the foundation to building security associations (SA) for establishing site-to-site VPN connections over Internet Protocol Security (IPsec).

IKE has two active versions, IKEv1 and IKE2.

IKE also has two phases of security association, phase 1 and phase 2. Phase 1 serves to establish the secure communication channel using a key exchange algorithm. Phase 2 then acts to use the secure channel established in phase 1 to negotiate the IPsec communication.

• IKE Phase 1 Minimum Settings

Table 14 – IPSec IKE Phase 1 minimum settings

• IKE Phase 2 Minimum Settings

Table 15 – IPSec IKE Phase 2 minimum settings

• Certification Authorities

Asymmetric cryptography is most frequently used for HTTPS certificates and other digital signatures to establish integrity, confidentiality, and authenticity. Certification Authorities (CA), also called Certificate Authorities, are components in the wider Public Key Infrastructure (PKI) used to manage certificates, certificate templates, chains of certification authority trusts, and validation of certificates.

There are two types of Certification Authorities: Root and Intermediate.

• Root Certification Authorities

Root Certification Authorities are the devices which form the ‘root’ of any chain of certificate hierarchy. The certificates or signatures that get issued to websites always track back up to a Root CA. In any Public Key Infrastructure or certificate chain there is only ever one Root CA for each chain.

• Intermediate Certification Authorities

Intermediate Certification Authorities are devices which are either issued a digital signature by a Root CA or another Intermediate CA higher in the certificate trust chain, which in turn then also issue certificates.

An Intermediate CA is also known as an ‘issuing CA’, in that opposed to being the root of a trust chain, the Intermediate CA issues out certificates or signatures on behalf of its’ parent Root or Intermediate CA. It is these devices which issue out web server certificates.

• Update Management

When compute is deployed to an environment, whether a virtual machine or container, it will need to receive operating system and application updates to remain secure. Within this environment, there are several different approaches to managing this process.

For the context of this Standard, an “update” is any patch, security or otherwise, which increments the version of the application or operating system. This does not include a version change, such as Windows 10 to Windows 11, or OpenSSL 1.1 to 3.0.

There are several considerations depending on the source operating system for each element.

• Linux operating systems do not have a set update release cycle. Instead, updates are released as soon as they are release-ready.
• Windows operating systems have updates released on a well-known schedule; the second Tuesday of every month (Patch Tuesday). Due to time delays, for Australia, this is a Wednesday.
• Operating system containers are released based on vendor release schedules, which are not necessarily public.
• Application containers are updated in-line with vendor release schedules, which are not necessarily public.

The below individual items will cover standard approaches. Exception management, such as a Linux kernel CVE patch or other equivalent critical fix, will always take effect over these standard approaches.

• Virtual machine image updates

Virtual Machine images, aka SOE images, will be created once per month.

To enable Windows VM images to receive the latest patches as part of build, the following schedule will be used to create VM images.

Table  – Virtual machine image update schedule

• Virtual machine update management

Azure Update Management Centre will be used to patch virtual machines. This supports manual and scheduled patching.

The following table describes the proposed standard schedule for updates to VM operating systems. As the virtual machines are not expected to directly face to the Internet, either behind a WAF or otherwise, there is not a sufficient need to patch virtual machines within a few days of a patch release.

Table  – Virtual machine update schedule

• Example calendar view

• 12^th of the month: Development
• 15^th of the month: Integration
• 17^th of the month: UET + Training
• 18^th of the month: Staging (Disaster Recovery)
• 22^nd of the month: Disaster Recovery
• 24^th of the month: Staging (Production)
• 29^th of the month: Production

• Container image updates

Container image updates are pushed or released on a schedule determined by the developer or vendor of the specific container.

Platform container images will be re-tagged and pushed to a hosted container registry within 2 business days of release.

Network Detailed Design

Contents

1       Target State Network Design. 6

1.1         Hub-and-spoke network topology. 6

1.2         IP address supernet allocation. 7

1.2.1          Allocation breakdown. 7

1.3         Resource groups. 7

1.3.1          Production. 7

1.3.2          Disaster Recovery. 7

1.3.3          Non-Production. 8

1.3.4          Sandpit. 8

1.4         Virtual networks. 8

1.4.1          Peering. 8

1.4.2          Subscription subnets. 9

1.5         Route tables. 10

1.5.2          Virtual Network Gateway route table. 11

1.5.3          Kubernetes subnets route table. 11

1.6         Network security groups. 12

1.6.1          ESLZ subnets. 12

1.6.2          Jira application subnets. 13

1.7         Public IP prefixes. 13

1.8         IP address management tooling. 13

1.9         ExpressRoute Connectivity. 13

1.9.1          ExpressRoute Circuit. 14

1.9.2          Virtual Network Gateway. 14

1.9.3          Peering Configuration. 14

1.9.4          Palo Alto VM-Firewall 15

2       Name Services. 15

2.1         DNS Zone Configuration. 15

2.1.1          Domain(s). 15

2.1.2          Internal DNS Zone(s). 16

2.1.3          Public DNS Zone(s). 16

2.1.4          Subdomain delegation. 16

2.2         Private endpoints. 17

3       Monitoring Customer Experience. 18

Tables

Table 1 – Project Details. 2

Table 2 – CBX Contact Details. 2

Table 3 – CBX Version Control Details. 2

Table 4 – CBX Version Control Changelog. 2

Table 5 – CIDR Supernet allocation. 7

Table 6 – CIDR allocation breakdown per environment tier. 7

Table 7 – Resource groups for Production network elements. 7

Table 8 – Resource groups for Disaster Recovery network elements. 8

Table 9 – Resource groups for Development network elements. 8

Table 10 – Resource groups for Sandpit network elements. 8

Table 11 – Virtual Networks and CIDR allocations. 8

Table 12 – Hub and spoke network peering assignments. 9

Table 13 – Connectivity Virtual Network subnets. 9

Table 14 – Identity Virtual Network subnets. 9

Table 15 – Security Virtual Network subnets. 9

Table 16 – Integration Virtual Network subnets. 10

Table 17 – Management Virtual Network subnets. 10

Table 18 – Atlas Virtual Network subnets. 10

Table 19 – Default Route Table configuration. 10

Table 20 – Example Default Route Table configuration. 11

Table 21 – VNG Route Table configuration. 11

Table 22 – K8s Route Table configuration. 12

Table 23 – Example K8s Route Table configuration. 12

Table 24 – AVD non-default NSG rules. 13

Table 25 – Active Public IP prefixes. 13

Table 26 ExpressRoute Circuit. 14

Table 27 Virtual Network Gateways. 14

Table 28 ExpressRoute Pre-requisites. 15

Table 29 Palo Alto VM Appliances. 15

Table 30 Environment domains in use. 16

Table 31 Internal DNS Domain. 16

Table 32 External DNS domains per environment. 16

Figures

Figure 1 – Hub-and-spoke Topology. 6

18                 Target State Network Design

This section describes the target, or end-state, design and topology of the network within the Azure environment.

18.1          Hub-and-spoke network topology

The below diagram illustrates the design intent for the hub-and-spoke network for this environment, and for the environment hubs for each environment tier.

Figure 1 – Hub-and-spoke Topology

18.2          IP address supernet allocation

The below table identifies the total IP address range allocation for Azure in Classless Inter-Domain Routing (CIDR) format. The CIDR format is used throughout this document.

Table 5 – CIDR Supernet allocation

18.2.1     Allocation breakdown

Note that not all the environments below will be used. They have been provided in advance for network allocation purposes only.

Table 6 – CIDR allocation breakdown per environment tier

18.3          Resource groups

The below sections identify all the resource groups in each Azure subscription where network elements will be deployed. Detail into each subscription name and type can be found in the Tenant Hierarchy Solution Design document.

18.3.1     Production

Table 7 – Resource groups for Production network elements

18.3.2     Disaster Recovery

Table 8 – Resource groups for Disaster Recovery network elements

18.3.3     Non-Production

Table 9 – Resource groups for Development network elements

18.3.4     Sandpit

Table 10 – Resource groups for Sandpit network elements

18.4          Virtual networks

Table 11 – Virtual Networks and CIDR allocations

18.4.1     Peering

Table 12 – Hub and spoke network peering assignments

18.4.2     Subscription subnets

Each CIDR allocation in the below sections and tables only identify the third (3^rd) and fourth (4^th) octet. This is due to the first (1^st) and second (2^nd) octets being consistent across each environment. The first and second octets are identified as <CIDR Prefix> in each row.

18.4.2.1     Connectivity

Table 13 – Connectivity Virtual Network subnets

18.4.2.2     Identity

Table 14 – Identity Virtual Network subnets

18.4.2.3     Security

Table 15 – Security Virtual Network subnets

18.4.2.4     Integration

Table 16 – Integration Virtual Network subnets

18.4.2.5     Management

Table 17 – Management Virtual Network subnets

18.4.2.6     Jira

Table 18 – Atlas Virtual Network subnets

18.5          Route tables

As part of the hub-and-spoke implementation, subnets in Virtual Networks need to be instructed where to send traffic. This is achieved in this design through User-Defined Routes (UDRs), implemented through Route Tables.

The below table defines the default route table attached to each subnet except where noted in subsequent sections.

Table 19 – Default Route Table configuration

The Vnet-Local route is required to be added to ensure that inter-subnet traffic is inspected by the firewall. Without this route, traffic in one subnet can pass directly to another subnet in the same virtual network.

18.5.1.1     Example Route Table for Nonproduction Management Subscription

Table 20 – Example Default Route Table configuration

18.5.2     Virtual Network Gateway route table

This route table defines the route table attached to the ‘GatewaySubnet’ in the Connectivity hub virtual network.

This specific table requires Internet access directly, as the Virtual Network Gateways require Public IP addresses, and, that traffic for each Virtual Network CIDR traverses the local region Azure Firewall.

Table 21 – VNG Route Table configuration

18.5.3     Kubernetes subnets route table

Due to the architecture of separating AKS system nodes from user nodes, to ensure separation of workloads and optimise network utilisation for scaling, Kubernetes route tables should be configured to ensure individual subnets communicate directly rather than traversing an inspection appliance.

This is particularly important for autoscaling nodes, where the IP address of each node may change over small intervals of time.