Long running process in Linux using PHP

 

Aligning Microsoft SDAF with a Pre-Existing Azure Landing Zone

Part 1: How to Architect Brownfield SAP Deployments on Azure Without Rebuilding Your Network

---

1. The Enterprise Dilemma: When Frameworks Meet Reality

In an ideal greenfield cloud implementation, automation tools are handed a blank slate. You give Microsoft’s SAP Deployment Automation Framework (SDAF) an empty Azure subscription, run a script, and watch it deploy a clean hub-and-spoke virtual network, subnets, route tables, network security groups (NSGs), Key Vaults, and compute nodes tailored strictly to SAP best practices.

However, in large enterprise environments—such as telecommunications, banking, and global conglomerates—cloud infrastructure rarely begins with a blank canvas.

Before the SAP basis or automation engineers even receive access, a cloud platform team or third-party vendor has typically already provisioned a centralized Enterprise Azure Landing Zone (ALZ). This landing zone comes with rigid corporate standards:

• Dedicated ExpressRoute circuits or VPN gateways terminated into a corporate Hub network.
• Pre-allocated Classless Inter-Domain Routing (CIDR) blocks carved out by enterprise IP Address Management (IPAM).
• Strictly governed Network Security Groups (NSGs) and User-Defined Routes (UDRs) funneling all egress through central Azure Firewalls.
• Multi-subscription segregation separating shared services, development, QA, and production.

Figure 1: High-level architectural overview of SDAF brownfield integration with an enterprise Azure landing zone. 

When our team was tasked with automating the deployment of SAP S/4HANA 2022 (FPS00 on SUSE Linux Enterprise Server 15 SP5 with SAP HANA 2.0 SPS07), we had a strategic mandate: Use Microsoft SDAF as the core deployment and orchestration engine.

The conflict was immediate:

• Do we tear down the vendor's landing zone? No. Tearing down the vendor's landing zone would break enterprise firewall routing, IP allocations, and multi-team governance.
• Do we abandon SDAF and write custom Terraform from scratch? No. Doing so would forfeit SDAF's automated HANA TDI storage layouts, High Availability clustering configurations, Bill of Materials (BOM) package orchestration, and Ansible playbooks.

The only viable enterprise strategy is Brownfield SDAF Integration: configuring Microsoft SDAF to adopt and operate seamlessly inside the pre-existing Azure Landing Zone.

In this first part of our series, we explain the mechanics of how SDAF models infrastructure and provide a step-by-step guide to configuring SDAF's .tfvars files to target pre-installed Azure networking without duplicating resources or breaking connectivity.

---

2. Understanding SDAF’s Layered Architecture

To integrate with an existing landing zone, you must first understand how Microsoft SDAF divides responsibilities across distinct layers:

1. The Control Plane (Deployer & SAP Library):
   • Deployer: A hardened Linux VM running Terraform and Ansible that acts as the execution runner.
   • SAP Library: Azure Storage accounts that store Terraform state files and the multi-gigabyte SAP installation binaries (BOMs).
2. The SAP Workload Zone (Landscape):
   • The foundational regional infrastructure for an environment (DEV, QAS, PRD).
   • In a greenfield setup, this layer deploys the VNet, subnets, route tables, NSGs, Azure Files NFS / NetApp Files shares, and Workload Key Vault.
   • In a brownfield setup, this layer is modified to reference existing Azure Resource Manager (ARM) IDs.
3. The SAP System:
   • The actual SAP virtual machines: HANA Database servers, Central Services (ASCS/ERS), Primary Application Servers (PAS), and Additional Application Servers (AAS).
   • This layer consumes the state of the Workload Zone to position VMs in the correct subnets and attach storage.
4. Ansible Configuration:
   • Post-Terraform software installation: OS tuning, SAP software staging, SAPinst execution, and post-install validations.

The Workload Zone is the linchpin where the vendor's landing zone and SDAF converge.

---

3. The Discovery Phase: Auditing the Vendor Landing Zone

Before writing a single line of Terraform variable configuration, you must extract the exact Azure Resource Manager (ARM) identifiers from the vendor's deployment.

In our environment, the vendor created the following resources in subscription <your-subscription-id> (Region: indonesiacentral):

• Resource Group: rg-sap-network-dev
• Virtual Network: vnet-sap-workload-dev (CIDR: 10.x.0.0/24)
• Subnets & NSGs:
  • ApplicationSubnet (10.x.1.0/26) & nsg-sap-app-dev
  • DatabaseSubnet (10.x.2.0/26) & nsg-sap-db-dev
  • AppGWSubnet (10.x.0.0/26) & nsg-sap-admin-dev
  • WebSubnet (10.x.3.0/26) & nsg-sap-web-dev

We executed a discovery script (check_vms_for_sap.sh and Azure CLI queries) to capture the exact resource IDs:

# Capture VNet Resource ID
az network vnet show \
  --resource-group rg-sap-network-dev \
  --name vnet-sap-workload-dev \
  --query id -o tsv

# List Subnets and their attached NSGs
az network vnet subnet list \
  --resource-group rg-sap-network-dev \
  --vnet-name vnet-sap-workload-dev \
  --query "[].{Name:name, ID:id, NSG:networkSecurityGroup.id, Prefix:addressPrefix}" -o table

---

4. Crafting the Brownfield Workload Zone (.tfvars)

In a greenfield deployment, the Workload Zone .tfvars file uses network_address_space = "10.x.0.0/24" and tells SDAF to create subnets dynamically.

To execute a brownfield deployment, you must override this behavior by supplying explicit ARM IDs for the virtual network, each individual subnet, and their respective Network Security Groups.

Here is the exact configuration we implemented in WORKSPACES/LANDSCAPE/DEV-IDCE-SAP01-INFRASTRUCTURE/DEV-IDCE-SAP01-INFRASTRUCTURE.tfvars:

# ============================================================================
# WORKLOAD ZONE CONFIGURATION: BROWNFIELD AZURE LANDZONE ALIGNMENT
# Environment: DEV | Target Region: indonesiacentral
# ============================================================================

environment = "DEV"
location    = "indonesiacentral"

# ----------------------------------------------------------------------------
# 1. Existing VNet Binding (BROWNFIELD)
# ----------------------------------------------------------------------------
network_logical_name = "SAP01"
network_arm_id       = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev"

# ----------------------------------------------------------------------------
# 2. Existing Subnets and NSG Mappings
# ----------------------------------------------------------------------------

# Application Subnet (Target for PAS and AAS instances)
app_subnet_arm_id     = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/ApplicationSubnet"
app_subnet_name       = "ApplicationSubnet"
app_subnet_nsg_arm_id = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/networkSecurityGroups/nsg-sap-app-dev"

# Database Subnet (Target for SAP HANA Database)
db_subnet_arm_id      = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/DatabaseSubnet"
db_subnet_name        = "DatabaseSubnet"
db_subnet_nsg_arm_id  = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/networkSecurityGroups/nsg-sap-db-dev"

# Admin / Gateway Subnet
admin_subnet_arm_id     = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/AppGWSubnet"
admin_subnet_name       = "AppGWSubnet"
admin_subnet_nsg_arm_id = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/networkSecurityGroups/nsg-sap-admin-dev"

# Web Dispatcher Subnet (4-Tier Isolation)
web_subnet_arm_id     = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/WebSubnet"
web_subnet_name       = "WebSubnet"
web_subnet_nsg_arm_id = "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/networkSecurityGroups/nsg-sap-web-dev"

What Happens Behind the Scenes?

When network_arm_id and the specific subnet *_arm_id parameters are populated, SDAF's internal Terraform modules:

1. Disable azurerm_virtual_network creation: Skips the VNet creation block.
2. Disable azurerm_subnet creation: Uses Terraform data sources to inspect and bind to existing subnets.
3. Preserves existing NSGs: Avoids replacing vendor security rules with SDAF defaults.

---

5. Critical Brownfield Gotchas & How We Solved Them

Simply passing ARM IDs is not enough. During our deployment, we uncovered four critical architectural hurdles that are not covered in generic documentation.

Gotcha #1: Control Plane Peering & Route Reachability

SDAF's Deployer VM lives in its own management VNet (10.x.100.0/25). However, Ansible playbooks must communicate directly with SAP VMs over private IP addresses (such as 10.x.1.x for App and 10.x.2.x for HANA).

If the Deployer cannot establish an SSH tunnel to the target subnets, the entire automation pipeline halts.

Solution: Enable bi-directional VNet peering directly inside the Workload Zone configuration:

# Explicitly peer the Workload VNet with the SDAF Deployer VNet
peer_with_control_plane_vnet = true

This instructs SDAF to configure azurerm_virtual_network_peering between DEV-IDCE-DEP01-INFRASTRUCTURE and vnet-sap-workload-dev, ensuring seamless Ansible connectivity without requiring extra jumpboxes.

---

Gotcha #2: Key Vault & Storage Account 403 Lockout

By default, when SDAF provisions the Workload Key Vault and shared storage accounts, it creates network firewalls.

In early test runs, we encountered severe 403 Forbidden errors: the storage accounts were provisioned with publicNetworkAccess = "Disabled", and because private DNS zones were not yet fully propagated, the Deployer VM was locked out.

Solution: We configured service endpoint integration and explicitly permitted the vendor subnets in the Key Vault and Storage firewall ACLs:

# Enable KeyVault and Storage firewalls with automated subnet authorization
enable_firewall_for_keyvaults_and_storage = true
public_network_access_enabled             = true
use_service_endpoint                      = true

# Explicitly register vendor subnets into the Key Vault / Storage allow-list
subnets_to_add_to_firewall_for_keyvaults_and_storage = [
  "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/DatabaseSubnet",
  "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/ApplicationSubnet",
  "/subscriptions/<your-subscription-id>/resourceGroups/rg-sap-network-dev/providers/Microsoft.Network/virtualNetworks/vnet-sap-workload-dev/subnets/AppGWSubnet"
]

This configuration ensures:

• Deny-by-default for untrusted traffic.
• Automatic allow-listing of the Deployer's public IP during initial bootstrap.
• Authorized access from the pre-existing SAP application and database subnets.

---

Gotcha #3: Shared Storage Strategy – Azure NetApp Files (ANF) vs. Azure Files Premium (AFS)

The original vendor blueprint called for Azure NetApp Files (ANF). However, during our initial discovery (CRITICAL_FINDING_ANF_NOT_AVAILABLE.md), we discovered that ANF was not available in our specific subscription/region quota in indonesiacentral.

Rather than halting the project to wait for regional capacity delegations, we adjusted our SDAF Workload Zone configuration to leverage Azure Files Premium NFS 4.1:

# Select Azure Files Premium with NFS 4.1
NFS_provider = "AFS"

With this single parameter, SDAF automatically provisioned an enterprise-grade Premium Storage Account configured for NFS 4.1 and created the three required SAP shared filesystems:

• /sapmnt (128 GB)
• /usr/sap/trans (128 GB)
• /hana/shared (256 GB)

---

Gotcha #4: The Region Code Mapping Flaw (indonesiacentral -> IDCE)

SDAF uses standardized 4-character region codes (e.g., WEEU for West Europe, EUS2 for East US 2) to build resource names and remote state keys: <ENVIRONMENT>-<REGION>-<LOGICAL_NAME>-INFRASTRUCTURE

When deploying to indonesiacentral, the out-of-the-box SDAF scripts did not contain a mapping entry for this newer region. As a result, SDAF defaulted to the region code UNKN (Unknown).

This created critical operational hazards:

1. Key Vault names became DEVUNKNSAP01userXXX.
2. When multiple deployments were attempted, soft-deleted Key Vaults collided with new runs.
3. Resource names failed corporate tagging and naming compliance.

The Fix: We patched SDAF's core utility script (sap-automation/deploy/scripts/deploy_utils.sh at line 518) to officially map indonesiacentral to IDCE:

# Add Indonesia Central mapping
"indonesiacentral") export region_code="IDCE" ;;

We then standardized all workspace directory structures and state keys around IDCE:

• Workload Zone: DEV-IDCE-SAP01-INFRASTRUCTURE
• Deployer: DEV-IDCE-DEP01-INFRASTRUCTURE
• SAP System: DEV-IDCE-SAP01-D01

Note : this actually happened in Dec 2025, meanwhile now the Microsoft have registered region code INCE for indonesiacentral.

---

6. Binding the SAP System to the Brownfield Workload Zone

Once the Workload Zone has successfully run and registered the vendor's landing zone into its Terraform state file (DEV-IDCE-SAP01-INFRASTRUCTURE.terraform.tfstate), configuring the actual SAP S/4HANA system (DEV-IDCE-SAP01-D01.tfvars) becomes clean and declarative.

Notice how the system tier does not need to redefine subnets or IP addresses—it simply references the Workload Zone's state file:

# ============================================================================
# SAP SYSTEM CONFIGURATION (D01 POC)
# ============================================================================

environment          = "DEV"
location             = "indonesiacentral"
network_logical_name = "SAP01"
sid                  = "D01"
database_sid         = "HDB"
license_type         = "SLES_BYOS"

# ----------------------------------------------------------------------------
# State File Inheritance: Connects to Brownfield Workload Zone
# ----------------------------------------------------------------------------
deployer_tfstate_key  = "DEV-IDCE-DEP01-INFRASTRUCTURE.terraform.tfstate"
landscape_tfstate_key = "DEV-IDCE-SAP01-INFRASTRUCTURE.terraform.tfstate"

# ----------------------------------------------------------------------------
# Database Tier: SAP HANA 2.0 SPS07 on Standard_E64s_v5
# ----------------------------------------------------------------------------
database_platform          = "HANA"
database_vm_sku            = "Standard_E64s_v5" # 64 vCPU, 432 GB RAM
database_server_count      = 1
database_high_availability = false
database_vm_zones          = [1]

database_vm_image = {
  os_type         = "linux"
  source_image_id = ""
  publisher       = "SUSE"
  offer           = "sles-sap-15-sp5"
  sku             = "gen2"
  version         = "latest"
  type            = "marketplace"
}

# HANA TDI Storage Disks
database_vm_storage_sku       = "Premium_LRS"
database_vm_data_disk_count   = 4
database_vm_data_disk_size_gb = 256 # /hana/data
database_vm_log_disk_count    = 3
database_vm_log_disk_size_gb  = 128 # /hana/log
database_vm_backup_disk_count = 1
database_vm_backup_disk_size_gb = 512 # /hana/backup

# ----------------------------------------------------------------------------
# Application Tier: Central Services (ASCS) & Primary App Server (PAS)
# ----------------------------------------------------------------------------
enable_app_tier_deployment = true
scs_server_count           = 1
scs_server_sku             = "Standard_D8ads_v5" # 8 vCPU, 32 GB RAM
scs_instance_number        = "00"
ers_instance_number        = "01"
application_server_count   = 0                   # Co-locating PAS on SCS VM for DEV
pas_instance_number        = "02"

# Mount NFS shares automatically from the Workload Zone
NFS_provider = "AFS"

---

7. Execution Workflow & Verification

To execute the deployment, we utilize SDAF’s deployment scripts from the Deployer VM:

# 1. Execute Workload Zone Deployment
cd /home/<automation-user>/Azure_SAP_Automated_Deployment/WORKSPACES/LANDSCAPE/DEV-IDCE-SAP01-INFRASTRUCTURE

/home/<automation-user>/Azure_SAP_Automated_Deployment/sap-automation/deploy/scripts/install_workloadzone.sh \
  --parameterfile DEV-IDCE-SAP01-INFRASTRUCTURE.tfvars \
  --deployer_environment DEV \
  --subscription <your-subscription-id> \
  --auto-approve

Verification Checklist:

When the command finishes, check the Azure Portal and Terraform outputs:

1. Zero New VNets: Confirm that no new Virtual Network was created.
2. Subnet Integrity: Inspect vnet-sap-workload-dev to ensure that ApplicationSubnet and DatabaseSubnet retained their original CIDRs and NSG attachments.
3. VNet Peering: Verify that vnet-sap-workload-dev has a status of Connected in peering to DEV-IDCE-DEP01-INFRASTRUCTURE.
4. Storage & NFS: Check that the Azure Files Premium storage account was provisioned and allows access from the vendor subnets.
5. Key Vault Secrets: Verify that SSH private keys and automation credentials are saved in the Workload Key Vault (DEVIDCESAP01user...).

---

8. Summary & Key Takeaways

Integrating an enterprise landing zone with Microsoft SDAF does not require compromising on cloud architecture or manual workarounds:

Challenge	Anti-Pattern	Brownfield SDAF Best Practice
Pre-existing VNet	Let SDAF deploy a new VNet and double-peer	Bind network_arm_id and pass existing Subnet / NSG ARM IDs
Deployer Access	Expose SAP VMs with public IPs	Enable peer_with_control_plane_vnet = true
Storage Lockout	Disable storage firewalls completely	Set enable_firewall... = true and populate subnets_to_add_to_firewall...
Regional Codes	Accept default UNKN naming	Patch deploy_utils.sh to map your region code cleanly (IDCE)
Shared Filesystems	Stall deployment waiting for ANF quota	Pivot cleanly to Azure Files Premium (NFS_provider = "AFS")

By treating the vendor-deployed landing zone as an authoritative foundation and declaring its ARM IDs inside SDAF's Workload Zone configuration, we achieved full automation without disrupting enterprise cloud standards.

---

Coming Up in Part 2:

Bootstrapping the SDAF Control Plane & Deployer VM — How to configure the Ubuntu Deployer VM, establish Managed Identity (MSI) and SPN authentication without privilege sprawl, and configure secure bastion tunnels for remote SAP GUI and CLI access.