Sometimes multiple external IP address on an Azure Virtual Machine (VM) are needed for a deployed application or script. We will setup a multi NIC VM with one public IP per interface and route some traffic through different interfaces.

Table of Contents

1. Introduction
2. Arm Template
   1. Parameters
   2. Variables
   3. Resource iteration
   4. Property iteration
3. Use multiple NICs with Ubuntu

Introduction

Azure has support for multiple network interface cards (NICs) as well as multiple public IP address resources for quite some time now. The number of NICs that can be attached to a VM depends on the SKU aka size aka badge of the VM. As we will deploy a template with 4 NICs, we will use a DS3v2 from the DSv2-series, as this is one of the sizes, where you can attach 4 NICs to a VM with 4 cores and have enough compute power. The limitations regarding the number of NICs are listed on the Microsoft Docs for the corresponding SKU, here are the ones for the DSv2-series:

Arm Template

We will create the following resources using our ARM template:

• A new VNET (you can of course use your own existing one, just see the ARM quickstart repo for references)
• n public IP addresses
• n network interface
• A virtual machine that uses all the above resources

Parameters

Let’s start with the basics. We need some parameters and some variables. For the sake of simplicity, we will not parameterize things like VNET reference or subnet prefix.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

{
  "parameters": {
    "vmName": {
      "type": "string",
      "defaultValue": "proxyVm",
      "metadata": {
        "description": "Name of the VM"
      }
    },
    "vmSku": {
      "type": "string",
      "defaultValue": "Standard_DS3_v2",
      "metadata": {
        "description": "VM SKU. Be aware of NIC count limitations."
      }
    },
    "adminUsername": {
      "type": "string",
      "metadata": {
        "description": "VM admin username, certain standard values are not allowed."
      }
    },
    "adminPassword": {
      "type": "securestring",
      "metadata": {
        "description": "Password for VM admin. Please chose a secure random password with at least 16 characters."
      }
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    },
    "nicCount": {
      "type": "int",
      "defaultValue": 4,
      "minValue": 1,
      "maxValue": 20,
      "metadata": {
        "description": "Numbers of NICs and PIPs to be deployed."
      }
    }
  },

Variables

The variables are quite simplistic. We create the name of the VNET from the vmName parameter and create the variables for the most current version of Ubuntu 19.04 from the Azure marketplace.

1
2
3
4
5
6
7
8
9
10
11

{
  "variables": {
    "subnetName": "subnet01",
    "virtualNetworkName": "[concat(parameters('vmName'), '-vnet')]",
    "publisher": "Canonical",
    "offer": "UbuntuServer",
    "sku": "19.04",
    "version": "19.04.201907241",
    "vnetPrefix": "10.0.0.0/24",
    "subnetPrefix": "10.0.0.0/24"
  },

Resource iteration

As we want to be able to change how many NICs are deployed, we will use the copy object of ARM templates. This object helps us to deploy multiple instances of a specific resource without duplicating the resource block. A resource, using the copy property looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

  {
    "apiVersion": "2015-06-15",
    "type": "Microsoft.Network/publicIPAddresses",
    "name": "[concat(parameters('vmName'), copyIndex(), '-pip')]",
    "location": "[parameters('location')]",
    "dependsOn": [
      "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
    ],
    "copy": {
      "name": "proxyVmPipCopy",
      "count": "[parameters('nicCount')]"
    },
    "properties": {
      "publicIPAllocationMethod": "Static"
    }
  },
...

Here, we use the copyIndex() function to access the current index of enumeration to generate the name of the public IP address.
We will use resource iteration with the copy object for public IP addresses and network interfaces.

In the below example, we use the copyIndex() function to generate the ipConfigurations property for the NICs.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

"properties": {
  "ipConfigurations": [
    {
      "name": "ipconfig1",
      "properties": {
        "primary": "[if(greater(copyIndex(), 0), 'false', 'true')]",
        "subnet": {
          "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]"
        },
        "privateIPAllocationMethod": "Dynamic",
        "publicIpAddress": {
          "id": "[resourceId('Microsoft.Network/publicIpAddresses', concat(parameters('vmName'), copyIndex(), '-pip'))]"
        }
      }
    }
  ]
}

With multiple NICs attached to a VM, there has to be one network interface that is marked as primary. To take this into consideration, we compare the current copyIndex with 0 and if the index is greater than 0, the value is set to true. In other words, the first NIC is the primary one, all following will be set as secondary.

Property iteration

The copy object becomes really useful, when used at a property level. So if we want to have multiple data disks deployed with a VM or like in our example multiple NICs attached, we can also use the copyIndex() function.
There is just one very important difference, we have to create a new “nested” copy object and refence to it by using the name as a parameter in the copyIndex() function.

1
2
3
4
5
6
7
8
9
10
11
12

"networkProfile": {
  "copy": [{
    "name": "networkInterfaces",
    "count": "[parameters('nicCount')]",
    "input": {
      "properties": {
        "primary": "[if(greater(copyIndex('networkInterfaces'), 0), 'false', 'true')]"
      },
      "id": "[resourceId('Microsoft.Network/networkInterfaces', concat(parameters('vmName'), copyIndex('networkInterfaces'), '-nic'))]"
    }
  }]
}

In line 2 we create a new copy object, but this time as an array object. The parameters name and count are the same as before and in addition the input property contains the block of ARM code that needs to be iterated.
As mentioned before, we have to use the name of the copy object, when getting the current iteration. This happens with copyIndex('networkInterfaces') in line 7 and 9.

Use multiple NICs with Ubuntu

If we want to use the multiple IP addresses for internet access, we have to adjust the routing a little bit. We basically set multiple standard gateways with different priorities and let the application that uses the network decide which interface to use. By default the one with the lowest priority will be used. I have configured the routes as follows:

1
2
3

ip route add default via 10.0.0.1 dev eth1 metric 101
ip route add default via 10.0.0.1 dev eth2 metric 102
ip route add default via 10.0.0.1 dev eth3 metric 103

The routes should look something like this afterwards:

1
2
3
4
5
6
7
8
9
10
11

root@proxyVm:/home/proxyVmadm# ip route show
default via 10.0.0.1 dev eth0 proto dhcp metric 100
default via 10.0.0.1 dev eth1 metric 101
default via 10.0.0.1 dev eth2 metric 102
default via 10.0.0.1 dev eth3 metric 103
10.0.0.0/24 dev eth0 proto kernel scope link src 10.0.0.6
10.0.0.0/24 dev eth3 proto kernel scope link src 10.0.0.4
10.0.0.0/24 dev eth2 proto kernel scope link src 10.0.0.7
10.0.0.0/24 dev eth1 proto kernel scope link src 10.0.0.5
168.63.129.16 via 10.0.0.1 dev eth0 proto dhcp metric 100
169.254.169.254 via 10.0.0.1 dev eth0 proto dhcp metric 100

Now let’s see, if all external interfaces are operational by using curl and ipify.org to determine our current external ip address:

You can find the full ARM template example on Github (truncated preview):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "vmName": {
      "type": "string",
      "defaultValue": "proxyVm",
      "metadata": {
        "description": "Name of the VM"
      }
    },
    "vmSku": {
      "type": "string",
      "defaultValue": "Standard_DS3_v2",
      "metadata": {

I hope this post helped explaining the possibilities of the copy object and how to use multiple external IPs in Azure. Like always, share the post if you like it and feel free to update and use the snippets in you own scripts 😉