In my recent ‘Getting started with Oracle Cloud VMware Solution (OVCS)’ post; Getting Started With Oracle Cloud VMware Solution (OCVS) – Deploying The SDDC With HCX we deployed ourselves a Software-Defined Data Center (SDDC) along with VMware HCX into Oracle Cloud.
Posts in this series:
In this post, I’m going to review the overall networking configuration, including NSX-T.
First, let’s take a look at how the ESXi Hosts are connected to the Oracle Cloud infrastructure.
A VCN Subnet consists of a contiguous range of IPv4 addresses that do not overlap with other subnets in the VCN. So if you were to deploy a second SDDC using the same VCN, you would have to specify a different CIDR block. If you try and use the same CIDR block that is used by another SDDC you’ll receive an error.
Subnet-tshirts-sddc | This Subnet is where the ESXi Host Management vNICs (vmk0) reside. It is also used, as the name suggests, during the provisioning of the ESXi Hosts |
A VLAN is an object within a VCN. VLANs are used to partition the VCN into Layer2 broadcast domains. Each VLAN has a Route-Table associated with it. The Route Table is responsible for the traffic forwarding to a specific destination. In addition to the Route Table, each VLAN has a Network Security Group or Security Rules (Firewall rules) associated with it. These Network Security Groups function in the same way as a Firewall, allowing and denying traffic in and out of the VLAN.
NOTE: When manually creating a new VLAN or Subnet, all traffic is denied by default. Rules will need to be added to allow traffic to flow.
Each VLAN within the VCN is automatically assigned a VLAN ID. These VLAN IDs are only local to the VCN, so there might be cases where you deploy an SDDC in another AD or Region and the same VLAN ID is used. Even though they might share the same VLAN ID, it’s important to understand that they are not the same VLAN.
The following VLANs are connected to each ESXi Host and are used throughout the deployment.
VLAN-tshirts-sddc-vSphere | This VLAN is regularly called ‘Management Network’ in on-premises environments. This is where vCenter, NSX-T management, HCX management, and NSX-T Edges live (Not ESXi host vmk0) |
VLAN-tshirts-sddc-vSAN | This VLAN dedicated to vSAN traffic |
VLAN-tshirts-sddc-vMotion | This VLAN dedicated to vMotion traffic |
VLAN-tshirts-sddc-NSX VTEP | This VLAN is used the ESXi Host TEPs (Tunnel Endpoints) where NSX-T overlay traffic (Geneve encapsulated) will flow East-West between the ESXi Hosts |
VLAN-tshirts-sddc-NSX Edge VTEP | This VLAN is used for the NSX-T Edge TEPs sending Geneve encapsulated traffic between the NSX-T Edges and the ESXi Hosts |
VLAN-tshirts-sddc-NSX Edge Uplink 1 | This VLAN is used for North-South communication between the SDDC, the native Oracle Cloud services and the internet |
VLAN-tshirts-sddc-NSX Edge Uplink 2 | This VLAN is not currently used at this time |
VLAN-tshirts-sddc-HCX | Used for VMware HCX traffic |
You can drill down into each VLAN by clicking on the VLAN name.
The Route table for the vSphere VLAN specifies the NAT Gateway (tshirts-ngw) as the Default Gateway for all traffic on this VLAN. This allows the virtual machines on this VLAN to access the internet if allowed in the Security Rules.
There are many Security Rules added to the Network Security Group (Firewall). These rules are automatically configured to allow all of the components of the SDDC and HCX to communicate between each other and out to the internet. Feel free to review each of the individual rules to understand what traffic flows where. It might help you sleep at night.
Let’s take a look a the virtual networking configuration inside vCenter.
Once you are logged into vCenter:
Virtual Distributed Switch (vDS) portgroups
vds01-vSphere | vSphere VLAN |
vds01-vSAN | vSAN VLAN (ESXi Host vmk2 lives here) |
vds01-vMotion | vMotion VLAN (ESXi Host vmk1 lives here) |
vds01-HCX | HCX VLAN |
Management Network | This network is used ONLY by Oracle during their deployment process and is routed to allow communication with the vSphere VLAN (ESXi Host vmk0 lives here) |
NSX-T Distributed Virtual Switch (N-VDS) portgroups
edge-ns | NSX Edge Uplink 1 VLAN |
edge-transport | NSX Edge VTEP VLAN |
workload | This is the NSX-T segment for our virtual machines that we specified during the configuration of the SDDC |
Standard vSphere portgroups
VM Network | This is deployed as part of the NSX-T deployment and is not used |
Now let’s take a look at the ESXi Host connectivity from within vCenter. Even though we have already taken a look at the ESXi Host connectivity within the Oracle Cloud interface earlier in this post, it might help clarify things by looking at it within vCenter, where we are likely more familiar.
vmnic0 | Assigned to the Distributed Virtual Switch (DSwitch) |
vmnic1 | Assigned to the NSX-T Distributed Virtual Switch (oci-w01-vds01) |
vmk0 | Added to the Management Network vDS and configured to be used for ESXi management traffic |
vmk1 | Added to the vds01-vMotion vDS and configured to be used for ESXi vMotion traffic |
vmk2 | Added to the vds01-vSAN vDS and configured to be used for ESXi vSAN traffic |
NSX-T is automatically deployed and configured for us when we deploy the SDDC. However, on-going management and configuration of NSX-T will be required if you need to add additional workload segments into the environment in the future.
NSX-T Segments
NSX-Edge-VCN-Segment | This segment is used for connectivity between the two Edge nodes |
workload | This segment is used for the workload network (192.168.1.0/24) that we specified during the deployment of the SDDC (You may not have this segment if you did not choose to deploy a workload network) |
NSX-T Logical Routers
There are two logical routers, Tier-0, and Tier-1.
Tier-0 | Connected to the NSX Edge VTEP VLAN which allows Geneve encapsulated traffic to flow between the Edge nodes and ESXi Host, and the NSX Edge Uplink 1 VLAN which will allow traffic in and out of the environment |
Tier-1 | Connected to the workload segment where our virtual machines will live. As we add additional segments, these will be connected to the Tier-1 |
NSX-T Transport Zones
There are three Transport Zones configured.
Overlay-TZ | An Overlay Transport Zone which includes all ESXi Hosts and both NSX-T Edges and is used by the workload segments. |
VLAN-TZ | A VLAN Transport Zone which includes all ESXi Hosts and both NSX-T Edges and is used by the edge-ns and edge-transport logical switches |
VLAN-TZ-2 | A VLAN Transport Zone which only includes both NSX-T Edges and is used for Edge-to-Edge Geneve encapsulated traffic over the NSX-Edge-VCN-Segment segment |
Now that we have taken a look at the overall networking configuration of the SDDC after initial deployment, hopefully, you have a better understanding of how everything is connected together. The following image illustrates how all of the components that make up the OCVS SDDC environment are connected. In future posts, we are likely to start to add additional networking configurations based on our use cases.
Another VMworld is upon us!!! Sadly, it's only virtual again this year. However, that does…
As part of my recent move to Google, I'm working on quickly getting up to…
I am delighted to announce the next chapter in my career. Today is my first…
In episode thirteen of The VCDX Podcast, I am joined by two special guests who…
In my recent ‘Getting started with Oracle Cloud VMware Solution (OVCS)’ post; Getting Started With…
In my recent ‘Getting started with Oracle Cloud VMware Solution (OVCS)’ post; Getting Started With…