Identifying different applications running in the same Kubernetes cluster at an external firewall can be challenging. By default, all services running on a node are SNATed to the node’s IP address.

With Antrea Egress we can make use of dedicated egress IPs for different Pods in a Cluster. If you just want to get to the quickstart, click here.

What is Antrea?

Antrea is a Kubernetes CNI which is currently a CNCF Sandbox Project. It is heavily used in Production by VMware and its Customers as it is the default CNI for VMware Tanzu.

It leverages Open vSwitch as dataplane and thus is pretty performant.

How does Antrea Egress work

Instead of applying an SNAT to outgoing traffic to the Nodes IP Antrea applies a SNAT Policy to your custom Egress-IP.

Quickstart

When already using Antrea as CNI it’s quite straight forward to make use of Antrea Egress:

1. For Antrea versions older than 1.0, you need to upgrade. For versions older than 1.6, you need to enable a FeatureGate. For newer versions, proceed to step 2.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: antrea-config
     namespace: kube-system
   data:
     antrea-agent.conf: |
       featureGates:
         Egress: true    
     antrea-controller.conf: |
       featureGates:
         Egress: true    
   

2. Create an Egress Policy:

   apiVersion: crd.antrea.io/v1beta1
   kind: Egress
   metadata:
     name: example-egress
   spec:
     appliedTo:
       namespaceSelector:
         matchLabels:
           env: custom-egress-ip
         podSelector:
           matchLabels:
             role: custom-egress-ip
     egressIP: 10.10.30.30
   

   With this policy, all pods labeled role: custom-egress-ip in namespaces labeled env: custom-egress-ip will use 10.10.30.30 as their egress IP when connecting to external IPs.