ingress-nginx

ingress-nginx Ingress controller for Kubernetes using NGINX as a reverse proxy and load balancer

To use, add ingressClassName: nginx spec field or the kubernetes.io/ingress.class: nginx annotation to your Ingress resources.

This chart bootstraps an ingress-nginx deployment on a Kubernetes cluster using the Helm package manager.

Requirements

Kubernetes: >=1.21.0-0

Get Repo Info

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Install Chart

Important: only helm3 is supported

helm install [RELEASE_NAME] ingress-nginx/ingress-nginx

The command deploys ingress-nginx on the Kubernetes cluster in the default configuration.

See configuration below.

See helm install for command documentation.

Uninstall Chart

helm uninstall [RELEASE_NAME]

This removes all the Kubernetes components associated with the chart and deletes the release.

See helm uninstall for command documentation.

Upgrading Chart

helm upgrade [RELEASE_NAME] [CHART] --install

See helm upgrade for command documentation.

Migrating from stable/nginx-ingress

There are two main ways to migrate a release from stable/nginx-ingress to ingress-nginx/ingress-nginx chart:

1. For Nginx Ingress controllers used for non-critical services, the easiest method is to uninstall the old release and install the new one
2. For critical services in production that require zero-downtime, you will want to:
   1. Install a second Ingress controller
   2. Redirect your DNS traffic from the old controller to the new controller
   3. Log traffic from both controllers during this changeover
   4. Uninstall the old controller once traffic has fully drained from it

Note that there are some different and upgraded configurations between the two charts, described by Rimas Mocevicius from JFrog in the "Upgrading to ingress-nginx Helm chart" section of Migrating from Helm chart nginx-ingress to ingress-nginx. As the ingress-nginx/ingress-nginx chart continues to update, you will want to check current differences by running helm configuration commands on both charts.

Configuration

See Customizing the Chart Before Installing. To see all configurable options with detailed comments, visit the chart's values.yaml, or run these configuration commands:

helm show values ingress-nginx/ingress-nginx

PodDisruptionBudget

Note that the PodDisruptionBudget resource will only be defined if the replicaCount is greater than one, else it would make it impossible to evacuate a node. See gh issue #7127 for more info.

Prometheus Metrics

The Ingress-Nginx Controller can export Prometheus metrics, by setting controller.metrics.enabled to true.

You can add Prometheus annotations to the metrics service using controller.metrics.service.annotations. Alternatively, if you use the Prometheus Operator, you can enable ServiceMonitor creation using controller.metrics.serviceMonitor.enabled. And set controller.metrics.serviceMonitor.additionalLabels.release="prometheus". "release=prometheus" should match the label configured in the prometheus servicemonitor ( see kubectl get servicemonitor prometheus-kube-prom-prometheus -oyaml -n prometheus)

ingress-nginx nginx_status page/stats server

Previous versions of this chart had a controller.stats.* configuration block, which is now obsolete due to the following changes in Ingress-Nginx Controller:

• In 0.16.1, the vts (virtual host traffic status) dashboard was removed
• In 0.23.0, the status page at port 18080 is now a unix socket webserver only available at localhost. You can use curl --unix-socket /tmp/nginx-status-server.sock http://localhost/nginx_status inside the controller container to access it locally, or use the snippet from nginx-ingress changelog to re-enable the http server

ExternalDNS Service Configuration

Add an ExternalDNS annotation to the LoadBalancer service:

controller:
  service:
    annotations:
      external-dns.alpha.kubernetes.io/hostname: kubernetes-example.com.

AWS L7 ELB with SSL Termination

Annotate the controller as shown in the nginx-ingress l7 patch:

controller:
  service:
    targetPorts:
      http: http
      https: http
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:XX-XXXX-X:XXXXXXXXX:certificate/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX
      service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
      service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
      service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: '3600'

Additional Internal Load Balancer

This setup is useful when you need both external and internal load balancers but don't want to have multiple ingress controllers and multiple ingress objects per application.

By default, the ingress object will point to the external load balancer address, but if correctly configured, you can make use of the internal one if the URL you are looking up resolves to the internal load balancer's URL.

You'll need to set both the following values:

controller.service.internal.enabled controller.service.internal.annotations

If one of them is missing the internal load balancer will not be deployed. Example you may have controller.service.internal.enabled=true but no annotations set, in this case no action will be taken.

controller.service.internal.annotations varies with the cloud service you're using.

Example for AWS:

controller:
  service:
    internal:
      enabled: true
      annotations:
        # Create internal NLB
        service.beta.kubernetes.io/aws-load-balancer-scheme: "internal"
        # Create internal ELB(Deprecated)
        # service.beta.kubernetes.io/aws-load-balancer-internal: "true"
        # Any other annotation can be declared here.

Example for GCE:

controller:
  service:
    internal:
      enabled: true
      annotations:
        # Create internal LB. More information: https://cloud.google.com/kubernetes-engine/docs/how-to/internal-load-balancing
        # For GKE versions 1.17 and later
        networking.gke.io/load-balancer-type: "Internal"
        # For earlier versions
        # cloud.google.com/load-balancer-type: "Internal"

        # Any other annotation can be declared here.

Example for Azure:

controller:
  service:
      annotations:
        # Create internal LB
        service.beta.kubernetes.io/azure-load-balancer-internal: "true"
        # Any other annotation can be declared here.

Example for Oracle Cloud Infrastructure:

controller:
  service:
      annotations:
        # Create internal LB
        service.beta.kubernetes.io/oci-load-balancer-internal: "true"
        # Any other annotation can be declared here.

The load balancer annotations of more cloud service providers can be found: Internal load balancer.

An use case for this scenario is having a split-view DNS setup where the public zone CNAME records point to the external balancer URL while the private zone CNAME records point to the internal balancer URL. This way, you only need one ingress kubernetes object.

Optionally you can set controller.service.loadBalancerIP if you need a static IP for the resulting LoadBalancer.

Ingress Admission Webhooks

With nginx-ingress-controller version 0.25+, the Ingress-Nginx Controller pod exposes an endpoint that will integrate with the validatingwebhookconfiguration Kubernetes feature to prevent bad ingress from being added to the cluster. This feature is enabled by default since 0.31.0.

With nginx-ingress-controller in 0.25.* work only with kubernetes 1.14+, 0.26 fix this issue

How the Chart Configures the Hooks

A validating and configuration requires the endpoint to which the request is sent to use TLS. It is possible to set up custom certificates to do this, but in most cases, a self-signed certificate is enough. The setup of this component requires some more complex orchestration when using helm. The steps are created to be idempotent and to allow turning the feature on and off without running into helm quirks.

1. A pre-install hook provisions a certificate into the same namespace using a format compatible with provisioning using end user certificates. If the certificate already exists, the hook exits.
2. The Ingress-Nginx Controller pod is configured to use a TLS proxy container, which will load that certificate.
3. Validating and Mutating webhook configurations are created in the cluster.
4. A post-install hook reads the CA from the secret created by step 1 and patches the Validating and Mutating webhook configurations. This process will allow a custom CA provisioned by some other process to also be patched into the webhook configurations. The chosen failure policy is also patched into the webhook configurations

Alternatives

It should be possible to use cert-manager/cert-manager if a more complete solution is required.

You can enable automatic self-signed TLS certificate provisioning via cert-manager by setting the controller.admissionWebhooks.certManager.enabled value to true.

Please ensure that cert-manager is correctly installed and configured.

Helm Error When Upgrading: spec.clusterIP: Invalid value: ""

If you are upgrading this chart from a version between 0.31.0 and 1.2.2 then you may get an error like this:

Error: UPGRADE FAILED: Service "?????-controller" is invalid: spec.clusterIP: Invalid value: "": field is immutable

Detail of how and why are in this issue but to resolve this you can set xxxx.service.omitClusterIP to true where xxxx is the service referenced in the error.

As of version 1.26.0 of this chart, by simply not providing any clusterIP value, invalid: spec.clusterIP: Invalid value: "": field is immutable will no longer occur since clusterIP: "" will not be rendered.

Values