Configure L4 and L7 Load Balancers

This page describes common setups for exposing AtScale with L4 and L7 load balancers, and provides configuration examples for Amazon Web Services, Google Cloud, and Microsoft Azure. When setting up a load balancer, you should use a configuration that matches your network policy, DNS strategy, public or private access requirements, and WAF or Google Cloud Armor needs.

Note

AtScale deployments expose HTTP traffic and TCP traffic for JDBC/PgWire clients on port 15432.

Configure your domains

First, configure the domains to use for HTTP and TCP traffic. In your values.yaml or values override file, set the following:

• ingressDomain: The domain used for AtScale HTTP traffic.
• ingressTCPDomain: Optional. The domain used for JDBC/PgWire client connection URLs. Only set this if you require a different domain for these.

Note

If ingressTCPDomain is not set, AtScale uses ingressDomain for TCP client URLs.

For example:

global:
  # Used for AtScale HTTP traffic.
  ingressDomain: "atscale.example.com"
  # Optional. Used for JDBC/PgWire client connection URLs.
  ingressTCPDomain: "tcp.example.com"

Setting ingressTCPDomain only changes the host generated for JDBC/PgWire clients; it does not create a second load balancer, DNS record, or firewall rule. You should set ingressTCPDomain in the following cases:

• HTTP and TCP traffic use different load balancers.
• JDBC clients connect from a different network path.
• Internal users and external BI tools require separate DNS names.

Configure certificates

After configuring your HTTP and TCP domains, you must configure TLS in your values file. AtScale recommends using a certificate issued by a well-known certificate authority. Managed certificates are not supported.

For more information, see Installing AtScale.

Choose a load balancer setup

Next, you must determine the load balancer configuration that best suits your organization's needs. AtScale supports multiple topologies:

• L4 for HTTP and TCP. A network load balancer can expose both. This is usually the simplest setup.
• Ingress or L7 for HTTP, and L4 for TCP. Use this configuration when the HTTP path needs WAF or Google Cloud Armor.
• Internal and public L4 endpoints, depending on the needs of users and external BI tools.

AtScale Service baseline

For L4 Service-based routing, you can expose HTTP and TCP through the AtScale Service. To do so, add the following to your values file:

atscale-proxy:
  service:
    type: LoadBalancer

This generates a load balancer in your cloud automatically, taking the default settings from AtScale's install location. For more information, see Configuring the AtScale Proxy Service.

Load balancer configuration examples

The following sections provide example load balancer configurations for common cloud providers.

Note

The examples below use common cloud provider annotations. Always confirm the supported annotations for your Kubernetes version and ingress controller.

Amazon Web Services

The following sections provide examples of common AWS load balancer implementations.

L4 network load balancer

To use an internal NLB to provide private access to users connecting to AtScale over VPN, add the following to your values file:

global:
  ingressDomain: "atscale.internal.example.com"

atscale-proxy:
  service:
    type: LoadBalancer
    annotations:
      ## Creates an internal AWS Network Load Balancer.
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#lb-scheme
      service.beta.kubernetes.io/aws-load-balancer-scheme: internal
      
      ## Set this if you are using an internal load balancer
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/service/annotations/#lb-internal
      service.beta.kubernetes.io/aws-load-balancer-internal: "true"
      
      ## Sends traffic to Kubernetes nodes. Use the target type supported by your cluster.
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#nlb-target-type
      service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip 
      
      ## (Optional) Set this if you have more than one node in multiple zones but a single copy of atscale-proxy
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#load-balancer-attributes
      service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true

To make a public NLB for when your users need public access, use an internet-facing annotation:

global:
  ingressDomain: "atscale.example.com"

atscale-proxy:
  service:
    type: LoadBalancer
    annotations:
      ## Creates a public AWS Network Load Balancer.
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#lb-scheme
      service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
      
      ## Sends traffic to Kubernetes Pod. Use the target type supported by your cluster.
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#nlb-target-type
      service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
      
      ## (Optional) Set this if you have more than one zone nodes but a single copy of atscale-proxy
      ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#load-balancer-attributes
      service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
    loadBalancerSourceRanges:
      - "<whitelist-cidr-block>"

Notes

• The specific AWS annotations you should use depends on whether your cluster uses the in-tree cloud provider, AWS Load Balancer Controller, or EKS Auto Mode. For details, see Annotations in the AWS Load Balancer Controller Service documentation.
• Before setting service.beta.kubernetes.io/aws-load-balancer-nlb-target-type to ip, ensure the Amazon VPC CNI plugin is enabled on your EKS cluster. If this plugin is not enabled, set this key to instance.

Once the service has loaded and you get an External IP, you can generate a CNAME record in your DNS provider for your ingressDomain.

L7 for HTTP and L4 for TCP

Use this setup when HTTP needs ALB features such as AWS WAF, path routing, or centralized ingress management, and JDBC/PgWire needs a TCP endpoint.

1. Configure the certificate through the AtScale values file. If AWS WAFv2 is used, create the web ACL first and reference its ARN in the ALB annotations.

2. Configure Ingress for the HTTP path. Add the following to your values file:

   global:
     ingressDomain: "atscale.example.com"
     ingressTCPDomain: "tcp.atscale.example.com"
     atscale:
       tls:
         existingSecret: "atscale-tls"
   
   atscale-proxy:
     ingress:
       enabled: true
       ingressClassName: "alb"
       annotations:
         ## Creates a public l7 loadbalancer
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/ingress/annotations/#scheme
         alb.ingress.kubernetes.io/scheme: internet-facing
         
         ## Choose the target type. We suggest you forward directly to the POD IP. VPC CNI plugin required.
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/ingress/annotations/#target-type
         alb.ingress.kubernetes.io/target-type: ip
         
         ## Force the load-balancer to redirect traffic to HTTPS
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/ingress/annotations/#ssl-redirect
         alb.ingress.kubernetes.io/ssl-redirect: "443"
         
         ## Create a listener to forward traffic to.
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/annotations/#listen-ports
         alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
         
         ## Extends the timeout of a long running client requests
         ## ref: https://docs.aws.amazon.com/elasticloadbalancing/latest/APIReference/API_LoadBalancerAttribute.html
         alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=3600
         
         ## Point the listener to the health check port
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/latest/guide/ingress/annotations/#healthcheck-port
         alb.ingress.kubernetes.io/healthcheck-port: "8888"
         
         ## (Optional) Setup a waf acl policy
         alb.ingress.kubernetes.io/wafv2-acl-arn: "<wafv2-web-acl-arn>"

3. Configure the Service used by the TCP path:

   atscale-proxy:
     service:
       type: LoadBalancer
       annotations:
         ## Creates a public AWS Network Load Balancer.
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#lb-scheme
         service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
         
         ## Sends traffic to Kubernetes Pod. Use the target type supported by your cluster.
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#nlb-target-type
         service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
         
         ## (Optional) Set this if you have more than one zone nodes but a single copy of atscale-proxy
         ## ref: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v3.4/guide/service/annotations/#load-balancer-attributes
         service.beta.kubernetes.io/aws-load-balancer-attributes: load_balancing.cross_zone.enabled=true
       loadBalancerSourceRanges:
         - "<whitelisted-cidr-block>"
   Note

   Refer to the AWS Load Balancer Controller documentation for information on ALB, WAF, and Network LoadBalancer annotations.

4. Create a DNS record in your DNS provider with the ingressDomain pointing to your ingress address, and the ingressTCPDomain pointing to your service External IP address.

Google Cloud

The following examples show common Google Cloud load balancer implementations.

L4 LoadBalancer Service

To use an internal GKE L4 load balancer for private access, add the following to your values file:

global:
  ingressDomain: "atscale.internal.example.com"

atscale-proxy:
  service:
    type: LoadBalancer
    ## @param annotations; Setup internal loadbalancer annotations
    ## ref: https://docs.cloud.google.com/kubernetes-engine/docs/concepts/service-load-balancer-parameters#service_parameters
    annotations:
      ## Create an internal loadbalancer
      networking.gke.io/load-balancer-type: "Internal"
      ## (Optional) Use this only when a reserved static IP is required.
      networking.gke.io/load-balancer-ip-addresses: "<reserved-ip-resource-name>"

Note

networking.gke.io/load-balancer-ip-addresses expects the reserved address resource name, not the IP address itself. For internal LoadBalancer Services, GKE subsetting must be enabled.

To use a public L4 load balancer for when external JDBC/PgWire clients need TCP access, add the following to your values file:

global:
  ingressDomain: "atscale.example.com"

atscale-proxy:
  service:
    type: LoadBalancer
    loadBalancerClass: "networking.gke.io/l4-regional-external"
    annotations:
      ## (Optional) Use this only when a reserved static IP is required.
      networking.gke.io/load-balancer-ip-addresses: "<reserved-ip-resource-name>"
    ## (Optional) Whitelist IPs which can connect to AtScale
    loadBalancerSourceRanges:
      - "<whitelisted-cidr-block>"

For more information, see LoadBalancer Service parameters and Create an internal load balancer in the Google GKE documentation.

GKE Ingress (L7) with Cloud Armor for HTTP, L4 for TCP

Use this setup when HTTP traffic needs Cloud Armor, and JDBC/PgWire needs a separate L4 load balancer.

1. Create the Cloud Armor security policy and reference the policy name from a GKE BackendConfig:

   apiVersion: cloud.google.com/v1
   kind: BackendConfig
   metadata:
     name: atscale-backend-config
     namespace: <namespace>
   spec:
     healthCheck:
       type: HTTP
       port: 8888
       requestPath: /
     securityPolicy:
       name: "<cloud-armor-policy-name>"

2. Configure GKE Ingress for HTTP. Add the following to your values file:

   global:
     ingressDomain: "atscale.example.com"
     ingressTCPDomain: "tcp.atscale.example.com"
   
   atscale-proxy:
     ingress:
       enabled: true
       ## @param annotations; Add annotations to the L7 loadbalancer created by the cluster ingress
       ## ref: https://docs.cloud.google.com/kubernetes-engine/docs/how-to/load-balance-ingress?hl=en#ingress_annotations
       annotations:
         ## Selects the cluster ingress class
         ## ref: https://docs.cloud.google.com/kubernetes-engine/docs/how-to/load-balance-ingress?hl=en#create-ingress
         kubernetes.io/ingress.class: "gce"
         ## (Optional) Create a LoadBalancer with a reserved global static ip
         kubernetes.io/ingress.global-static-ip-name: "<reserved-global-ip-name>"
     
     service:
       type: NodePort
       annotations:
         ## (Optional) Create a Network Endpoint Group
         ## ref: https://docs.cloud.google.com/kubernetes-engine/docs/how-to/ingress-configuration?hl=en#ingress_sync_failure
         cloud.google.com/neg: '{"ingress": true}'
         ## Apply Backend policy to the L7 LoadBalancer.
         cloud.google.com/backend-config: '{"ports": {"80":"atscale-backend-config"}}'

3. Create a separate Service for the PgWire L4 load balancer:

   apiVersion: v1
   kind: Service
   metadata:
     annotations:
         networking.gke.io/load-balancer-ip-addresses: "<name of your reserved global ip>"
     name: atscale-ingress-gateway-external
     namespace: atscale
   spec:
     type: LoadBalancer
     # (Optional) allow only specific ips to the ingress gateway
     loadBalancerSourceRanges:
       - "<whitelisted-cidr-block>"
     loadBalancerClass: "networking.gke.io/l4-regional-external"
     ports:
     - name: atscale-pgwire-15432
       port: 15432
       protocol: TCP
       targetPort: 15432
     selector:
       app.kubernetes.io/instance: atscale
       app.kubernetes.io/name: ingress-gateway

For information on using Cloud Armor with GKE Ingress, refer to Integrate with other Google products and Ingress configuration in the Google Cloud Armor and GKE documentation.

Microsoft Azure

The following examples show common Microsoft Azure Load Balancer configurations.

L4 Azure Load Balancer

To use an internal Azure Load Balancer for private access, add the following to your values file:

global:
  ingressDomain: "atscale.internal.example.com"

atscale-proxy:
  service:
    type: LoadBalancer
    annotations:
      ## Overrides the default external load-balancer with an internal one
      service.beta.kubernetes.io/azure-load-balancer-internal: "true"

Note

service.beta.kubernetes.io/azure-load-balancer-ipv4 is optional. If set, the IP must be available in the target subnet.

To use a public Azure Load Balancer for when external JDBC/PgWire clients need TCP access, add the following to your values file:

global:
  ingressDomain: "atscale.example.com"

atscale-proxy:
  service:
    type: LoadBalancer
     ## (Optional) allow only specific ips to the ingress gateway
    loadBalancerSourceRanges:
      - "<whitelisted-cidr-block>"

For more information, refer to Azure LoadBalancer in the Cloud Provider Azure documentation, and Use an internal load balancer with Azure Kubernetes Service (AKS) and Use a public standard load balancer in Azure Kubernetes Service (AKS) in the Microsoft documentation.

L7 Ingress with Application Gateway for Containers

Use this setup when HTTP traffic needs Azure L7 ingress features. In this configuration, JDBC/PgWire traffic uses a separate Azure Load Balancer.

To configure Ingress for HTTP traffic, add the following to your values file:

global:
  ingressDomain: "atscale.example.com"

atscale-proxy:
  ingress:
    ## @param ingressClassName; This specifies the ingress controller class defined in the cluster
    ## ref: https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/quickstart-deploy-application-gateway-for-containers-alb-controller-addon?tabs=azure-cli%2Cazure-cli2
    ingressClassName: "azure-alb-external"
    enabled: true
    ## @param annotations
    ## ref: https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/quickstart-create-application-gateway-for-containers-managed-by-alb-controller?tabs=new-subnet-aks-vnet
    annotations:
      alb.networking.azure.io/alb-name: "<ALB-CR-NAME>"
      alb.networking.azure.io/alb-namespace: "ALB-NAMESPACE"
    
    ## @param extraRules; Adds additional paths to the ingress controller
    ## This is required as Azure uses pathType: Prefix instead of ImplementationSpecific.
    extraRules:
      - host: atscale.example.com
        http:
          paths:
            - path: /
              pathType: Prefix
              backend:
                service:
                  name: <release>-ingress-gateway
                  port:
                    name: http

For more information, refer to What is Application Gateway for Containers? in the Microsoft documentation.

Verification

Once you have configured your load balancers in AtScale, you should verify the connection paths.

If you set up a single domain:

• Design Center: https://<ingressDomain>
• JDBC/PgWire: jdbc:postgresql://<ingressDomain>:15432/atscale_catalogs

If you set up separate domains for HTTP and TCP traffic:

• Design Center: https://<ingressDomain>
• JDBC/PgWire: jdbc:postgresql://<ingressTCPDomain>:15432/atscale_catalogs

For more information on client connection details, see Getting Model Connection Information.