c12s-kubespray/docs/metallb.md

# MetalLB

MetalLB hooks into your Kubernetes cluster, and provides a network load-balancer implementation.
It allows you to create Kubernetes services of type "LoadBalancer" in clusters that don't run on a cloud provider, and thus cannot simply hook into 3rd party products to provide load-balancers.
The default operating mode of MetalLB is in ["Layer2"](https://metallb.universe.tf/concepts/layer2/) but it can also operate in ["BGP"](https://metallb.universe.tf/concepts/bgp/) mode.

## Prerequisites

You have to configure arp_ignore and arp_announce to avoid answering ARP queries from kube-ipvs0 interface for MetalLB to work.

```yaml
kube_proxy_strict_arp: true
```

## Install

You have to explicitly enable the MetalLB extension.

```yaml
metallb_enabled: true
metallb_speaker_enabled: true
metallb_avoid_buggy_ips: true
metallb_ip_range:
  - 10.5.0.0/16
```

By default only the MetalLB BGP speaker is allowed to run on control plane nodes. If you have a single node cluster or a cluster where control plane are also worker nodes you may need to enable tolerations for the MetalLB controller:

```yaml
metallb_controller_tolerations:
  - key: "node-role.kubernetes.io/master"
    operator: "Equal"
    value: ""
    effect: "NoSchedule"
  - key: "node-role.kubernetes.io/control-plane"
    operator: "Equal"
    value: ""
    effect: "NoSchedule"
```

## Pools

First you need to specify all of the pools you are going to use:

```yaml
metallb_config:

  address_pools:

    primary:
      ip_range:
        - 172.18.113.167-172.18.113.169
      auto_assign: true

    pool1:
      ip_range:
        - 172.18.113.20-172.18.113.39
        - 172.18.113.120-172.18.113.139
      auto_assign: false # When set to false, you need to explicitly set the loadBalancerIP in the service!

    pool2:
      ip_range:
        - 172.18.113.40-172.18.113.59
        - 172.18.113.140-172.18.113.159
      auto_assign: false
```

## Layer2 Mode

Pools that need to be configured in layer2 mode, need to be specified in a list:

```yaml
metallb_config:

  layer2:
    - primary
```

## BGP Mode

When operating in BGP Mode MetalLB needs to have defined upstream peers and link the pool(s) specified above to the correct peer:

```yaml
metallb_config:

  layer3:
    defaults:

      peer_port: 179 # The TCP port to talk to. Defaults to 179, you shouldn't need to set this in production.
      hold_time: 120s # Requested BGP hold time, per RFC4271.
      keepalive_time: 120s # Requested BGP keepalive time, per RFC4271.

    communities:
      vpn-only: "1234:1"
      NO_ADVERTISE: "65535:65282"

    metallb_peers:

        JRITEST:
          peer_address: 192.0.2.1
          peer_asn: 64512
          my_asn: 4200000000
          communities:
            - vpn-only
          address_pool:
            - pool1

          # (optional) The source IP address to use when establishing the BGP session. In most cases the source-address field should only be used with per-node peers, i.e. peers with node selectors which select only one node. CURRENTLY NOT SUPPORTED BY US (ACC ICT)
          source_address: 172.18.113.161

          # (optional) The router ID to use when connecting to this peer. Defaults to the node IP address.
          # Generally only useful when you need to peer with another BGP router running on the same machine as MetalLB.
          router_id: 1.2.3.4

          # (optional) Password for TCPMD5 authenticated BGP sessions offered by some peers.
          password: "changeme"

        JRITEST2:
          peer_address: 192.0.2.2
          peer_asn: 64513
          my_asn: 4200000000
          communities:
            - NO_ADVERTISE
          address_pool:
            - pool2

          # (optional) The source IP address to use when establishing the BGP session. In most cases the source-address field should only be used with per-node peers, i.e. peers with node selectors which select only one node. CURRENTLY NOT SUPPORTED BY US (ACC ICT)
          source_address: 172.18.113.162

          # (optional) The router ID to use when connecting to this peer. Defaults to the node IP address.
          # Generally only useful when you need to peer with another BGP router running on the same machine as MetalLB.
          router_id: 1.2.3.5

          # (optional) Password for TCPMD5 authenticated BGP sessions offered by some peers.
          password: "changeme"
```

When using calico >= 3.18 you can replace MetalLB speaker by calico Service LoadBalancer IP advertisement.
See [calico service IPs advertisement documentation](https://docs.projectcalico.org/archive/v3.18/networking/advertise-service-ips#advertise-service-load-balancer-ip-addresses).
In this scenario you should disable the MetalLB speaker and configure the `calico_advertise_service_loadbalancer_ips` to match your `metallb_ip_range`

```yaml
metallb_speaker_enabled: false
metallb_avoid_buggy_ips: true
metallb_ip_range:
  - 10.5.0.0/16
calico_advertise_service_loadbalancer_ips: "{{ metallb_ip_range }}"
```

If you have additional loadbalancer IP pool in `metallb_additional_address_pools` , ensure to add them to the list.

```yaml
metallb_speaker_enabled: false
metallb_ip_range:
  - 10.5.0.0/16
metallb_additional_address_pools:
  kube_service_pool_1:
    ip_range:
      - 10.6.0.0/16
    protocol: "bgp"
    auto_assign: false
    avoid_buggy_ips: true
  kube_service_pool_2:
    ip_range:
      - 10.10.0.0/16
    protocol: "bgp"
    auto_assign: false
    avoid_buggy_ips: true
calico_advertise_service_loadbalancer_ips:
  - 10.5.0.0/16
  - 10.6.0.0/16
  - 10.10.0.0/16
```