eBPF and Cilium 2026: Cloud-Native Networking & Security Deep Dive

eBPF is revolutionizing how we approach networking, security, and observability in Linux. Combined with Cilium, it provides a powerful platform for cloud-native infrastructure. This comprehensive guide covers everything from eBPF fundamentals to production Cilium deployments.

What is eBPF?

eBPF (extended Berkeley Packet Filter) is a revolutionary technology that allows running sandboxed programs in the Linux kernel without changing kernel source code or loading kernel modules. Originally designed for packet filtering, eBPF has evolved into a general-purpose execution engine that powers everything from networking to security to observability.

Why eBPF Matters

Traditional approaches to kernel-level operations required either:

eBPF solves these problems by providing:

πŸ’‘ Key Insight

eBPF is not just a technologyβ€”it's a paradigm shift. It enables a new class of tools that can observe and secure systems with minimal overhead, making previously impossible tasks routine.

eBPF Architecture

Understanding eBPF's architecture is crucial for leveraging its power effectively.

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ User Space β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ bpftrace β”‚ β”‚ BCC β”‚ β”‚ Cilium β”‚ β”‚ Your Tool β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ libbpf/bpf β”‚ β”‚ β”‚ β”‚ (User Space) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β” β”‚ BPF Syscall β”‚ β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ Kernel Space β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ BPF Verifier β”‚ β”‚ β”‚ β”‚ (Safety Check) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ JIT Compiler β”‚ β”‚ β”‚ β”‚ (x86/ARM/etc.) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ BPF Programs β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ β”‚ β”‚ Network β”‚ β”‚ Kprobe β”‚ β”‚ Tracepointβ”‚ β”‚ LSM β”‚ β”‚ XDP β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ Filter β”‚ β”‚ Entry β”‚ β”‚ Events β”‚ β”‚ Hooks β”‚ β”‚ Hook β”‚ β”‚ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”‚ β”‚ BPF Maps β”‚ β”‚ β”‚ β”‚ (Hash Tables, Arrays, Ring Buffers, Stack Traces, etc.) β”‚ β”‚ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

eBPF Program Types

Program Type Use Case Example Tools
XDP (eXpress Data Path) High-performance packet processing at NIC driver level Cilium, Katran
TC (Traffic Control) Packet filtering, redirection, modification Cilium, Calico
Kprobes/Kretprobes Dynamic kernel function tracing bcc, bpftrace
Tracepoints Static kernel instrumentation points Tetragon, bpftrace
LSM (Linux Security Modules) Security policy enforcement Tetragon, Katran
Uprobes/Uretprobes User-space function tracing Pixie, bpftrace
Socket Filters Socket-level packet filtering Cilium, Suricata

Introduction to Cilium

Cilium is an open-source, cloud-native solution for providing, securing, and observing network connectivity between workloads. It uses eBPF to provide high-performance networking, security, and observability without requiring sidecar proxies.

Cilium's Core Components

Why Cilium?

Traditional Kubernetes networking solutions rely on iptables, which:

Cilium solves these problems with eBPF:

Installing Cilium

Prerequisites

Installation with Helm

# Add Cilium Helm repository
helm repo add cilium https://helm.cilium.io/
helm repo update

# Install Cilium
helm install cilium cilium/cilium \
  --namespace kube-system \
  --set hubble.enabled=true \
  --set hubble.relay.enabled=true \
  --set hubble.ui.enabled=true \
  --set prometheus.enabled=true \
  --set operator.prometheus.enabled=true

# Verify installation
kubectl -n kube-system get pods -l k8s-app=cilium
kubectl -n kube-system exec cilium-xxxxx -- cilium status

Installation with Cilium CLI

# Install Cilium CLI
curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-amd64.tar.gz.sha256sum
tar xzvfC cilium-linux-amd64.tar.gz /usr/local/bin
rm cilium-linux-amd64.tar.gz{,.sha256sum}

# Install Cilium
cilium install \
  --version 1.15.0 \
  --set cluster.name=my-cluster \
  --set cluster.id=1

# Enable Hubble
cilium hubble enable --ui

# Verify
cilium status --wait
cilium connectivity test

Kernel Requirements Check

# Check kernel configuration
kubectl create -f https://raw.githubusercontent.com/cilium/cilium/main/examples/kubernetes/kernel-check/kernel-check.yaml
kubectl logs kernel-check

# Required kernel features:
# - CONFIG_BPF=y
# - CONFIG_BPF_SYSCALL=y
# - CONFIG_NET_CLS_BPF=y
# - CONFIG_BPF_JIT=y
# - CONFIG_HAVE_EBPF_JIT=y
# - CONFIG_CGROUP_BPF=y

# Check eBPF filesystem
mount | grep bpf
# Should show: /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)

Cilium Networking

Cilium's Data Path

# Cilium Packet Flow
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                         Pod Network Namespace                   β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                                               β”‚
β”‚  β”‚   Pod A     β”‚                                               β”‚
β”‚  β”‚  (eth0)     β”‚                                               β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜                                               β”‚
β”‚         β”‚                                                       β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”                                               β”‚
β”‚  β”‚   veth pair β”‚                                               β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜                                               β”‚
β”‚         β”‚                                                       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
          β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚         β”‚              Host Network Namespace                   β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”                                               β”‚
β”‚  β”‚  lxc_XXXXX  β”‚  ← Cilium-managed virtual interface           β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜                                               β”‚
β”‚         β”‚                                                       β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”                                               β”‚
β”‚  β”‚  eBPF Hook  β”‚  ← Cilium eBPF program (tc/ingress)           β”‚
β”‚  β”‚  (Identity  β”‚     - Identity lookup                         β”‚
β”‚  β”‚   Lookup)   β”‚     - Policy check                            β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜     - Load balancing                          β”‚
β”‚         β”‚                                                       β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”                                               β”‚
β”‚  β”‚  eBPF Maps  β”‚  ← Conntrack, LB backends, Policy rules       β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜                                               β”‚
β”‚         β”‚                                                       β”‚
β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”                                               β”‚
β”‚  β”‚   eth0      β”‚  ← Physical interface (or overlay)              β”‚
β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                                               β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

IP Address Management (IPAM)

Cilium supports multiple IPAM modes:

# Cluster Pool (default) - Cilium manages pod CIDRs
helm install cilium cilium/cilium \
  --namespace kube-system \
  --set ipam.mode=cluster-pool \
  --set ipam.operator.clusterPoolIPv4PodCIDRList=10.0.0.0/8 \
  --set ipam.operator.clusterPoolIPv4MaskSize=24

# Kubernetes Host Scope - Uses node CIDRs from Kubernetes
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set ipam.mode=kubernetes

# ENI Mode (AWS) - Uses AWS ENIs for pods
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set ipam.mode=en \
  --set egressGateway.enabled=true

# Azure IPAM
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set ipam.mode=azure

# GCP IPAM
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set ipam.mode=cluster-pool \
  --set gke.enabled=true

Load Balancing with Cilium

Cilium provides high-performance load balancing using eBPF:

# Enable NodePort and LoadBalancer acceleration
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set nodePort.enabled=true \
  --set nodePort.mode=dsr \
  --set loadBalancer.mode=dsr \
  --set loadBalancer.acceleration=native

# DSR (Direct Server Return) mode avoids extra hops
# Native acceleration uses XDP for packet processing

# Verify load balancer
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf lb list

Bandwidth Manager

Cilium can enforce pod bandwidth limits using eBPF:

# Enable bandwidth manager
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set bandwidthManager.enabled=true

# Apply bandwidth limits via pod annotations
apiVersion: v1
kind: Pod
metadata:
  name: limited-pod
  annotations:
    kubernetes.io/egress-bandwidth: "10M"
    kubernetes.io/ingress-bandwidth: "10M"
spec:
  containers:
  - name: app
    image: nginx

Network Security with Cilium

Identity-Based Security

Unlike traditional firewalls that use IP addresses, Cilium uses security identities derived from labels:

# Cilium assigns identities based on labels
# Pod with labels: app=frontend, tier=web, team=platform
# Gets identity: 12345 (arbitrary number)

# Security policies reference identities, not IPs
# This allows pods to move between nodes without policy changes

# View identities
kubectl -n kube-system exec cilium-xxxxx -- cilium identity list

Network Policies

CiliumNetworkPolicy extends Kubernetes NetworkPolicy with powerful features:

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: frontend-policy
  namespace: production
spec:
  endpointSelector:
    matchLabels:
      app: frontend
  ingress:
    # Allow from ingress controller
    - fromEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: ingress-nginx
            app.kubernetes.io/name: ingress-nginx
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: GET
                path: "/api/.*"
              - method: POST
                path: "/api/users"
                headers:
                  - name: X-Request-ID
                    required: true
  egress:
    # Allow to backend
    - toEndpoints:
        - matchLabels:
            app: backend
      toPorts:
        - ports:
            - port: "8080"
              protocol: TCP
          rules:
            http:
              - method: GET
                path: "/health"
              - method: POST
                path: "/api/.*"
    # Allow DNS
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP
          rules:
            dns:
              - matchPattern: "*.cluster.local"
              - matchName: "api.external-service.com"
    # Allow to external API with TLS inspection
    - toFQDNs:
        - matchName: "api.stripe.com"
      toPorts:
        - ports:
            - port: "443"
              protocol: TCP
          terminatingTLS:
            certificate: "external-api-ca"
          rules:
            http:
              - method: POST
                path: "/v1/charges"

Layer 7 Protocol Support

Cilium can enforce policies at the application layer:

# HTTP-specific rules
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
spec:
  endpointSelector:
    matchLabels:
      app: api-gateway
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: web-client
      toPorts:
        - ports:
            - port: "80"
              protocol: TCP
          rules:
            http:
              - method: GET
                path: "/public/.*"
              - method: GET
                path: "/api/v1/users"
                headers:
                  - name: Authorization
                    presence: true

# Kafka-specific rules
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
spec:
  endpointSelector:
    matchLabels:
      app: kafka-client
  egress:
    - toEndpoints:
        - matchLabels:
            app: kafka
      toPorts:
        - ports:
            - port: "9092"
              protocol: TCP
          rules:
            kafka:
              - role: produce
                topic: "events.*"
              - role: consume
                topic: "notifications"

# gRPC-specific rules
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
spec:
  endpointSelector:
    matchLabels:
      app: grpc-service
  ingress:
    - fromEndpoints:
        - matchLabels:
            app: grpc-client
      toPorts:
        - ports:
            - port: "50051"
              protocol: TCP
          rules:
            http:
              - method: POST
                path: "/my.package.Service/AllowedMethod"

Host Firewall

Cilium can protect the host itself, not just pods:

# Enable host firewall
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set hostFirewall.enabled=true

# Policy to protect host
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: host-firewall
spec:
  nodeSelector:
    matchLabels:
      node-type: worker
  ingress:
    # Allow SSH from management network
    - fromCIDR:
        - 10.0.0.0/24
      toPorts:
        - ports:
            - port: "22"
              protocol: TCP
    # Allow kubelet from control plane
    - fromEntities:
        - host
        - remote-node
      toPorts:
        - ports:
            - port: "10250"
              protocol: TCP
  egress:
    # Allow DNS
    - toEndpoints:
        - matchLabels:
            k8s:io.kubernetes.pod.namespace: kube-system
            k8s-app: kube-dns
      toPorts:
        - ports:
            - port: "53"
              protocol: UDP

Observability with Hubble

What is Hubble?

Hubble is a fully distributed networking and security observability platform built on top of Cilium and eBPF. It provides deep visibility into service communication without requiring sidecar proxies.

Hubble Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                           Hubble UI                             β”‚
β”‚                    (Service Graph, Flow Viz)                    β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                      Hubble Relay                           β”‚
β”‚              (Aggregates flows from all nodes)              β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
        β”‚                     β”‚                     β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Hubble       β”‚    β”‚  Hubble       β”‚    β”‚  Hubble       β”‚
β”‚  Observer     β”‚    β”‚  Observer     β”‚    β”‚  Observer     β”‚
β”‚  (Node 1)     β”‚    β”‚  (Node 2)     β”‚    β”‚  (Node 3)     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜
        β”‚                     β”‚                     β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”
β”‚  Cilium       β”‚    β”‚  Cilium       β”‚    β”‚  Cilium       β”‚
β”‚  Agent        β”‚    β”‚  Agent        β”‚    β”‚  Agent        β”‚
β”‚  (eBPF)       β”‚    β”‚  (eBPF)       β”‚    β”‚  (eBPF)       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Flow Monitoring

# View flows in real-time
hubble observe -n production

# Filter by source and destination
hubble observe --from-pod frontend/ --to-pod backend/

# Filter by protocol and port
hubble observe --protocol TCP --port 8080

# Show dropped flows only
hubble observe --verdict DROPPED

# Export flows for analysis
hubble observe --output json > flows.json

Service Dependency Graph

# Access Hubble UI
kubectl -n kube-system port-forward svc/hubble-ui 12000:80
# Open http://localhost:12000

# Or use CLI for service map
hubble status
hubble list

# Get metrics
kubectl -n kube-system exec deploy/hubble-relay -- hubble observe --metric-count

Prometheus Metrics

# Enable Prometheus metrics
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set prometheus.enabled=true \
  --set operator.prometheus.enabled=true \
  --set hubble.metrics.enabled="{dns,drop,tcp,flow,icmp,http}"

# Key metrics:
# - hubble_flows_processed_total
# - hubble_drop_total
# - hubble_tcp_flags_total
# - hubble_http_requests_total
# - hubble_http_response_time_seconds
# - hubble_dns_queries_total
# - hubble_icmp_total

# Example Prometheus queries:
# Drop rate by reason
sum(rate(hubble_drop_total[5m])) by (reason)

# HTTP request rate by status
sum(rate(hubble_http_requests_total[5m])) by (status)

# DNS query rate
sum(rate(hubble_dns_queries_total[5m]))

# Flows by source namespace
sum(rate(hubble_flows_processed_total[5m])) by (source_namespace)

Runtime Security with Tetragon

What is Tetragon?

Tetragon is a Kubernetes-aware security observability and runtime enforcement tool that uses eBPF to provide:

Installing Tetragon

# Install Tetragon
helm repo add cilium https://helm.cilium.io
helm install tetragon cilium/tetragon -n kube-system

# Verify installation
kubectl -n kube-system rollout status -w deployment/tetragon-operator
kubectl -n kube-system rollout status -w ds/tetragon

# Access logs
kubectl -n kube-system logs -l app.kubernetes.io/name=tetragon -c export

Tracing Policies

# Monitor process execution
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: process-execution
spec:
  kprobes:
    - call: "__x64_sys_execve"
      syscall: true
      args:
        - index: 0
          type: "string"
        - index: 1
          type: "string"
      selectors:
        - matchBinaries:
            - operator: "In"
              values:
                - "/bin/bash"
                - "/bin/sh"

# Monitor file access
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: sensitive-files
spec:
  kprobes:
    - call: "security_file_permission"
      args:
        - index: 0
          type: "string"
      selectors:
        - matchArgs:
            - index: 0
              operator: "Prefix"
              values:
                - "/etc/shadow"
                - "/etc/passwd"

# Network-based detection
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: network-connections
spec:
  kprobes:
    - call: "tcp_connect"
      syscall: false
      args:
        - index: 0
          type: "sock"
      selectors:
        - matchArgs:
            - index: 1
              operator: "NotDAddr"
              values:
                - "10.0.0.0/8"
                - "172.16.0.0/12"
                - "192.168.0.0/16"
          matchActions:
            - action: "Notify"
            - action: "FollowFD"
            - action: "Sigkill"

Runtime Enforcement

# Kill processes that access /etc/shadow
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: protect-shadow
spec:
  kprobes:
    - call: "security_file_permission"
      args:
        - index: 0
          type: "string"
      selectors:
        - matchArgs:
            - index: 0
              operator: "Equal"
              values:
                - "/etc/shadow"
          matchActions:
            - action: "Sigkill"

# Block reverse shells
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: block-reverse-shell
spec:
  kprobes:
    - call: "tcp_connect"
      args:
        - index: 1
          type: "sockaddr"
      selectors:
        - matchArgs:
            - index: 1
              operator: "NotDAddr"
              values:
                - "10.0.0.0/8"
          matchBinaries:
            - operator: "In"
              values:
                - "/bin/bash"
                - "/bin/sh"
          matchActions:
            - action: "Sigkill"

Cilium Service Mesh

Cilium vs Istio

Cilium Service Mesh provides many service mesh features without sidecar proxies:

Feature Cilium Service Mesh Istio (Sidecar)
Architecture Kernel-level (eBPF) Sidecar proxy (Envoy)
Latency Overhead Minimal (~1-2%) Higher (~5-10%)
Resource Usage Low (no extra containers) Higher (sidecar per pod)
mTLS Yes (via WireGuard or certificate) Yes (Envoy mTLS)
Traffic Management Basic (L3/L4 + some L7) Advanced (full L7)
Observability Excellent (Hubble) Excellent (Kiali, Grafana)

Enabling Service Mesh Features

# Enable Cilium Service Mesh
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set serviceMesh.enabled=true \
  --set loadBalancer.l7.backend=envoy

# Enable mTLS with WireGuard
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set encryption.enabled=true \
  --set encryption.type=wireguard

# Or use IPsec
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set encryption.enabled=true \
  --set encryption.type=ipsec

# Verify encryption
kubectl -n kube-system exec cilium-xxxxx -- cilium status | grep Encryption

Ingress Controller

# Enable Cilium Ingress Controller
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set ingressController.enabled=true \
  --set ingressController.loadbalancerMode=dedicated

# Create an Ingress resource
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-app
  annotations:
    ingress.cilium.io/loadbalancer-mode: dedicated
spec:
  ingressClassName: cilium
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: my-service
                port:
                  number: 80
  tls:
    - hosts:
        - app.example.com
      secretName: app-tls

Gateway API Support

# Enable Gateway API
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set gatewayAPI.enabled=true

# Install Gateway API CRDs
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml

# Create a Gateway
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
spec:
  gatewayClassName: cilium
  listeners:
    - name: https
      protocol: HTTPS
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
          - name: my-cert

# Create HTTPRoute
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-route
spec:
  parentRefs:
    - name: my-gateway
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /api
      backendRefs:
        - name: api-service
          port: 8080

Performance Optimization

eBPF Map Sizing

# Tune eBPF map sizes for your workload
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set bpf.mapDynamicSizeRatio=0.0025 \
  --set bpf.policyMapMax=65536 \
  --set bpf.lbMapMax=65536

# Monitor map pressure
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf map list
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf map events cilium_ipcache

XDP Acceleration

# Enable XDP for maximum performance
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set bpf.masquerade=true \
  --set enableXDPAcceleration=true

# Check XDP mode
kubectl -n kube-system exec cilium-xxxxx -- cilium status | grep XDP

# Modes: native (best), generic (fallback), disabled

CPU Optimization

# Pin Cilium to specific CPUs
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[0].labelSelector.matchLabels."app.kubernetes.io/name"=cilium \
  --set affinity.podAntiAffinity.requiredDuringSchedulingIgnoredDuringExecution[0].topologyKey=kubernetes.io/hostname

# Enable CPU profiling for analysis
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set pprof.enabled=true

# Access pprof
kubectl -n kube-system port-forward pod/cilium-xxxxx 6060:6060
curl http://localhost:6060/debug/pprof/profile > cpu.prof

Production Deployment

High Availability Setup

# Production values
helm upgrade cilium cilium/cilium \
  --namespace kube-system \
  --set operator.replicas=2 \
  --set operator.resources.requests.cpu=100m \
  --set operator.resources.requests.memory=128Mi \
  --set operator.resources.limits.cpu=500m \
  --set operator.resources.limits.memory=256Mi \
  --set resources.requests.cpu=100m \
  --set resources.requests.memory=512Mi \
  --set resources.limits.cpu=1000m \
  --set resources.limits.memory=1Gi \
  --set hubble.relay.replicas=2 \
  --set hubble.relay.resources.requests.cpu=100m \
  --set hubble.relay.resources.requests.memory=128Mi \
  --set hubble.relay.resources.limits.cpu=500m \
  --set hubble.relay.resources.limits.memory=256Mi \
  --set hubble.ui.replicas=2 \
  --set prometheus.enabled=true \
  --set operator.prometheus.enabled=true \
  --set hubble.metrics.enabled="{dns:query,drop,tcp,flow,icmp,http,port-distribution,flows-to-world}"

Backup and Disaster Recovery

# Backup Cilium identities and policies
kubectl get ciliumidentities -o yaml > cilium-identities-backup.yaml
kubectl get ciliumnetworkpolicies -o yaml > cilium-policies-backup.yaml
kubectl get ciliumclusterwidenetworkpolicies -o yaml > cilium-cluster-policies-backup.yaml

# Backup etcd (if using etcd mode)
etcdctl snapshot save cilium-etcd-backup.db

# Restore procedure
# 1. Reinstall Cilium with same configuration
# 2. Restore identities: kubectl apply -f cilium-identities-backup.yaml
# 3. Restore policies: kubectl apply -f cilium-policies-backup.yaml

Monitoring and Alerting

# Key alerts to configure

# High drop rate
sum(rate(hubble_drop_total[5m])) by (reason) > 100

# Identity allocation pressure
cilium_identity_count > 50000

# BPF map pressure
cilium_bpf_map_pressure > 0.9

# Node connectivity issues
cilium_node_connectivity_status{status!="reachable"} > 0

# Policy import failures
cilium_policy_import_errors_total > 0

# IPAM exhaustion
cilium_ipam_available < 10

# Hubble dropped flows
hubble_dropped_flows_total > 1000

Troubleshooting

# Check Cilium status
kubectl -n kube-system exec cilium-xxxxx -- cilium status

# Check for errors
kubectl -n kube-system logs -l k8s-app=cilium --tail=100 | grep -i error

# Verify eBPF programs loaded
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf prog list

# Check policy enforcement
kubectl -n kube-system exec cilium-xxxxx -- cilium policy get

# Debug connectivity
kubectl -n kube-system exec cilium-xxxxx -- cilium-health status

# Capture flows
kubectl -n kube-system exec cilium-xxxxx -- cilium monitor -v

# Check BPF maps
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf endpoint list
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf ipcache list
kubectl -n kube-system exec cilium-xxxxx -- cilium bpf policy get

# Sysdump for support
cilium sysdump

Conclusion

eBPF and Cilium represent a fundamental shift in how we approach cloud-native networking, security, and observability. By moving functionality from user space and sidecar proxies into the kernel, they provide better performance, lower resource usage, and deeper visibility than traditional approaches.

The ecosystem around eBPF continues to grow rapidly. In 2026, Cilium is the default CNI for many Kubernetes distributions, and eBPF is becoming the standard for kernel-level observability and security. Tools like Tetragon and Hubble extend Cilium's capabilities into runtime security and deep observability.

For organizations running Kubernetes at scale, Cilium is no longer optionalβ€”it's essential. The combination of high-performance networking, identity-based security, and comprehensive observability provides a platform that scales with your infrastructure while reducing operational complexity.

πŸš€ Next Steps

1. Evaluate Cilium in a test environment
2. Plan your migration from existing CNI
3. Implement network policies incrementally
4. Enable Hubble for observability
5. Consider Tetragon for runtime security
6. Join the Cilium community for support