NetFlow and sFlow Traffic Analysis for Network Visibility

Introduction

NetFlow and sFlow are two industry-standard protocols for collecting network traffic statistics. As a junior infrastructure engineer, understanding traffic analysis helps you answer questions like: "Who is consuming all our bandwidth?", "Is there unusual traffic that could indicate a breach?", and "Which applications are most used?" This guide covers both protocols, their differences, and how to deploy a complete traffic analysis stack.

NetFlow vs sFlow: Core Differences

NetFlow (Cisco proprietary, now standardized as IPFIX) works by tracking every TCP/UDP flow — a unique combination of source IP, destination IP, source port, destination port, and protocol. The router/switch caches these flows and exports them as records to a collector. sFlow (RFC 3176) uses statistical sampling — it randomly samples 1 in N packets (e.g., 1 in 512) and exports the raw packet headers. This is more scalable for high-speed links but less precise for small flows.

Key difference: NetFlow gives you exact flow counts but requires CPU on the router. sFlow samples traffic so it scales to 100G+ links without router CPU impact.

Setting Up a NetFlow Collector with nfdump/nfcapd

BASH

# Install nfdump on Ubuntu/Debian
sudo apt install nfdump

# Create directory for flow data
sudo mkdir -p /var/netflow/data
sudo chown netflow:netflow /var/netflow/data

# Start nfcapd (collector daemon) listening on UDP 9995
sudo nfcapd -D -l /var/netflow/data -p 9995 -P /var/run/nfcapd.pid

# Verify it's listening
sudo netstat -ulnp | grep 9995

Configure a Cisco router to export NetFlow:

TEXT

! On Cisco IOS router
interface GigabitEthernet0/0
 ip flow ingress
 ip flow egress

ip flow-export version 9
ip flow-export destination 192.168.1.100 9995
ip flow-export source Loopback0

ip flow-cache timeout active 1
ip flow-cache timeout inactive 15

Reading NetFlow Data with nfdump

BASH

# List top 10 talkers by bytes in last hour
nfdump -R /var/netflow/data -t "2024-01-15/13:00:00-2024-01-15/14:00:00"   -s srcip/bytes -n 10

# Show all traffic from suspicious IP
nfdump -R /var/netflow/data -t "2024-01-15/13:00:00"   "src ip 10.0.0.100" -o extended

# Find top destination ports (services being accessed)
nfdump -R /var/netflow/data -s dstport/flows -n 20

# Show traffic between two hosts
nfdump -R /var/netflow/data "(src ip 10.0.0.50 and dst ip 8.8.8.8)"

Output example:

TEXT

Date first seen          Duration  Proto   Src IP Addr:Port    Dst IP Addr:Port    Bytes
2024-01-15 13:05:22.450   0.012    TCP    10.0.0.50:54321  ->  172.16.0.5:443    15234
2024-01-15 13:05:22.462   0.001    UDP    10.0.0.50:1024   ->  8.8.8.8:53          82

Deploying ntopng for Visual Traffic Analysis

ntopng provides a web dashboard on top of NetFlow/sFlow data:

BASH

# Add ntop repository
wget -qO- https://packages.ntop.org/apt-stable/ntop.key | sudo apt-key add -
echo "deb https://packages.ntop.org/apt-stable/22.04/ x64/" | sudo tee /etc/apt/sources.list.d/ntop.list

sudo apt update && sudo apt install ntopng nprobe

# Configure ntopng
sudo nano /etc/ntopng/ntopng.conf

TEXT

# ntopng.conf
--interface=eth0          # Interface to monitor (or use nprobe for NetFlow)
--http-port=3000
--redis=127.0.0.1
--data-dir=/var/lib/ntopng
--user=ntopng

BASH

sudo systemctl enable ntopng && sudo systemctl start ntopng
# Access at http://your-server:3000 (admin/admin default)

sFlow Configuration and Collection

Configure sFlow on a Linux server using Host sFlow agent:

BASH

# Install hsflowd
sudo apt install hsflowd

# Configure /etc/hsflowd.conf
sudo cat > /etc/hsflowd.conf << 'CONF'
sflow {
  sampling = 512           # Sample 1 in 512 packets
  polling = 30             # Poll counters every 30 seconds
  agentIP = 192.168.1.10   # This server's IP to identify in collector
  DNSSD = off

  collector {
    ip = 192.168.1.100     # Collector IP
    udpport = 6343         # sFlow port
  }

  pcap { dev = eth0 }
}
CONF

sudo systemctl enable hsflowd && sudo systemctl start hsflowd

Building a Full Stack with ELK + Logstash

TEXT

# logstash pipeline for NetFlow
input {
  udp {
    port  => 9996
    codec => netflow {
      versions => [5, 9]
    }
  }
}

filter {
  mutate {
    add_field => { "[@metadata][index]" => "netflow" }
  }

  geoip {
    source => "[netflow][ipv4_src_addr]"
    target => "src_geo"
  }

  geoip {
    source => "[netflow][ipv4_dst_addr]"
    target => "dst_geo"
  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "netflow-%{+YYYY.MM.dd}"
  }
}

Detecting Anomalies in Traffic Data

Common patterns to look for:

BASH

# Detect port scans: one source hitting many destination ports
nfdump -R /var/netflow/data -t "2024-01-15/13:00:00-14:00:00"   -s srcip/flows -n 100 |   awk '$4 > 1000 {print "POSSIBLE SCAN:", $0}'

# Detect data exfiltration: high bytes to external IPs
nfdump -R /var/netflow/data "bytes > 100000000 and not (dst net 10.0.0.0/8)"   -o "fmt: %ts %sa %da %byt %fl"

# Find DNS amplification (large DNS responses)
nfdump -R /var/netflow/data "proto udp and src port 53 and bytes > 512" -s dstip/bytes

Grafana Dashboard for Traffic Visualization

YAML

# docker-compose.yml for traffic analysis stack
version: '3.8'
services:
  influxdb:
    image: influxdb:2.7
    ports: ["8086:8086"]
    environment:
      DOCKER_INFLUXDB_INIT_MODE: setup
      DOCKER_INFLUXDB_INIT_USERNAME: admin
      DOCKER_INFLUXDB_INIT_PASSWORD: password123
      DOCKER_INFLUXDB_INIT_ORG: myorg
      DOCKER_INFLUXDB_INIT_BUCKET: netflow

  grafana:
    image: grafana/grafana:latest
    ports: ["3000:3000"]
    depends_on: [influxdb]

  pmacct:
    image: pmacct/pmacctd:latest
    network_mode: host
    volumes:
      - ./pmacctd.conf:/etc/pmacct/pmacctd.conf

Capacity Planning with Traffic Data

PYTHON

#!/usr/bin/env python3
# analyze_traffic.py - Parse nfdump output for capacity planning
import subprocess
import datetime

def get_hourly_traffic(date_str, interface_net="10.0.0.0/8"):
    '''Get hourly traffic summary for capacity planning.'''
    results = []
    for hour in range(24):
        start = f"{date_str}/{hour:02d}:00:00"
        end = f"{date_str}/{hour:02d}:59:59"

        cmd = [
            "nfdump", "-R", "/var/netflow/data",
            "-t", f"{start}-{end}",
            f"net {interface_net}",
            "-o", "csv", "-q"
        ]

        result = subprocess.run(cmd, capture_output=True, text=True)
        total_bytes = sum(
            int(line.split(',')[13])
            for line in result.stdout.splitlines()[1:]
            if line and not line.startswith('#')
        )
        results.append({'hour': hour, 'bytes': total_bytes,
                        'mbps': total_bytes * 8 / 3600 / 1_000_000})

    return results

if __name__ == "__main__":
    data = get_hourly_traffic("2024-01-15")
    peak = max(data, key=lambda x: x['mbps'])
    print(f"Peak traffic: {peak['mbps']:.1f} Mbps at {peak['hour']:02d}:00")
    print(f"Recommendation: provision at least {peak['mbps'] * 1.5:.0f} Mbps")

Troubleshooting Common Issues

Flows not arriving at collector:

BASH

# Check if router is sending (packet capture)
sudo tcpdump -i eth0 udp port 9995 -n

# Verify nfcapd is running and writing files
ls -la /var/netflow/data/
tail -f /var/log/syslog | grep nfcapd

Missing flows (flow cache full):

TEXT

! Cisco: reduce cache timeout
ip flow-cache timeout active 1    # export active flows every 1 minute
ip flow-cache timeout inactive 5  # expire idle flows after 5 seconds
ip flow-cache entries 65536       # increase cache size

High CPU on router due to NetFlow:

TEXT

! Disable NetFlow on internal interfaces, only monitor edge
interface GigabitEthernet0/0.100
 no ip flow ingress   ! Internal VLAN - remove flow monitoring

Traffic analysis is one of the most powerful tools in a network engineer's toolkit. Start with basic nfdump queries to understand traffic patterns, then graduate to visual tools like ntopng or Grafana dashboards for ongoing monitoring.