Lab 03 – Monitoring System: Setup monitoring system with Suricata and Prometheus

Overview

Monitoring System using with Suricata, Filebeat, Elasticsearch, Prometheus and Grafana

Setup monitoring system with suricata, prometheus

Requirements:

• Kali Linux (4GB ram + 2 cores): attack (ping, nmap, hping3, …) to create suricata log
• Ubuntu1 20.04 (4GB ram + 4 cores + 30GB disk): install Suricata + Filebeat + Node Exporter
• Ubuntu2 20.04 (4GB ram + 4 cores + 30GB disk): install Elasticsearch + Prometheus + Grafana

Installation & Configuration

Suricata

Installation

• Update system:
  • sudo apt update
  • sudo apt upgrade
• Prequisites:
  • sudo apt -y install libpcre3 libpcre3-dbg libpcre3-dev build-essential autoconf automake libtool libpcap-dev libnet1-dev libyaml-0-2 libyaml-dev zlib1g zlib1g-dev libcap-ng-dev libcap-ng0 make libmagic-dev libjansson-dev libjansson4 pkg-config
• Install the latest stable Suricata version from PPA:
  • sudo apt-get install software-properties-common
  • sudo add-apt-repository ppa:oisf/suricata-stable
  • sudo apt update
• Install jq to display information from Suricata’s EVE JSON output:
  • sudo apt install suricata jq
• Check version and service state of suricata:
  • sudo suricata –build-info
  • sudo systemctl status suricata

Step by Step Guide to Install Suricata on Ubuntu 20.04

Basic configuration

• Setup network interface:
  • ifconfig
    => Get result as: ens33
  • Use that information to setup configuration file :
    • sudo gedit /etc/suricata/suricata.yaml
      => edit: (line 620)
  • af-packet:
    – interface: ens33
• Update signatures – ruleset for suricata intrusion:
  • sudo suricata-update
  • sudo suricata-update list-sources
  • sudo suricata-update enable-source tgreen/hunting
• Check the suricata configuration file:
  • suricata -T -c /etc/suricata/suricata.yaml -v
    • -T : run Suricata in test mode.
    • -v : print some additional information.
    • -c : tells Suricata where to find its configuration files.
• Run suricata:
  • systemctl start suricata.service (for first time)
  • systemctl start suricata
  • systemctl stop suricata
• Check status:
  • systemctl status suricata
• Start with system:
  • systemctl enable suricata

Default ruleset path

• /var/lib/suricata/rules/suricata.rules
• Ruleset file components: /usr/share/suricata/rules/ ….
  Configure on /etc/suricata/suricata.yaml (line 2162): default-rule-path: /var/lib/suricata/rules
  rule-files:
  • suricata.rules
  • test.rules
  • test.rules => create myself ruleset (signatures) file.

Output file

• Fast log records network alerts log:
  • /var/log/suricata/fast.log
• Eve json records all network events log:
  • /var/log/suricata/eve.json
• Packet counters and memory use log:
  • /var/log/suricata/stats.log
• Pcap mode:
  • Configure on /etc/suricata/suricata.yaml:
    • default-log-dir: /var/log/suricata/ (line 61)
    • outputs: (line 82) # a line based alerts log similar to Snort’s fast.log
      • – fast:
        enabled: yes
        filename: fast.log
        append: yes
        ………
      • – pcap-log: (line 397)
        enabled: yes
        filename: log.pcap

Test

Executing the following command creates an HTTP request, which will return a response that matches Suricata’s warning rules:

curl http://testmynids.org/uid/index.html
=> return alerts log in fast.log

Check realtime log in file /var/log/suricata/fast.log
sudo tail -f /var/log/suricata/fast.log

Filter the eve.json by searching for ID 2100498:
jq ‘select(.alert .signature_id==2100498)’ /var/log/suricata/eve.json

Filebeat

Installation

Download and install Filebeat:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.15.0-amd64.deb

sudo dpkg -i filebeat-8.15.0-amd64.deb

Connect to Elasticsearch

Set the connection information in filebeat.yml:

output.elasticsearch: (line 145)hosts: [“https://192.168.38.134:9200”] (elasticsearch host)
username: “elastic” (elasticsearch username)
password: ” zqAXUy=_XKfLBVkBDrcw ” (elasticsearch password)
ssl:
enabled: true
ca_trusted_fingerprint: “created by using openssl fingerprint http_ca.crt in elasticsearch host to get CA certificate”

Comment Logstash and Kibana if not use

Logstash => line 166

Kibana => line 112

Configure Input

Line 15 in filebeat.yml:

filebeat.inputs:

– type: filestream
id: fastlog-id
enabled: true

paths:
– /var/log/suricata/fast.log
json.keys_under_root: true
json.add_error_key: true
json.message_key: log

Collect log data

Configure data collection modules:

filebeat modules list

Enable one or more modules:

filebeat modules enable suricata

Configure /etc/filebeat/modules.d/suricata.yml:

– module: suricata
access:
enabled: true
var.paths: [“/var/log/ suricata /fast.log”]

Setup configuration and run filebeat

sudo filebeat setup -e
sudo systemctl start/restart/stop/enable filebeat

Test configuration

sudo filebeat test config
sudo filebeat test output