使用Suricata构建网络层入侵检测

0x00 前言

小团队,安全投入有限,入侵检测能力不足,攻防对抗不对等,实在尴尬,迫不得已选择开源方案!

以下是个人折腾Suricata的一些纪录、想法,不是很成熟、甚至可能观点有错,分享出来,万一有共鸣呢?

0x01 Suricata简介

Suricata是一款基于TCP/IP协议栈解析与安全数据分析引擎:

  • 能够进行实时入侵检测(IDS)、内联入侵预防(IPS)、网络安全监控(NSM)和离线PCAP处理,全面支持Snort规则;
  • Suricata使用强大而广泛的规则和签名语言检查网络流量,并具有强大的Lua脚本支持来检测复杂的威胁;
  • 使用标准的输入和输出格式(如yaml和json),与现有的siem、splunk、logstash/elasticsearch、kibana和其他数据库等工具的集成变得很容易;
  • 入侵检测规则更新活跃,具有较强的社区支持。

支持数据包解码:

  • IPv4, IPv6, TCP, UDP, SCTP, ICMPv4, ICMPv6, GRE
  • Ethernet, PPP, PPPoE, Raw, SLL, VLAN, QINQ, MPLS, ERSPAN
  • HTTP,SSL,TLS,SMB,DCERPC,SMTP,FTP,SSH,DNS,Modbus,ENIP / CIP,DNP3,NFS,NTP,DHCP,TFTP,KRB5,IKEv2

0x02 选择Suricata的原因

要做网络层的入侵检、流量分析,能想到的就是Snort、Suricata、Bro,它们是业界比较成熟的开源方案,许多安全公司招聘也列出了熟悉 Suricata 优先,也是很‘优秀’。

  • Snort始于1998,对于适度流量场景,算是一个比较好的解决方案;
  • Suricata始于2009,我个人理解为是针对大规模网络的snort扩展,在大流量环境下,丢包率源低于snort,性能表现更优秀;
  • Bro是一种被动的开源网络流量分析器,可以检查链路上的所有流量,以查看可疑活动的迹象,大流量环境下表现比较优秀。Bro支持甚至在安全域之外的各种流量分析任务,包括性能测量和帮助解决问题,与Snort或Suricata中的规则集相比,其强大的脚本功能绝对具有更大的优势。

可以参考:

https://blog.csdn.net/yrx0619/article/details/81267236

https://bricata.com/resources/white-paper/bro-vs-snot-or-suricata/

最终,选择Suricata的原因是方便上手,与snort相通,社区资料也比较多,Bro的资料相比之下就少很多了,并且还要花时间研究怎么写脚本,精力与回报不成正比。

0x03 DIY Suricata

实际业务环境下:

  • 机房分布也比较多,需要镜像过来做分析的流量分散;
  • 边界流量比较大(单个机房最少的,一分钟也有7、8个g的量),对于这种大流量,直接部署Suricata肯定没啥用,根本扛不住;
  • 分析结果怎么存储展示?
  • 告警一大堆,谁来看?

所以:

  • Suricata分布式部署,适配多机房的业务场景,数据统计上报到es;
  • 流量若扛不住,就将导入的镜像流量使用dumpcap进行切割后再给Suricata进行分析;
  • Suricata分析出来的日志存elasticsearch(elk),大数据分析;
  • diy了一个安全分析后台,把已有的hids数据、日志系统的数据关联起来,发现更有价值、紧急层度更高的攻击事件;
  • 综合分析出来的攻击事件,通过硬件FW、系统iptables阻断。

image

0x04 部署

Suricata部署:

centos7上部署,部署的版本为:Suricata 4.0.5

1
2
3
4
yum install epel-release
yum install suricata

yum install wget libpcap-devel libnet-devel pcre-devel gcc-c++ automake autoconf libtool make libyaml-devel zlib-devel file-devel jansson-devel nss-devel

ELK部署

我部署的6.2版本,去网上下载,参照着部署即可,具体过程略

1
2
3
elasticsearch-6.2.0.rpm
logstash-6.2.0.rpm
kibana-6.2.0-x86_64.rpm

Suricata规则及配置:

1、直接更新替换

1
wget  https://rules.emergingthreats.net/open/suricata-4.0/emerging.rules.tar.gz

2、suricata规则更新可以使用suricata-update来进行更新

1
2
3
yum install python-pip python-yaml

pip install --pre --upgrade suricata-update

输入suricata-update 会自动进行规则更新,显示当前已经更新与启用了多少规则

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[root@test_nsm_nids suricata]# suricata-update
6/5/2019 -- 11:40:35 - <Info> -- Using data-directory /var/lib/suricata.
6/5/2019 -- 11:40:35 - <Info> -- Using Suricata configuration /etc/suricata/suricata.yaml
6/5/2019 -- 11:40:35 - <Info> -- Using /etc/suricata/rules for Suricata provided rules.
6/5/2019 -- 11:40:35 - <Info> -- Found Suricata version 4.0.5 at /usr/sbin/suricata.
6/5/2019 -- 11:40:35 - <Info> -- Loading /etc/suricata/suricata.yaml
6/5/2019 -- 11:40:35 - <Info> -- Disabling rules with proto ntp
6/5/2019 -- 11:40:35 - <Info> -- Disabling rules with proto modbus
6/5/2019 -- 11:40:35 - <Info> -- Disabling rules with proto enip
6/5/2019 -- 11:40:35 - <Info> -- Disabling rules with proto dnp3
6/5/2019 -- 11:40:35 - <Info> -- Disabling rules with proto nfs
6/5/2019 -- 11:40:35 - <Info> -- No sources configured, will use Emerging Threats Open
6/5/2019 -- 11:40:35 - <Info> -- Checking https://rules.emergingthreats.net/open/suricata-4.0.5/emerging.rules.tar.gz.md5.
6/5/2019 -- 11:40:46 - <Info> -- Fetching https://rules.emergingthreats.net/open/suricata-4.0.5/emerging.rules.tar.gz.
100% - 2352266/2352266
6/5/2019 -- 11:40:50 - <Info> -- Done.
6/5/2019 -- 11:40:50 - <Info> -- Ignoring file rules/emerging-deleted.rules
6/5/2019 -- 11:40:55 - <Info> -- Loaded 24532 rules.
6/5/2019 -- 11:40:56 - <Info> -- Disabled 0 rules.
6/5/2019 -- 11:40:56 - <Info> -- Enabled 0 rules.
6/5/2019 -- 11:40:56 - <Info> -- Modified 0 rules.
6/5/2019 -- 11:40:56 - <Info> -- Dropped 0 rules.
6/5/2019 -- 11:40:56 - <Info> -- Enabled 38 rules for flowbit dependencies.
6/5/2019 -- 11:40:56 - <Info> -- Backing up current rules.
6/5/2019 -- 11:41:02 - <Info> -- Writing rules to /var/lib/suricata/rules/suricata.rules: total: 24532; enabled: 19617; added: 783; removed 8; modified: 1338
6/5/2019 -- 11:41:03 - <Info> -- Testing with suricata -T.

3、Suricata.yaml配置文件

网络配置:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
##
## Step 1: inform Suricata about your network
##

vars:
# more specifc is better for alert accuracy and performance
address-groups:
#HOME_NET: "[221.101.0.0/16]"
#HOME_NET: "[192.168.0.0/16]"
#HOME_NET: "[10.0.0.0/8]"
#HOME_NET: "[172.16.0.0/12]"
HOME_NET: "any"

#EXTERNAL_NET: "!$HOME_NET"
EXTERNAL_NET: "any"

HTTP_SERVERS: "$HOME_NET"
SMTP_SERVERS: "$HOME_NET"
SQL_SERVERS: "$HOME_NET"
DNS_SERVERS: "$HOME_NET"
TELNET_SERVERS: "$HOME_NET"
AIM_SERVERS: "$EXTERNAL_NET"
DNP3_SERVER: "$HOME_NET"
DNP3_CLIENT: "$HOME_NET"
MODBUS_CLIENT: "$HOME_NET"
MODBUS_SERVER: "$HOME_NET"
ENIP_CLIENT: "$HOME_NET"
ENIP_SERVER: "$HOME_NET"

port-groups:
HTTP_PORTS: "80,8081,8080,443"
SHELLCODE_PORTS: "!80"
ORACLE_PORTS: 1521
SSH_PORTS: "22,37222"
DNP3_PORTS: 20000
MODBUS_PORTS: 502
FILE_DATA_PORTS: "[$HTTP_PORTS,110,143]"
FTP_PORTS: 21

选择加载的规则:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
##
## Step 2: select the rules to enable or disable
##

default-rule-path: /etc/suricata/rules
rule-files:
# - botcc.rules
- emerging-dos.rules
- emerging-exploit.rules
- emerging-ftp.rules
- emerging-activex.rules
- emerging-attack_response.rules
- emerging-imap.rules
- emerging-info.rules
- emerging-malware.rules
- emerging-misc.rules
- emerging-netbios.rules
- emerging-pop3.rules
- emerging-rpc.rules
- emerging-scan.rules
- emerging-shellcode.rules
- emerging-smtp.rules
- emerging-snmp.rules
- emerging-sql.rules
- emerging-telnet.rules
- emerging-tftp.rules
- emerging-user_agents.rules
- emerging-web_client.rules
- emerging-web_server.rules
- emerging-web_specific_apps.rules
- emerging-worm.rules
- tor.rules
# - emerging-icmp_info.rules
- emerging-icmp.rules
# botcc.portgrouped.rules
- ciarmy.rules
- compromised.rules
# - drop.rules
# - dshield.rules
# - emerging-chat.rules
# - emerging-current_events.rules
# - emerging-dns.rules
# - emerging-games.rules
# - emerging-inappropriate.rules
# - emerging-mobile_malware.rules
# - emerging-p2p.rules
# - emerging-policy.rules
# - emerging-scada.rules
# - emerging-scada_special.rules
- emerging-trojan.rules
# - emerging-voip.rules

# - decoder-events.rules # available in suricata sources under rules dir
# - stream-events.rules # available in suricata sources under rules dir
# - http-events.rules # available in suricata sources under rules dir
# - smtp-events.rules # available in suricata sources under rules dir
# - dns-events.rules # available in suricata sources under rules dir
# - tls-events.rules # available in suricata sources under rules dir
# - modbus-events.rules # available in suricata sources under rules dir
# - app-layer-events.rules # available in suricata sources under rules dir
# - dnp3-events.rules # available in suricata sources under rules dir
# - ntp-events.rules # available in suricata sources under rules dir

输出检测日志:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
##
## Step 3: select outputs to enable
##

outputs:
- eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
types:
- alert:
metadata: yes
tagged-packets: yes
xff:
enabled: yes
mode: extra-data
- http:
extended: yes
- dns:
query: yes # enable logging of DNS queries
answer: yes # enable logging of DNS answers
- tls:
extended: yes # enable this for extended logging information
- files:
force-magic: no # force logging magic on all logged files
- smtp:
extended: yes # enable this for extended logging information
- ssh
- flow

参考:https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml

logstash配置

suricata_logstash.conf,将suricata入侵检测数据采集到es中:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
input {
file {
path => ["/var/log/suricata/eve.json*"]
codec => "json"
type => "SuricataIDS"
}

}

filter {
if [type] == "SuricataIDS" {
date {
match => [ "timestamp", "ISO8601" ]
}
ruby {
code => "
if event.get('[event_type]') == 'fileinfo'
event.set('[fileinfo][type]', event.get('[fileinfo][magic]').to_s.split(',')[0])
end
"
}

ruby{
code => "
if event.get('[event_type]') == 'alert'
sp = event.get('[alert][signature]').to_s.split(' group ')
if (sp.length == 2) and /\A\d+\z/.match(sp[1])
event.set('[alert][signature]', sp[0])
end
end
"
}
}

if [src_ip] {
geoip {
source => "src_ip"
target => "geoip"
#database => "/etc/logstash/conf.d/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
if ![geoip.ip] {
if [dest_ip] {
geoip {
source => "dest_ip"
target => "geoip"
#database => "/etc/logstash/conf.d/GeoLiteCity.dat"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
}
mutate {
convert => [ "[geoip][coordinates]", "float" ]
}
}
}
}
}

output {
#stdout { codec => rubydebug }
elasticsearch {
hosts =>"elastic.*****.com:35608"
index => "suricata_log%{+YYYY.MM}"
}
}

0x05 数据分析

1)suricata数据

创建Kibana看板,进行数据分析最为简易,可以下载Kibana看板所需的json文件,并添加到Kibana中:

1
2
3
https://aka.ms/networkwatchersuricatadashboard
https://aka.ms/networkwatchersuricatavisualization
https://aka.ms/networkwatchersuricatasavedsearch

启动suricata进行网络入侵检测后,生成eve.json文件,使用ELK组件处理该文件,并在Kibana上展示告警,具体界面如下:

image

2)综合联动分析

这里的综合联动分析,就是吧目前有点数据关联起来,比如hids、waf(基于elk)、cmdb等

例如:

情景1、suricata检测到有大量扫描爆破行为,我把该事件的源IP跟cmdb数据进行关联,如果匹配上了,那么很有可能是内部机器沦陷发起了进一步的扫描攻击,事件紧急程度高,得赶紧响应!

image

情景2、suricata检测到大量针对系统层面的攻击行为,关连hids日志,若有一定的匹配,那么攻击成功的可能性比较高,需要引起安全的高度关注。

……

我这里提到的关联分析也是比较简单粗暴,主要目的是为了减少单一告警的误报率,关联分析过后,准确率会有一定提升(没数据支撑,说个卵)

经过分析之后,可以将攻击ip推送给硬件防火墙进行封禁,没过墙的业务推给服务器iptables封禁。

0x06 面临的困境(坑点)

  • Suricata规则基本依托于社区能力,没有人力来进行规则维护
  • 机房出口流量较大,多地多机房,流量镜像过来,Suricata检测引擎的性能会成为瓶颈(服务器成本不小)
  • 镜像过来的流量,可能有掉包,目前还咩想到啥好的方案
  • 告警一大堆,根本没人看(得过来)
  • 这玩意儿,搞着搞着可能就成了半成品
  • 总之,有总比没有好,至少敢根老板说,我们安全是有感知的了,呵呵