A while ago I switched from NXLog to Winlogbeat. One of the things I was missing: Plain ASC File parsing to the appender!
With NXLog it was easy to parse plain log files and ship them towards my Graylog Server!
Luckily, my mate @Mokkujin asked: “Why not use Filebeat? I use that all the time when I need to parse and ship plain ASC logs?”
Elastic promotes Filebeat with one simple sentence:
Aggregate, “ tail -f ” & search
And (at least in my opinion) this is true!
Here is my approach to ship Windows Defender Firewall Logs with Filebeat:
filebeat.inputs:
- type: log
paths:
- C:\Windows\System32\LogFiles\Firewall\pfirewall.log
exclude_lines: ['\s\bff02::.*?\s\bRECEIVE\b', '\s\b5353\b\s\b5353\b\s.*?\s\bRECEIVE\b', '\s\b(80|443)\b\s.*?\s\bSEND\b','\s\bDROP\b\s\b(TCP|UDP|ICMP)\b\s\b(10\.16\.1\.1|10\.16\.1\.128)\b\s.*?\s\bRECEIVE\b']
encoding: utf-8
close_inactive: 1m
ignore_older: 2m
clean_inactive: 24h
processors:
- drop_event:
when:
regexp:
message: ['\s\bff02::.*?\s\bRECEIVE\b']
processors:
- drop_event:
when:
regexp:
message: ['\s\b5353\b\s\b5353\b\s.*?\s\bRECEIVE\b']
processors:
- drop_event:
when:
regexp:
message: ['\s\b(80|443)\b\s.*?\s\bSEND\b']
processors:
- drop_event:
when:
regexp:
message: ['\s\bDROP\b\s\b(TCP|UDP|ICMP)\b\s\b(10\.16\.1\.1|10\.16\.1\.128)\b\s.*?\s\bRECEIVE\b']
output.logstash:
hosts: ["XXX.XXX.XXX.XXX:XXXX"]
Like with Winlogbeat, please mind the following:
- Replace
"XXX.XXX.XXX.XXX:XXXX"
with you logging host, with port! - The config is based on YAML! Mind the spaces! Use a good editor that supports YAML checks (I recommend VSCode)
- Use the config file from the dedicated GitHub Repository. The config above is more meant as a showcase for this article!
- You can parse any other plain ASC log file. Just add it and build your filters!
Yes, you need a second tool (Filebeat), but that brings a lot of possibilities and the two functions (Event Log shipping and shipping plain ASC logs) are now separated.