IT Dribble

I had the fun times of trying to write grok rules in a particular way along with a complicated pipeline. I got tried of pushing the rules and restarting logstash, there had to be a better way!

This is want I ended up doing on my development system:

wget https://artifacts.elastic.co/downloads/logstash/logstash-6.3.1.rpm
yum localinstall logstash-6.3.1.rpm 

Create your pipeline in: /etc/logstash/conf.d/

Create the following example files:

/tmp/input.txt:

2018-07-16T01:53:28.716258+00:00 acme-host1 sshd[12522]: Disconnected from 8.8.8.8 port 37972

000-file-in.conf:

input {
    file {
	path => [ "/tmp/input.txt" ]
	start_position => beginning
	type => "test"
	add_field => { "sourcetype" => "test" }
	sincedb_path => "/dev/null"
    }
}

25-filter.conf:

filter {
    if [type] == "test" {
        grok {
            match => { "message" => "%{TIMESTAMP_ISO8601} %{SYSLOGHOST:logsource} %{SYSLOGPROG}?: %{GREEDYDATA:message}" }
            overwrite => [ "message" ]
            add_tag => [ "p25vls" ]
        }
    
        date {
            locale => "en"
            match => [ "timestamp", "MMM dd HH:mm:ss", "MMM  d HH:mm:ss"  ]
            timezone => "UTC"
        }
    }
}

999-output.conf:

output {
    stdout { codec => rubydebug }
}

Run:

/usr/share/logstash/bin/logstash -r -f /etc/logstash/conf.d/

Give it a minute, because well Java

Now in a second window, modify you pipeline (or file 25-filter.conf etc), save it.

You should see Logstash reprocess the data from ‘/tmp/input.txt’

Happy iterational development :-)

In order for a Linux (client) box to communicate with (and perform NTLM auth) a Windows domain controller through a restrictive firewall you would need the following ports opened at a minimum:

udp 53
tcp/udp 88
tcp/udp 135
tcp 139
tcp 389
tcp 445
tcp/udp 464

Upgrading a server from Debian 8 to Debian 9 - I noticed in /etc/ssh/sshd_config that ‘PermitRootLogin’ had the argument ‘prohibit-password’. Having not seen that before I wondered what the difference was between that and ‘without-password’.
Turns out that mean and do the same thing - but ‘prohibit-password’ was introduced to be less ambigous. So there you have it!

Check out the release notes here for proof :-)

openssl s_client -servername NAME -connect HOST:PORT 2>/dev/null | openssl x509 -noout -dates

If you work in the ISP space you might need to check if a down or upstream server is set up to allow Zone Transfers (AXFR).

Test via:

dig -b your-dns-server-ip-with-permission-address @their-dns-server-ip-address exampleDomain.com AXFR
eg. dig -b 8.8.8.8 @208.67.222.222 exampleDomain.com AXFR

And it should return some records about the zone!