Ansible Conditionals and Parentheses evaluate to True
I had fun wasting hours working out how to do correct ‘when’ statements in Ansible - In end up consulting #ansible on IRC to get the answers.
Anyway I hope the following playbook makes sense to you. Note that ‘admintool’ is a valid group in my situation.
- name: Debug all the things hosts: all tasks: - set_fact: renew_cert="renew" # Valid - Should pause - name: Test 0 PASS pause: prompt="Test" seconds=1 when: '"admintool" in group_names and renew_cert == "renew"' # Valid - Should skip - name: Test 1 SKIP pause: prompt="Test" seconds=1 when: '"I-Dont-Exist" in group_names and renew_cert == "renew"' # Valid - Should skip - name: Test 2 SKIP pause: prompt="Test" seconds=1 when: - "'i-dont-exist' in group_names" - renew_cert == "renew" # Valid - Should pause - name: Test 3 PASS pause: prompt="Test" seconds=1 when: - "'admintool' in group_names" - renew_cert == "renew" # Invalid - Should skip - but eval's True - DONT USE - name: Test 4 SKIP pause: prompt="Test" seconds=1 when: ("'admintool' in group_names" and renew_cert == "renew") # Invalid - Should skip - but eval's True - DONT USE - name: Test 5 SKIP pause: prompt="Test" seconds=1 when: ("'I-dont-exist' in group_names") # Valid - Should pause - name: Test 6 PASS pause: prompt="Test" seconds=1 when: ("admintool" in group_names and renew_cert == "renew") # Valid - Should skip - name: Test 7 SKIP pause: prompt="Test" seconds=1 when: ("I-dont-exist" in group_names and renew_cert == "renew")
List comparison and list manipulation in Ansible
I keep saying time and time again that Ansible is not a programming language, it’s similar to one, it can do some programming things but ultimately it’s messy and I hate it BUT I can make it do some strange things.
List manipulation being one of those.
In this example I have two directories that I want to compare, directory one (/tmp/1) and directory two (/tmp/2). Directory one is the Source, that I want directory two to look like.
The use case is I want to sync /tmp/1 to /tmp/2 but you only want to remove the files in that are no longer /tmp/1, then you can sync (copy/template) the /tmp/1 directory knowing that nothing exists /tmp/2 that shouldn’t be there.
The ansible code is this with debug statements:
- hosts: local become: false tasks: - name: find 1 find: path=/tmp/1 register: one - debug: msg="{{ one }}" - name: find 2 find: path=/tmp/2 register: two - debug: msg="{{ item.path }}" with_items: - "{{ two.files }}" - set_fact: one_list: [] two_list: [] new_list: [] - name: append set_fact: one_list="{{ one_list }} + [ '{{ item.path | basename }}' ]" with_items: - "{{ one.files }}" - name: append set_fact: two_list="{{ two_list }} + [ '{{ item.path | basename }}' ]" with_items: - "{{ two.files }}" - debug: msg="{{ one_list }}" - debug: msg="{{ two_list }}" - set_fact: new_list="{{ two_list | difference(one_list) }}" - debug: msg="{{ new_list }}"
The final result is new_list is a list (array) that contains what needs to be removed from /tmp/2 to bring it in line with /tmp/1
Docker and IPtables Firewall Merger
The problem: Modifying firewall rules on a host that runs Docker or Rancher (cattle) causes the docker-bridges and rancher NAT rules to be blown away, causing all your containers networking to break.
The solution: Modify /etc/sysconfig/iptables as normal and instead of running iptables-restore /etc/sysconfig/iptables run as root: dockerFirewallMerge.py
I’d appreciate some constructive feedback! https://github.com/c … /DockerFirewallMerge
Squid HTTPS interception and filtering without client certificates
I had a requirement to filter (all) web traffic on a few servers. This is typically easy with Squid and using it’s transparent proxy function. Where it gets difficult is filtering domains for HTTPS traffic.
I don’t want to SSL intercept the traffic, I don’t want to install CA certificates on the clients, I only want to filter the URLs based on a whitelist to which it can access. This is how it is done:
yum install squid # I used squid 3.5.20 /usr/lib64/squid/ssl_crtd -c -s /var/lib/ssl_db chown -R squid.squid /var/lib/ssl_db mkdir /etc/squid/ssl_cert/ chown -R squid.squid /etc/squid/ssl_cert/ cd /etc/squid/ssl_cert openssl req -new -newkey rsa:1024 -days 1365 -nodes -x509 -keyout myca.pem -out myca.pem echo "www.google.com" > /etc/squid/whitelist chmod 640 /etc/squid/whitelist chown root:squid /etc/squid/whitelist
/etc/squid/squid.conf:
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network acl localnet src 127.0.0.1/32 # RFC1918 possible internal network acl localnet src 172.16.0.0/12 # RFC1918 possible internal network acl localnet src 192.168.0.0/16 # RFC1918 possible internal network acl localnet src fc00::/7 # RFC 4193 local private network range acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager acl step1 at_step SslBump1 acl whitelist_ssl ssl::server_name "/etc/squid/whitelist" acl whitelist dstdomain "/etc/squid/whitelist" acl port_80 port 80 acl http proto http ssl_bump peek step1 ssl_bump splice whitelist_ssl ssl_bump terminate all !whitelist_ssl http_access deny http port_80 localnet !whitelist http_access allow localnet http_access deny all https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myca.pem key=/etc/squid/ssl_cert/myca.pem http_port 3128 transparent coredump_dir /var/spool/squid refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 20% 4320
# Test it with:
iptables -m owner --uid-owner cm -t nat -A OUTPUT -p tcp --dport 80 -j DNAT --to 127.0.0.1:3128 iptables -m owner --uid-owner cm -t nat -A OUTPUT -p tcp --dport 443 -j DNAT --to 127.0.0.1:3127
# Closing notes and thoughts
Around this section here: http_access deny http port_80 localnet !whitelist http_access allow localnet http_access deny all
It looks a bit funny because we ‘allow localnet’ which typically allows our clients open access. However assessing:
ssl_bump terminate all !whitelist_ssl http_access deny http port_80 localnet !whitelist
rules first, you see that we filter out all sites other than the whitelist with an explicit ‘deny’ or ssl ‘terminate’.
Also trying to use a proxy-aware application with the above configuration will not work because the proxy is configured in transparent / intercept mode ONLY. This is likely due to not having a normal http_port directive, this is good for me as it’s minimizing the abuse avenues.
Also for a final, final step, you need to configure your edge (or local) firewall to do destination NAT’ing back to the two Squid ports.
Block network traffic based on UID / User and GID / Group
I just found out that you can apply different IPTables rules based on UID and GID.
Just check that your kernel / iptables supports the module:
iptables -m owner --help
Which should output near the bottom like:
owner match options: [!] --uid-owner userid[-userid] Match local UID [!] --gid-owner groupid[-groupid] Match local GID [!] --socket-exists Match if socket exists
Then make a rule as required. Eg. User ‘cm’ gets their web traffic transparently proxied via Squid.
iptables -m owner --uid-owner cm -t nat -A OUTPUT -i eth0 -p tcp --dport 80 -j DNAT --to 127.0.0.1:3128
Pretty cool!