How to Extract Bot IPs From Nginx Access Log
Recently, I’ve noticed an aggressive uptick in bots crawling websites and applications I host. To update the firewall with undesirable IPs, you can run the following command:
sudo cat /var/log/nginx/access.log* | \
grep MauiBot | \
awk '{print $1}' | \
sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | \
uniq | \
sudo tee -a badhosts.txt
Afterwards, merge badhosts.txt with your firewall settings.