Finding the top 10 visitors by parsing the Nginx access logs

in Troubleshooting

To find the top 10 visitors of your Magento shop, you can easily use the Nginx access logging tot get these statistics. With one command on the CLI, you will be able to get many insights on your access logging.

Our Nginx access logging are generated in the JSON format. This way we can easily parse them from scripts, without having to use a special Nginx logging parser.

To show the Nginx access logging in a more human readable format, we created a little command line tool called hypernode-parse-nginx-log

 Parsing Nginx logs

  • To get started: Go to the Nginx log directory
  cd /var/log/nginx
  • Get the top 10 of today per domain
  for domain in magentotestpakket.nl devguppie.com testflip.hypernode.io; do 
    echo "Top 10 ip's for domain $domain"
    cat access.log | grep $domain | hypernode-parse-nginx-log | awk '{ print $4}' | sort | uniq -c | sort -n | tail -n 10
  done
  • Get the top 40 of yesterday
zcat access.log.1.gz | hypernode-parse-nginx-log | awk '{ print $4}' | sort | uniq -c | sort -n | tail -n 40
  • Get the top 100 of all logs present
zcat access.log* | hypernode-parse-nginx-log | awk '{ print $4}' | sort | uniq -c | sort -n | tail -n 100
  • Finding IP’s that do many POST requests
cat access.log | hypernode-parse-nginx-log | grep POST | awk '{print $4}' | sort | uniq -c | sort -n

 Write the output to a file

To redirect the output of your one-liners to a file as well to your screen, you can use tee:

cat access.log | hypernode-parse-nginx-log | grep POST | tee /tmp/some_file.txt

 

0