A website is unreachable, and the client needs to understand why and what to do.
Edit me

DDoS Attack Identification Using Web Log Analysis

How to verify if a website is under DDoS attack

Problem

A website is suspected to be under DDoS attack, and the client needs to mitigate the effects of the attack.

Solution

Make sure to check Article #171: Website Down before proceeding with the following steps.

NOTE: The following instructions are mainly aimed at how to use web log analysis to detect DDoS attacks which occurred on the application layer. If you do not find evidence of a DDoS attack through the following process, but still suspect that a DDoS attack occurred, it’s recommended that you direct the client to contact their hosting provider for further assistance. For more information on attacks on other layers, check the Comments section at the end of this article.

Guide Questions

  • What is the error the client is encountering?
  • How long has the website been behaving this way?
  • Is the website loading intermittently?
  • Is it not loading at all?
  • Is the web host accessible?
  • Have they experienced this issue before?
  • Did the client receive a notification from their web hosting provider? What could be the problem?
  • Can the client extract or request the traffic logs?
  • What is the bandwidth capacity of their website? What is the average traffic they receive everyday?
  • Is the website already using DDoS protection services (e.g. Cloudflare, Deflect, etc.)?
  • What could be the motivation of the attackers for taking down their website?

Analyzing Traffic Logs

  1. Request the traffic logs from the client. It should capture at least one month of logs including the date of attacks. The steps below are applicable for GoDaddy hosted websites:

    a. Login to the account.

    b. Click ‘Web Hosting.’

    c. Next to the account, click ‘Manage.’

    d. From the Files & FTP menu, select ‘FTP File Manager.’

    e. From the directory structure that displays, click ‘Apache Logs.’

  2. Get useful information from the traffic logs:

    a. Top 5 IP addresses source:

    awk '{ print $1}' traffic.log | sort | uniq -c | sort -nr | head -n 5

    b. Top 5 User-Agents used:

    awk -F\" '{print $6}' traffic.log | sort | uniq -c | sort -nr | head -n 5

    c. Top 5 website directories being requested:

    awk -F\" '{print $2}' traffic.log | sort | uniq -c | sort -nr | head -n 5

    d. Top 5 points in time with largest traffic per minute:

    awk '{ print $4}' traffic.log | sort | uniq -c | sort -nr | head -n 5

    e. Top 5 HTTP Referrers:

    awk -F\" '{print $4}' traffic.log | sort | uniq -c | sort -nr | head -n 5

    f. Top 5 HTTP codes responded by the server:

    awk '{ print $9}' traffic.log | sort | uniq -c | sort -nr | head -n 5

    Note: Top results could be changed from 5 to 2,3,4,6, etc. depending which might provide better understanding of the logs. This can be done by changing the numeric value after head -n. Also note that the above commands are applicable for Apache logs.

    g. Get the average traffic per day:

    # Excluding the date of attack indicated by ``'DD/Mmm'`` (example: 15/May):
awk '{print $4}' traffic.log | cut -d: -f1 | uniq -c | grep -v 'DD/Mmm' | awk '{s+=$1} END {print s/NR}'

# Only the date dates of attack, the dates are indicated after -e flag
awk '{print $4}' traffic.log | cut -d: -f1 | uniq -c | grep -e 'DD/Mmm' -e 'DD/Mmm'

    If the client is hosted by GoDaddy, their traffic statistics can be check through here

    h. Get the total traffic in kilobytes for the duration of the logs given:

    awk '{totalkb += $10} END {printf ("%9.2f Kb\n", totalkb/1024);}' traffic.log

  3. Interpret the results from Step 2:

    a. Check the information about the IP addresses: owner, location, if blacklisted, etc.

    b. Check what are the User-Agent being used: legitimate visitors, crawlers, bots, etc.

    c. Check the directories being requested: specific website page, random characters, unavailable page, etc.

    d. Check if the date with the largest traffic per minute’s is the date of the suspected attack, and compare it with the average traffic they have on normal days.

    e. Check if the referrers are just coming from social media websites, as the articles could be shared on social media platforms. IP sources should also be checked in this way. This may appear to be requested by one individual, but in reality it is coming from a link shared on a social media platform and clicked by several followers.

    f. Check if the codes sent in response by the server indicate resource unavailability, unauthorized access, server error, etc.

    g. Check if the traffic per day during normal days and the dates of attack are different.

    h. Check if the total traffic for the duration of logs is within the bandwidth of their hosting plan. It could cause a problem with the availability of the website once the traffic they are catering is larger than their bandwidth limit. Bandwidth limits for websites hosted with GoDaddy can be checked using the instructions here

  4. Proceed in helping the client set-up DDoS protection service if necessary:
  5. If they already have DDoS protection installed, it is recommended to reach out to Qurium. If the client agrees to proceed with the referral to Qurium, send an email to [email protected] containing:
    • Analysis of the traffic logs
    • Traffic logs: back-end server, and from DDoS protection

Comments

Analyzing web logs is only helpful in detecting application layer DDoS attacks (also known as ‘level 7 attacks’). These attacks use the application layer protocols, including HTTP and FTP, to deliver the attack. Because Apache logs record these attacks, we’re able to see evidence of the attacks by analyzing the web logs.

If you suspect an attack may have occurred on another layer, then you may want to recommend that the client seek assistance from their hosting provider, as the client themselves likely will not have access to the information necessary to detect attacks on other layers.

Examples of DDoS attacks on other layers:

  • Transport layer (layer 4): SYN flood attack
  • Network layer (layer 3): CMP flooding

Links for further reading:

Tags: ddos_attack articles faq