Setting up a blacklist proxy with automatic updates using Squid and SquidGuard

Posted in Howto, Linux, Proxy, Security by steelmon on December 9, 2010

The versatile, open source proxy server Squid can be used together with the plug-in SquidGuard to set up a flexible blacklist proxy server. Together with a simple cron job and a shell script, the database of blacklisted sites is kept up to date. This article describes the process step-by-step of how to get up and running.

I will be setting up the solution on an Ubuntu 9 server which conveniently has the necessary software available in its repositories. The setup should be very similar for other Linux environments, but you might have to compile the software from scratch.

Install and configure Squid

First of all, install and configure Squid. I did this in a previous post when I was looking at configuring a whitelist proxy.

# sudo apt-get install squid

Edit the Squid configuration file, /etc/squid/squid.conf and find the http_port tag. By default Squid listens to port 3128 for requests. If you want to change it, uncomment the line and change the port number.

Next, define who is allowed to access the proxy. Find the TAG: http_access heading and below it the ‘INSERT YOUR OWN RULE(S) HERE…‘ Uncomment the line :

#http_access allow localnet

You will also need to define what is meant by localnet. Find the TAG: ACL heading, and look for something like the following line:

#acl localnet src 192.168.1.0/24 192.168.2.0/24

Change the IP address and netmask above so that it matches your local network. In my case, I am on a local network with addresses ranging from 192.168.0.1 to 192.168.0.255. This means that the netmask is 255.255.255.0 – i.e. 3 bytes of “ones”, or 24 bits. So for my network it looks like this:

acl localnet src 192.168.0.0/24

Now start Squid if it’s not already running and then tell it to reload its configuration:

sudo /etc/init.d/squid start
squid -k reconfigure

You should now be able to use the proxy server from your web browser. You will not be able to get anything blocked just yet, but you should get pages served if everything was set up correctly.

Install SquidGuard

Start by installing SquidGuard using apt-get:

sudo apt-get install squidguard

Next, prepare Squid for use with SquidGuard, so once more open up /etc/squid/squid.conf in your favorite text editor.

You need to tell squid where SquidGuard is. Find the TAG: url_rewrite_program heading. There is no default setting so add a new line:

url_rewrite_program /usr/bin/squidGuard –c /etc/squid/squidGuard.conf

Prepare the blacklist database

Before going in to further configuration of SquidGuard, having access to a database of blacklisted sites and URLs is desirable.

Download the file getlists.odt, set the executable flag and rename it getlists.sh:

wget http://steelmon.files.wordpress.com/2010/12/getlists.odt
sudo mv getlists.odt  /usr/local/bin/getlists.sh
sudo chmod +x /usr/local/bin/getlists.sh

The file ending is odt rather than sh since wordpress does not allow shell scripts to be uploaded.

Now, create the database by executing the script:

sudo getlists.sh

You should now see some output from the script, and after some time of processing, you should be able to see the output by listing the contents of the blacklists database directory:

ls -l /var/lib/squidguard/db/blacklists/

Configure SquidGuard

Open the SquidGuard configuration file, /etc/squid/squidGuard.conf for edit, and replace the contents with the following:

#
# CONFIG FILE FOR SQUIDGUARD
#
dbhome /var/lib/squidguard/db/blacklists
logdir /var/log/squid
dest ads {
  domainlist ads/domains
  urllist ads/urls
} 

dest aggressive {
  domainlist aggressive/domains
  urllist aggressive/urls
}

dest drugs {
  domainlist drugs/domains
  urllist drugs/urls
}

dest hacking {
  domainlist hacking/domains
  urllist hacking/urls
}

dest porn {
  domainlist porn/domains
  urllist porn/urls
}

dest redirector {
  domainlist redirector/domains
  urllist redirector/urls
}

dest suspect {
  domainlist suspect/domains
  urllist suspect/urls
}

dest warez {
  domainlist warez/domains
  urllist warez/urls
}

dest audio-video {
  domainlist audio-video/domains
  urllist audio-video/urls
}

dest gambling {
  domainlist gambling/domains
  urllist gambling/urls
}

dest mail {
  domainlist mail/domains
}

dest proxy {
  domainlist proxy/domains
  urllist proxy/urls
}

dest spyware {
  domainlist spyware/domains
  urllist spyware/urls
}

dest violence {
  domainlist violence/domains
  urllist violence/urls
}

acl {
  default {
    pass !ads !aggressive !drugs !hacking !porn !redirector !suspect !warez !audio-video !gambling !mail !proxy !spyware !violence all
    redirect http://www.x509.se/block.html
  }
}

Among the last lines, there is a URL to a page that gets served whenever there is blocked content. You should change the URL to your own block page (unless your happy with my extremely sparse one in Swedish) .

Compile the SquidGuard database. This may take a while to complete:

sudo squidGuard –C all

Start Squid, which in turn will start SquidGuard, and reconfigure

sudo /etc/init.d/squid start
sudo squid -k reconfigure

Troubleshooting

If you are having problems, most likely it’s related to permissions. You can get some useful information by running SquidGuard from the command line:

sudo su – proxy
echo "http://www.ubuntu.com {client ip address}/ - - GET" | squidGuard -d -c /etc/squid/squidGuard.conf

You can change the URL to whatever you’d like to test for access or denial. The IP address is the address of the computer you want to simulate as surfing the net from.

If you encounter any problems with permissions, you may try the following:

sudo chown proxy:proxy /etc/squid/squidGuard.conf
sudo chown -R proxy:proxy /var/lib/squidguard/db
sudo chown -R proxy:proxy /var/log/squid/
chmod 644 /etc/squid/squidGuard.conf
chmod -R 640 /var/lib/squidguard/db
chmod -R 644 /var/log/squid/
find /var/lib/squidguard/db -type d -exec chmod 755 \{\} \; -print
chmod 755 /var/log/squid

There are more detailed trouble shooting available in the reference section.

Automating the blacklist updates

When everything is up and running, you may want to automate the update procedure. This is easily accomplished by setting up a cron job. Open the cron table in interactive mode:

Steelmon's tech stuff