Thursday, June 18, 2009

Squid - WCCP and Cisco ASA setup

Setup squid first:

Then on the box (in this case LINUX) running squid do the following:

modprobe ip_gre
iptunnel add gre1 mode gre remote $ASA_ROUTER_IDENTIFIER local $SQUID_SERVER_IP dev eth0
ifconfig gre1 inet 1.2.3.4 netmask 255.255.255.0 up


Updated to reflect great comment from DanO'Reilly :)

The ASA_ROUTER_IDENTIFIER is first of all just identifier and its the highest ip that is on your ASA. Please use the highest IP on your ASA for the iptunnel command. You can get list of all the ips on your asa by doing "show ip".

Next....

Enable forwarding on squid server:


echo 1 > /proc/sys/net/ipv4/ip_forward


Disable reverse path filter.


echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/gre1/rp_filter



Add IPTABLES rule to redirect all the traffic coming on port 80 over to the squid listening port. i.e: 3128 in this case:


iptables -t nat -A PREROUTING -i gre1 -p tcp -m tcp --dport 80 -j REDIRECT --to-port 3128


Now to configure squid.conf:

Make sure you compiled squid with transparency support (required for PF , IPFW etc). Do config --help | grep transp to see all the firewalls that need this option. For Cisco ASA etc, you don't need to have any transparent option.

anyways back to squid.conf add the following line:


http_port 3128 transparent


"transparent" is the key word here.

Now to configure WCCPv2 in squid.conf:


wccp2_router <$ASA_IP_ADDRESS>
wccp_version 4
wccp2_forwarding_method 1
wccp2_return_method 1




Enabling/configuring WCCPv2 on CISCO ASA(7.x/8.x):

I will go over two scenarios that I came across. First for testing I only wanted certain IPs to hit the squid box before opening it to the rest of the world. Additionaly I didn't want ALL THE PROTOCOLS to go through the proxy server. Only http (80) to go through squid: so here is what i created:

1. First created acl for the clients I wanted to allow access through squid for port 80

ASA# access-list proxyclients extended permit tcp host 192.168.100.202 any eq www

2. I added acl for my proxy servers too.

ASA# access-list proxyservers extended permit ip host 192.168.100.3 any

*** 192.168.100.3 - IP address of my squid box.

3. Now the wccp part:

ASA# wccp web-cache redirect-list proxyclients group-list proxyservers
ASA# wccp interface inside web-cache redirect in

4. Additionaly you can issue:

ASA# wccp web-cache


This should get you going :)

Now if you wanted to let EVERYONE in the world go through and go through proxy: I would remove all the ACLs and remove one wccp line:


ASA# no access-list proxyclients extended permit tcp host 192.168.100.202 any eq www
ASA# no access-list proxyservers extended permit ip host 192.168.100.3 any

This is tricky. You have to "no" the wccp interface command first so...


ASA# no wccp interface inside web-cache redirect in
ASA# no wccp web-cache redirect-list proxyclients group-list proxyservers

Now lets add the wccp interface command back


ASA# wccp web-cache
ASA# wccp interface inside web-cache redirect in


At this point all your traffic hitting the firewall would be redirected to the squid server.

** note about https transparency: packets are encrypted ..so by the time packets arrive at SQUID ..it cannot understand. So other words https transparency, I don't think its supported.

As superpaia pointed out in his comments:

important: "The only topology that the security appliance supports is when client and cache engine are behind the same interface of the security appliance and the cache engine can directly communicate with the client without going through the security appliance." from Cisco.

24 comments:

Unknown said...

That was a great write up. Thanks!

Anonymous said...

Hi,

so how did you tell wccp in your second way (for all users on the network) to redirect the traffic to the squid server?

I don't see the group proxysevers mentioned in the 2 lines for wccp...

thx

Parvinder Bhasin said...

In the second method , provided you only have ONE proxy server, you don't need the proxyservers and proxyclients acl. You just need the 2 wccp commands on your firewall and the proxy server setup.

Hope this helps.

Another thing you can do is simple on the first method if you wanted to allow everyone , just change the proxyclients ACL to:

ASA# access-list proxyclients extended permit tcp host any any eq www


Leave the server ACL configured with your proxy server.

Hope this helps.

Unknown said...

I have followed this article and many other similar ones, but for the life of me I cannot get this working.

WCCP on the cisco sees the Squid server and appears to be sending requests accross, however if I do a tcpdump on the gre1 interface I see no traffic, so I believe the tunnel is not set up correctly.

Any help would be greatly appreciated.

Parvinder Bhasin said...

Noel,

Without actually looking at your config, I cannot be sure but somethings I would mention are that make sure you have configured SQUID properly for TRANSPARENT proxying.

http_port 3128 transparent.
Make sure you match the config mentioned in my blog for the router/firewall.

The fact that you see nothing on tcpdump and yet you see the tunnel on the firewall/router, could be that your ACLs are not setup correctly ..maybe?

But again without your actual config I cannot be sure. Feel free to contact me if you need further help. Would be glad to help you out :)

-Parvinder Bhasin

Unknown said...

Hi Pravider,

Thanks very much for getting back to me. I will send you through all the details.

output of iptunnel:
gre1: gre/ip remote 10.200.237.254 local 10.200.237.21 dev eth0 ttl inherit

output of ifconfig:
eth0 Link encap:Ethernet HWaddr 3E:A4:7F:CF:3D:DA
inet addr:10.200.237.21 Bcast:10.200.237.255 Mask:255.255.255.0
inet6 addr: fe80::3ca4:7fff:fecf:3dda/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1495858 errors:0 dropped:0 overruns:0 frame:0
TX packets:23562 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:332220029 (316.8 MiB) TX bytes:2804884 (2.6 MiB)

gre1 Link encap:UNSPEC HWaddr 0A-C8-ED-15-80-BF-68-E0-00-00-00-00-00-00-00-00
inet addr:192.168.172.2 P-t-P:192.168.172.2 Mask:255.255.255.252
UP POINTOPOINT RUNNING NOARP MTU:1476 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)

I have compiled squid right as far as I can tell with the appropriate modules.

output squid -v
/usr/local/squid/sbin/squid -v
Squid Cache: Version 2.6.STABLE17
configure options: '--enable-smartfilter' '--enable-async-io' '--enable-linux-netfilter' '--enable-underscores' '--prefix=/usr/local/squid' '--enable-auth=ntlm,basic' '--enable-external-acl-helpers=wbinfo_group' '--enable-delay-pools' '--enable-wccpv2' '--enable-wccp' '--with-samba-sources=/usr/src/samba-3.4.0/'

iptables:
iptables -L
Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT tcp -- 10.200.237.0/24 anywhere tcp dpt:squid
ACCEPT gre -- 10.200.237.254 10.200.237.21
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
LOG all -- anywhere anywhere LOG level warning

iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
REDIRECT tcp -- anywhere anywhere tcp dpt:http redir ports 3128

My config for the firewall is pretty much a copy of yours, apart from the IP addresses.

Parvinder Bhasin said...

Hi Noel,

Can you provide output of

sh wccp web-cache detail

And

sh wccp web-cache

Also just for troubleshooting purposes..can you change your default Iptables policy to ACCEPT.
Those would be the starting points.

Additionally can you provide your firewall side of config too (just for wccp).

-Parvinder Bhasin

Unknown said...

Hi Pravinder,

Please find output below.
I believe my default policy for IPTABLES is accept. below is the output of /etc/sysconfig/iptables

# Generated by iptables-save v1.3.5 on Mon Feb 22 15:14:07 2010
*nat
:PREROUTING ACCEPT [9:679]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A PREROUTING -i gre1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
COMMIT
# Completed on Mon Feb 22 15:14:07 2010
# Generated by iptables-save v1.3.5 on Mon Feb 22 15:14:07 2010
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [1:136]
-A INPUT -s 10.200.237.0/255.255.255.0 -p tcp -m tcp --dport 3128 -j ACCEPT
-A INPUT -s 10.200.237.254 -d 10.200.237.21 -p gre -j ACCEPT
-A INPUT -i eth0 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i gre1 -j ACCEPT
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -j LOG
COMMIT
# Completed on Mon Feb 22 15:14:07 2010


show wccp web-cache detail

WCCP Cache-Engine information:
Web Cache ID: 10.200.237.21
Protocol Version: 2.0
State: Usable
Initial Hash Info: 00000000000000000000000000000000
00000000000000000000000000000000
Assigned Hash Info: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF
Hash Allotment: 256 (100.00%)
Packets Redirected: 2259
Connect Time: 1d22h

sh wccp web-cache

Global WCCP information:
Router information:
Router Identifier: external
Protocol Version: 2.0

Service Identifier: web-cache
Number of Cache Engines: 1
Number of routers: 1
Total Packets Redirected: 2259
Redirect access-list: proxyclients
Total Connections Denied Redirect: 0
Total Packets Unassigned: 2
Group access-list: proxyservers
Total Messages Denied to Group: 0
Total Authentication failures: 0
Total Bypassed Packets Received: 0

show run wccp
wccp web-cache redirect-list proxyclients group-list proxyservers
wccp interface inside web-cache redirect in
access-list proxyclients extended permit tcp host 10.200.237.24 any eq www
access-list proxyservers extended permit ip host 10.200.237.21 any

Parvinder Bhasin said...

Noel,

Your output looks like you've got the intercept at your firewall going.

Are you seeing hitcounts on your ACLs in the ASA??

sh access-list proxyclients

If you are ..then you are good on the firewall side. Which I think you already are. I think problem is with some iptables rules.

Try this ..if you can , replace your /etc/sysconfig/iptables rules with:

[root@squidnclamav ~]# cat /etc/sysconfig/iptables
# Generated by iptables-save v1.3.5 on Thu Jul 9 13:06:36 2009
*nat
:PREROUTING ACCEPT [3:926]
:POSTROUTING ACCEPT [7:1048]
:OUTPUT ACCEPT [7:1048]
-A PREROUTING -i gre1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
COMMIT
# Completed on Thu Jul 9 13:06:36 2009

Unknown said...

Hi Pravider,

Thanks for you help I really apprecaite it.

The access list is getting hits:
sh access-list proxyclients
access-list proxyclients; 1 elements
access-list proxyclients line 1 extended permit tcp host 10.200.237.24 any eq www (hitcnt=4390) 0x28bbd853

The problem is when I do a tcpdump on the gre1 interface I get no traffic, even if I ssh to that gre1 ip address locally.

If I do a tcpdum on the lo interface when I ssh to the IP of the gre1 interface locally that is when I get traffic coming up.

Parvinder Bhasin said...

No Problems Noel! You are welcome.

So you are getting hit counts on the ACLs. This means everything is good on the firewall side. It is forwarding traffic over to your proxy server. Just in case do a "sh access-list proxyserver" and make sure you see hitcounts.

Also did you try doing a tcpdump on the the physical interface itself? Do you see the packets??? You SHOULD.

Can you cat the following:

cat /proc/sys/net/ipv4/ip_forward
cat /proc/sys/net/ipv4/conf/default/rp_filter
cat /proc/sys/net/ipv4/conf/eth0/rp_filter
*** or the physical nic that ASA communicates on.
cat /proc/sys/net/ipv4/conf/gre1/rp_filter

one last very important thing. What is the IP address of you firewall itself. What IP does your firewall communicate on to the squid server.

** Make sure both the firewall and your proxy server are on the SAME network.

Lastly try a dummy ip (1.2.3.4) for your GRE1. ASAs don't actually create a GRE tunnel. In this case its just mode of transport for WCCP.

so do the following:

modprobe ip_gre
iptunnel add gre1 mode gre remote $ASA_IP_ADDRESS local $SQUID_SERVER_IP dev eth0
ifconfig gre1 inet 1.2.3.4 netmask 255.255.255.0 up

-Parvinder Bhasin

Unknown said...

Hi Parvinder,

Please see what I have done in relation to your comments below:

Getting hits on the server access list:
show access-list proxyservers
access-list proxyservers; 1 elements
access-list proxyservers line 1 extended permit ip host 10.200.237.21 any (hitcnt=38297)

added the tunnel and greIP as you suggested:
modprobe ip_gre
iptunnel add gre1 mode gre remote 10.200.237.254 local 10.200.237.21 dev eth0
ifconfig gre1 inet 1.2.3.4 netmask 255.255.255.0 up

am getting traffic on the eth0 interface of squid when trying to browse from a client. Although I see the IP address coming from is the ASA outsie or public rather than the internal as I would expect

[root@AHM-SQUID ~]# tcpdump -nn -i eth0 |grep gre-proto
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
12:29:41.623659 IP PUBLIC IP ASA.82 > 10.200.237.21: GREv0, length 60: gre-proto-0x883e
12:29:44.603944 IP PUBLIC IP ASA.82 > 10.200.237.21: GREv0, length 60: gre-proto-0x883e
12:29:50.610363 IP PUBLIC IP ASA.82 > 10.200.237.21: GREv0, length 56: gre-proto-0x883e

Output pof /proc/sys/net as requested

[root@AHM-SQUID ~]# cat /proc/sys/net/ipv4/ip_forward
1
[root@AHM-SQUID ~]# cat /proc/sys/net/ipv4/conf/default/rp_filter
0
[root@AHM-SQUID ~]# cat /proc/sys/net/ipv4/conf/eth0/rp_filter
0
[root@AHM-SQUID ~]# cat /proc/sys/net/ipv4/conf/gre1/rp_filter
0

The firewall internal IP is 10.200.237.254, squid is 10.200.237.21, I think I have configured the ACLS and WCCP on the fire wall correctl:

AHM-ASA(config)# show running-config wccp
wccp web-cache redirect-list proxyclients group-list proxyservers
wccp interface inside (10.200.237.254) web-cache redirect in

access-list proxyclients extended permit tcp host 10.200.237.24 (client machine) any eq www
access-list proxyservers extended permit ip host 10.200.237.21 (squid server) any

Parvinder Bhasin said...

Noel,

based on traffic you are seeing on you SQUID machine's eth0 interface. It tells me there is some issue with your ASA configuration or some routing issue. Can't say for sure. But overall configuration of WCCP and Firewall side of WCCP config looks ok.

-Parvinder Bhasin

Dan O'Reilly said...

There is one thing in all this that through me for a loop and I thought I would share it in case anyone else came across it.

this command
iptunnel add gre1 mode gre remote $ASA_IP_ADDRESS local $SQUID_SERVER_IP dev eth0

should read
iptunnel add gre1 mode gre remote $ASA_ROUTER_IDENTIFIER local $SQUID_SERVER_IP dev eth0

By default cisco wccp will use the highest ip on any of its interfaces for its router identifier. So in most cases by default you have a managment interface with the ip 192.168.1.1 which ends up being higher than the 10.x.x.x numbers many of us use on the other interfaces.
I could not wrap my head around the fact that I would be using that ip, because their was nothing on that interface at all. But I used that ip and everything worked like a charm.

Otherwise wccp would appear to connect but nothing would go through except a couple introduction packets.

use
sh wccp
on your asa to see your router identifier.

Great how to everything worked great otherwise.

Unknown said...

Hi Parvinder,

So when doing a tcpdum on the eth0 interface should I be seeing the below with the inside IP of the ASA rather than the public one?

[root@AHM-SQUID ~]# tcpdump -nn -i eth0 |grep gre-proto
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
10:44:55.511982 IP PUBLIC.IP.82 > 10.200.237.21: GREv0, length 60: gre-proto-0x883e

so something like:

10:44:55.511982 IP 10.200.237.254 > 10.200.237.21: GREv0, length 60: gre-proto-0x883e

The ASA config appears standard the access list for inside traffic.

access-list inside_access_in extended permit ip any any
access-group inside_access_in in interface inside

and one static route
route outside 0.0.0.0 0.0.0.0 PUBLIC.IP.81 1

Unknown said...

Ok thanks Guys!!!
This is now working.

using the below fiexed my problem.
should read
iptunnel add gre1 mode gre remote $ASA_ROUTER_IDENTIFIER local $SQUID_SERVER_IP dev eth0

I have another question which is slightly off topic. The whole point of getting wccp working with squid, was to allow us to have browsing working if squid should go down and to also not have to configure a Gp for proxy settings. Our squid is configured with ntlm authentication, however the user information does not come accross when using wccp. Do any of you know if this will work.

Thanks for all you help!
Noel

Parvinder Bhasin said...

Thanks!!!

Unknown said...

Great write up and very helpful comments but I'm still having a problem with my ASA 5505 and the router identifier. My problem is I have 2 outside ips from different isp (a primary cable connection, and a backup T1). The router identifier is the T1 ip (bbb.bbb.bbb.bbb) instead of the cable ip (aaa.aaa.aaa.aaa).

I'm running squid 2.6 on Ubuntu 8.04. Here are my are my configurations:

ASA 5505 (192.168.0.1 inside ip)
T1 (bbb.bbb.bbb.bbb)
Cable (aaa.aaa.aaa.aaa)
My PC (192.168.0.10)
Proxy (192.168.0.12)

squid.conf
http_port 3128 transparent
wccp2_router 192.168.0.1
wccp_version 4
wccp2_rebuild_wait on
wccp2_forwarding_method 1
wccp2_return_method 1
wccp2_service standard 0

rc.local
ip tunnel add wccp0 mode gre remote aaa.aaa.aaa.aaa local 192.168.0.12 dev eth0
ifconfig wccp0 inet 127.0.0.2 netmask 255.255.255.0 up
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/default/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter
echo 0 > /proc/sys/net/ipv4/conf/wccp0/rp_filter
iptables -t nat -A PREROUTING -i wccp0 -p tcp -m tcp --dport 80 -j REDIRECT --to-port 3128

asa 5505 (I did these commands)
ASA# access-list proxyclients extended permit tcp host 192.168.0.10 any eq www
ASA# access-list proxyservers extended permit ip host 192.168.0.12 any
ASA# wccp web-cache redirect-list proxyclients group-list proxyservers
ASA# wccp interface inside web-cache redirect in
ASA# wccp web-cache

But when I run sh wccp, I get this:
Global WCCP information:
Router information:
Router Identifier: bbb.bbb.bbb.bbb
Protocol Version: 2.0

Service Identifier: web-cache
Number of Cache Engines: 1
Number of routers: 1
Total Packets Redirected: 75
Redirect access-list: proxyclients
Total Connections Denied Redirect: 0
Total Packets Unassigned: 0
Group access-list: proxyservers
Total Messages Denied to Group: 0
Total Authentication failures: 0
Total Bypassed Packets Received: 0

Parvinder Bhasin said...

Rudie,

Thanks for the comments. I am not sure I understand your problem. One thing that always come in handy to troubleshoot is seeing if you are getting hitcounts on your ACLS (ASA).

But then again I am not sure I understand the problem.

-Parvinder Bhasin

Unknown said...

After some hours of trying, we change the PIX/ASA IP in the iptunnel statement from the INSIDE interface to the IP from the OUTSIDE:

iptunnel add wccp0 mode gre remote [outside IP from PIX] local [IP from squid Server] dev eth0

Anyway, now it's okay, but why?

Pramod said...

This is a super cool document . thanks.
Also , could you help me in how to block p2p using squid integrated with wccp.

Parvinder Bhasin said...

Hi Prats,

Thanks for you nice comment :)

Regarding not allowing p2p access through squid, if you look at the ACL that we set on the ASA:

access-list proxyclients extended permit tcp host 192.168.100.202 any eq www
access-list proxyservers extended permit ip host 192.168.100.3 any

The above acl is only allowing port 80 (www) traffic from host 192.168.100.202 over to the proxyserver 192.168.100.3. You can see that we are redirecting ONLY port 80 traffic through the squid. No other protocol would be routed to the squid box.

However your p2p requests would go out through ASA. For that you can enable some ACLs to block outbound p2p traffic.

Hope this helps :)

-Parvinder Bhasin

superpaia said...

important: "The only topology that the security appliance supports is when client and cache engine are behind the same interface of the security appliance and the cache engine can directly communicate with the client without going through the security appliance." from Cisco.

by the way great guide.

Unknown said...

I Got Question

$ASA_ROUTER_IDENTIFIER --> is the highest ip address in asa, can i change it in my cisco ASA, because the highest ip doesnt have access/allow to internet.

$SQUID_SERVER_IP --> i have one leg/interface squid(eth0) , doest this mean $SQUID_SERVER_IP is my eth0 address ?

ifconfig gre1 inet 1.2.3.4 netmask 255.255.255.0 up --> whats 1.2.3.4 mean ? can i use ip from the same network form my squid box ?

Thx,
very very newbie