Slow ssh and httpd connections with weighted routing…

[arfon]

I’m running weighted routing and I added the following rules:

1 ppp1 * MARK tcp opt — in ppp1 out * 0.0.0.0/0 -> 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp spt:80 MARK set 0x66 DSL2 (ppp1)
2 ppp1 * MARK tcp opt — in ppp1 out * 0.0.0.0/0 -> 0.0.0.0/0 tcp spt:22 MARK set 0x66 DSL2 (ppp1)
3 * * MARK all opt — in * out * 192.168.1.70 -> 0.0.0.0/0 MARK set 0x66 DSL2 (ppp1)
4 * * MARK tcp opt — in * out * 192.168.1.10 -> 0.0.0.0/0 tcp spt:22 MARK set 0x66 DSL2 (ppp1)

Here’s the details-
1) 192.168.1.70 is my httpd server. I have all incoming PPP1 port 80 traffic forwarded to it.
2) 192.168.1.10 is my workstation. I have all incoming PPP1 port 22 traffic forward to it.

Now, when I ssh into my work station, there’s HUGE delays. When I disable PPP0, it works fine. Same with http requests to my webserver.

Any ideas what I’m doing wrong???

[atheling]

Which version of Zeroshell are you running?

Are you running any patches to that? Specifically are you running my net balance and QoS patch show on this thread:

http://www.zeroshell.net/eng/forum/viewtopic.php?t=2125

If you use my patch, you should not need the rules you have listed.

To what chain did you add those iptable rules? Perhaps you could show the output of the following shell command:

iptables -t mangle -L -vn

[arfon]

iptables -t mangle -L -vn:
pkts bytes target prot opt in out source destination
521K 632M MARK tcp — ppp0 * 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp spt:80 MARK set 0x65
5535K 3429M ACCEPT all — * * 0.0.0.0/0 0.0.0.0/0 MARK match !0x0
76 18051 MARK tcp — ppp1 * 0.0.0.0/0 0.0.0.0/0 state NEW,RELATED,ESTABLISHED tcp spt:80 MARK set 0x66
76 18051 ACCEPT all — * * 0.0.0.0/0 0.0.0.0/0 MARK match !0x0
0 0 MARK tcp — ppp1 * 0.0.0.0/0 0.0.0.0/0 tcp spt:10 MARK set 0x66
0 0 ACCEPT all — * * 0.0.0.0/0 0.0.0.0/0 MARK match !0x0

Which version of Zeroshell are you running?
Release 1.0.beta12

Are you running any patches to that?
I haven’t applied any. Maybe update has…

And let me point out that I’m actually running sshd on port 10, not 22.

[ppalias]

If you are running sshd on port 10 then these rules won’t work as they allow incoming connections on port 22. Apply atheling’s patch, it is rather efficient in these situations, where incoming connections are not returned back from the correct wan link.

[arfon]

For ssh on port 10, couldn’t I just change it to:

Index: kerbynet.cgi/scripts/fw_initrules

===================================================================

RCS file: /home/atheling/cvsroot/Zeroshell/Zeroshell/kerbynet.cgi/scripts/fw_initrules,v

retrieving revision 1.1.1.1

diff -w -u -6 -r1.1.1.1 fw_initrules

--- kerbynet.cgi/scripts/fw_initrules   26 Nov 2009 22:13:35 -0000   1.1.1.1

+++ kerbynet.cgi/scripts/fw_initrules   1 Dec 2009 03:51:40 -0000

@@ -2,13 +2,13 @@

. /etc/kerbynet.conf

CHAIN="$1"

[ -z "$CHAIN" ] && exit 1

CONFIG="$REGISTER/system/net/FW/"

if [ "$CHAIN" == QoS ] ; then

TABLE="-t mangle"

-  CH=FORWARD

+  CH=QoS

else

if [ "$CHAIN" == NetBalancer ] ; then

TABLE="-t mangle"

CH=NetBalancer

else

TABLE=""

@@ -23,12 +23,16 @@

iptables -A INPUT -j SYS_INPUT

iptables -A INPUT -p tcp --dport 80 -j SYS_HTTPS

iptables -A INPUT -p tcp --dport 443 -j SYS_HTTPS

iptables -A INPUT -p tcp --dport 10 -j SYS_SSH

fi

[ "$CHAIN" == OUTPUT ] && iptables -A OUTPUT -j SYS_OUTPUT

+  # If we are doing the QoS chain, thenlear any marks left over from

+  # Netbalancing/failover routing. The QoS chain is applied after

+  # routing so there is no conflict.

+  [ "$CHAIN" == "QoS" ] && iptables $TABLE -A $CH -j MARK --set-mark 0x0

if [ -d $CONFIG/Chains/$CHAIN/Rules ] ; then

cd $CONFIG/Chains/$CHAIN/Rules

RULES=`ls`

for RULE in $RULES ; do

ENABLED="`cat $RULE/Enabled 2>/dev/null`"

if [ "$ENABLED" == yes ] ; then

Index: kerbynet.cgi/scripts/fw_makerule

===================================================================

RCS file: /home/atheling/cvsroot/Zeroshell/Zeroshell/kerbynet.cgi/scripts/fw_makerule,v

retrieving revision 1.1.1.1

diff -w -u -6 -r1.1.1.1 fw_makerule

--- kerbynet.cgi/scripts/fw_makerule   26 Nov 2009 22:13:35 -0000   1.1.1.1

+++ kerbynet.cgi/scripts/fw_makerule   1 Dec 2009 03:32:42 -0000

@@ -4,13 +4,13 @@

RULE="$2"

OPT="$3"

[ -z "$CHAIN" -a -z "$RULE" ] && exit 1

CONFIG="$REGISTER/system/net/FW"

if [ "$CHAIN" = QoS ] ; then

TABLE="-t mangle"

-  CH=FORWARD

+  CH=QoS

else

if [ "$CHAIN" = NetBalancer ] ; then

TABLE="-t mangle"

CH=NetBalancer

else

TABLE=""

@@ -411,13 +411,13 @@

iptables $TABLE $IPT $TGT

if [ "$CHAIN" == QoS ] ; then

TGTDSCP=`cat $REGISTER/system/net/QoS/Class/$TARGET/DSCP 2>/dev/null`

if [ -n "$TGTDSCP" ] ; then

iptables $TABLE $IPT -j DSCP --set-dscp $TGTDSCP

fi

-      iptables -t mangle -A FORWARD -m mark ! --mark 0 -j ACCEPT

+      iptables -t mangle -A QoS -m mark ! --mark 0 -j ACCEPT

fi

if [ "$CHAIN" == NetBalancer ] ; then

[ "$TARGET" != Auto ] && iptables -t mangle -A NetBalancer -m mark ! --mark 0 -j ACCEPT

fi

fi

fi

Index: kerbynet.cgi/scripts/fw_start

===================================================================

RCS file: /home/atheling/cvsroot/Zeroshell/Zeroshell/kerbynet.cgi/scripts/fw_start,v

retrieving revision 1.1.1.1

diff -w -u -6 -r1.1.1.1 fw_start

--- kerbynet.cgi/scripts/fw_start   26 Nov 2009 22:13:35 -0000   1.1.1.1

+++ kerbynet.cgi/scripts/fw_start   30 Nov 2009 22:10:47 -0000

@@ -10,12 +10,18 @@

iptables -t mangle -F NetBalancer 2>/dev/null

iptables -t mangle -X NetBalancer 2>/dev/null

iptables -t mangle -N NetBalancer 2>/dev/null

iptables -t mangle -F OpenVPN 2>/dev/null

iptables -t mangle -X OpenVPN 2>/dev/null

iptables -t mangle -N OpenVPN 2>/dev/null

+iptables -t mangle -F QoS 2>/dev/null

+iptables -t mangle -X QoS 2>/dev/null

+iptables -t mangle -N QoS 2>/dev/null

+iptables -t mangle -F NB_CT_PRE 2>/dev/null

+iptables -t mangle -X NB_CT_PRE 2>/dev/null

+iptables -t mangle -N NB_CT_PRE 2>/dev/null

[ "$CPGW" == yes ] && iptables -N CapPort

$SCRIPTS/fw_https_chain

$SCRIPTS/fw_ssh_chain

$SCRIPTS/fw_sys_chain

CHAINS=`ls`

for C in $CHAINS ; do

Index: kerbynet.cgi/scripts/fw_viewchain

===================================================================

RCS file: /home/atheling/cvsroot/Zeroshell/Zeroshell/kerbynet.cgi/scripts/fw_viewchain,v

retrieving revision 1.1.1.1

diff -w -u -6 -r1.1.1.1 fw_viewchain

--- kerbynet.cgi/scripts/fw_viewchain   26 Nov 2009 22:13:35 -0000   1.1.1.1

+++ kerbynet.cgi/scripts/fw_viewchain   30 Nov 2009 19:30:43 -0000

@@ -1,7 +1,7 @@

#!/bin/sh

. /etc/kerbynet.conf

CHAIN="$1"

[ -z "$CHAIN" ] && exit 1

-[ "$CHAIN" == QoS ] && CHAIN="FORWARD -t mangle"

+[ "$CHAIN" == QoS ] && CHAIN="QoS -t mangle"

[ "$CHAIN" == NetBalancer ] && CHAIN="NetBalancer -t mangle"

iptables -n -v -L $CHAIN

Index: kerbynet.cgi/scripts/nb_fw

===================================================================

RCS file: /home/atheling/cvsroot/Zeroshell/Zeroshell/kerbynet.cgi/scripts/nb_fw,v

retrieving revision 1.1.1.1

diff -w -u -6 -r1.1.1.1 nb_fw

--- kerbynet.cgi/scripts/nb_fw   26 Nov 2009 22:13:35 -0000   1.1.1.1

+++ kerbynet.cgi/scripts/nb_fw   10 Apr 2010 13:44:21 -0000

@@ -1,23 +1,35 @@

#!/bin/sh

. /etc/kerbynet.conf

iptables -t mangle -D PREROUTING -j CONNMARK --restore-mark 2>/dev/null

+iptables -t mangle -D PREROUTING -m state --state NEW -j NB_CT_PRE 2>/dev/null

iptables -t mangle -D PREROUTING -j NetBalancer 2>/dev/null

+iptables -t mangle -D INPUT -m state --state NEW -j NB_CT_POST 2>/dev/null

iptables -t mangle -D INPUT -j NetBalancer 2>/dev/null

+iptables -t mangle -D OUTPUT -j CONNMARK --restore-mark 2>/dev/null

iptables -t mangle -D OUTPUT -j NetBalancer 2>/dev/null

iptables -t mangle -D OUTPUT -j OpenVPN 2>/dev/null

iptables -t mangle -D POSTROUTING -m state --state NEW -j NB_CT_POST 2>/dev/null

iptables -t mangle -D POSTROUTING  -j NB_STAT 2>/dev/null

+# Need QoS to be done in mangle POSTROUTING. Note that if NetBalance

+# is enabled then we will insert those rules/chains first. So any

+# routing marks will be handled before we blow them away with QoS

+# marks.

+iptables -t mangle -D POSTROUTING  -j QoS 2>/dev/null

+iptables -t mangle -I POSTROUTING 1 -j QoS 2>/dev/null

if [ "`cat $REGISTER/system/net/nb/Enabled 2>/dev/null`" = yes ] ; then

iptables -t mangle -I PREROUTING 1 -j CONNMARK --restore-mark

-  iptables -t mangle -I PREROUTING 2 -j NetBalancer

+  iptables -t mangle -I PREROUTING 2 -m state --state NEW -j NB_CT_PRE 2>/dev/null

+  iptables -t mangle -I PREROUTING 3 -j NetBalancer

+  iptables -t mangle -I INPUT 1 -m state --state NEW -j NB_CT_POST 2>/dev/null

+  iptables -t mangle -I INPUT 2 -j NetBalancer

+  iptables -t mangle -I OUTPUT 1 -j CONNMARK --restore-mark

+  iptables -t mangle -I OUTPUT 2 -j NetBalancer

+  iptables -t mangle -I OUTPUT 3 -j OpenVPN

iptables -t mangle -I POSTROUTING 1 -m state --state NEW -j NB_CT_POST 2>/dev/null

iptables -t mangle -I POSTROUTING 2 -j NB_STAT 2>/dev/null

-  iptables -t mangle -I INPUT 1 -j NetBalancer

-  iptables -t mangle -I OUTPUT 1 -j NetBalancer

-  iptables -t mangle -I OUTPUT 2 -j OpenVPN

fi

$SCRIPTS/nb_vpn 2> /dev/null

$SCRIPTS/nb_setautomarking 2>/dev/null







Index: kerbynet.cgi/scripts/nb_setautomarking

===================================================================

RCS file: /home/atheling/cvsroot/Zeroshell/Zeroshell/kerbynet.cgi/scripts/nb_setautomarking,v

retrieving revision 1.1.1.1

diff -w -u -6 -r1.1.1.1 nb_setautomarking

--- kerbynet.cgi/scripts/nb_setautomarking   26 Nov 2009 22:13:35 -0000   1.1.1.1

+++ kerbynet.cgi/scripts/nb_setautomarking   4 Dec 2009 03:41:47 -0000

@@ -3,27 +3,56 @@

CONFIG=$REGISTER/system/net/nb/Gateways

cd $CONFIG

function set_gwmark {

xGW="$1"

INTERFACE=`cat $xGW/Interface 2>/dev/null`

IP=`cat $xGW/IP 2>/dev/null`

+  # Set up the pre-routing chain for new connections from this Gateway. We want

+  # to mark all traffic originating from this gateway to be routed back out to the

+  #same gateway.

+

+  # If this Gateway has no interface device defined for it, see if we can get

+  # one based on the next hop IP address

+  if [ "$INTERFACE" == "" ] ; then

+    if [ "$IP" != "" ] ; then

+        INTERFACE=`ip route get $IP | grep -o "dev w*" | awk 'BEGIN {FS=" "}{print $2}'`

+    fi

+  fi

+  # If we have found the interface, then mark all traffic coming in on it to use

+  # it for outbound responses

+  if [ "$INTERFACE" != "" ] ; then

+    if ! iptables -t mangle -L NB_CT_PRE -n | grep -q -w `echo 1$xGW |awk '{printf ("0x%x",$0)}'` ; then

+       [ "`cat $xGW/Enabled 2>/dev/null`" = yes ] && iptables -t mangle -I NB_CT_PRE 1 -i $INTERFACE -j MARK --set-mark 1$xGW

+     else

+       [ "`cat $xGW/Enabled 2>/dev/null`" != yes ] && iptables -t mangle -D NB_CT_PRE -i $INTERFACE -j MARK --set-mark 1$xGW

+    fi

+  fi

+

+  # In the post routing phase, we want to get the the routing realm used for new

+  # connections and save it in the connection. First setp here is to get the mark

+  # and put it on the packet. Our caller will emit the code to save the marks to

+  # the connection.

if ! iptables -t mangle -L NB_CT_POST -n | grep -q -w `echo 1$xGW |awk '{printf ("0x%x",$0)}'` ; then

[ "`cat $xGW/Enabled 2>/dev/null`" = yes ] && iptables -t mangle -I NB_CT_POST 1 -m realm --realm 1$xGW -j MARK --set-mark 1$xGW

else

[ "`cat $xGW/Enabled 2>/dev/null`" != yes ] && iptables -t mangle -D NB_CT_POST -m realm --realm 1$xGW -j MARK --set-mark 1$xGW

fi

+

+  # Make the entry in the statistics chain so we can track how much traffic went

+  # over each gateway

if ! iptables -t mangle -L NB_STAT -n | grep -q -w `echo 1$xGW |awk '{printf ("0x%x",$0)}'` ; then

[ "`cat $xGW/Enabled 2>/dev/null`" = yes ] && iptables -t mangle -I NB_STAT 1 -m mark --mark 1$xGW

else

[ "`cat $xGW/Enabled 2>/dev/null`" != yes ] && iptables -t mangle -D NB_STAT -m mark --mark 1$xGW

fi

}

GW="$1"

if [ -z "$GW" ] ; then

GW=`ls -d ?? 2>/dev/null`

iptables -t mangle -F NB_CT_POST

+  iptables -t mangle -F NB_CT_PRE

iptables -t mangle -F NB_STAT

for G in $GW ; do

set_gwmark $G

done

iptables -t mangle -D NB_CT_POST -j CONNMARK --save-mark 2> /dev/null

iptables -t mangle -A NB_CT_POST -j CONNMARK --save-mark 

[ppalias]

I suppose you can, you need to check iptables after you apply the patch that allows port 10.

[Author]

Posts