Author Topic: wanos throughput on kvm  (Read 8936 times)

peter

  • Contributor
  • Full Member
  • *****
  • Posts: 51
    • View Profile
wanos throughput on kvm
« on: June 12, 2014, 06:13:11 AM »
Hi,
First, I need to put wanos on VMware is very good.
After, I tried to put wanos(1.8) on kvm virtual machine by dd HDD image since KVM is only pure free open source virtual platform and with commands to manage VM.
But, the optimization did not work at all and even slower than no optimisation.  So,  I have to wonder if I do something wrong with the setup.  Please shed light on it.  Thanks. 

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos on kvm
« Reply #1 on: June 12, 2014, 10:14:38 AM »
Hi Peter,

Thanks for testing wanos on KVM. It is difficult to say, but I suspect it could be due to the network driver. We can test this theory by switching off optimization e.g. place the device in Edge mode and then do a test to see how much traffic it can handle. Then in the second test change the network adapters to virtio and redo the test to see if throughput is increased.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

peter

  • Contributor
  • Full Member
  • *****
  • Posts: 51
    • View Profile
Re: wanos on kvm
« Reply #2 on: August 18, 2014, 05:29:26 AM »
hi,
     I am still trying to put wanos on kvm virtual machine.  As my test on the first post was not promising, I tried to rule out degrading effect from kvm platform this time.

 Steps are as follows, 

1)  I put a Ubuntu server on kvm virtual machine with two nic interfaces.
2)  Bridge two nics and let network traffic through.  As switch is 100M, I got 10-12mbytes which is very reasonable.
3)  With exactly above setting(virtio nic and virtio disk drivers loaded ), dd 1.2.1 to kvm virtual disk and got wanos running.  But, throughput rate went down dramatically to 17k.

As regular linux distribution did not degrade on kvm, my guess is that wanos internal design does not fit on kvm platform.  However, I am still not going to give it up on KVM and would like to ask you guys to give a test,please.

Thanks

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos throughput on kvm
« Reply #3 on: August 18, 2014, 12:40:27 PM »
Hi Peter,

My knowledge of kvm is very introductory, but Silver-Peak supports kvm, so there is a way to make it work well. They also recommend the virtio nic and disk, but since you have that configured, the bottleneck must be somewhere else.

Does edge and core produce the same throughput results?

Would it be possible to run the following test:
first disable the wanos processes (/etc/init.d/wanosd stop-all) from the console, then run:
Test 1) click -e "FromDevice(lan0, PROMISC true, SNIFFER false, METHOD PCAP) -> Queue -> ToDevice(wan0, METHOD PCAP); FromDevice(wan0, PROMISC true, SNIFFER false, METHOD PCAP) -> Queue -> ToDevice(lan0, METHOD PCAP);"
Test 2) click -e "FromDevice(lan0, PROMISC true, SNIFFER false, METHOD LINUX) -> Queue -> ToDevice(wan0, METHOD LINUX); FromDevice(wan0, PROMISC true, SNIFFER false, METHOD LINUX) -> Queue -> ToDevice(lan0, METHOD LINUX);"

Compare the throughput of test 1 & 2 with the wanos Core and Edge results.
« Last Edit: August 18, 2014, 01:03:43 PM by ahenning »
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

peter

  • Contributor
  • Full Member
  • *****
  • Posts: 51
    • View Profile
Re: wanos throughput on kvm
« Reply #4 on: September 02, 2014, 08:21:31 AM »
Good news is wanos seems to work on KVM, but I will keep testing to make sure it
 is.

But both test commands did not seem to work on both core and edge nodes within 10 minutes testing period.

Test 1 (PCAP) on both core and edge only gave following output:

config:1: While initializing 'FromDevice@1 :: FromDevice':
   warning: lan0: no IPv4 address assigned

config:1: While initializing 'FromDevice@4 :: FromDevice':
   warning: lan0: no IPv4 address assigned

Test 2 (LINUX) on both core and edge nodes gave nothing output.

Thanks

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos throughput on kvm
« Reply #5 on: September 02, 2014, 10:28:30 AM »
Hi Peter,

Ok, great to hear that wanos is working on kvm. What throughput are you getting?

The test commands are just that, tests, to see the different throughput. Another method will be available soon as well.

Do you perhaps have a short guide or list of steps to get a basic wanos on kvm setup going?

What is the kvm end goal though? E.g. min/max optimization at x Mbps.
« Last Edit: September 02, 2014, 10:52:01 AM by ahenning »
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

peter

  • Contributor
  • Full Member
  • *****
  • Posts: 51
    • View Profile
Re: wanos throughput on kvm
« Reply #6 on: September 12, 2014, 06:38:19 AM »
Step for setup on KVM
Many  kudoes to Silverpeak kvm virtual machine setup guide.

Base system:

Ubuntu server 10.04 (LTS) (ubuntu-10.04.4-server-amd64.iso)
with 3 nics as follows:
nic wan and lan interfaces for virtual machine bridging
nic mgmt ip 192.168.2.100

Step1 : basic package install
Pls do apt-get update first

apt-get install qemu-kvm  (virtual machine kernel module)
apt-get install bridge-utils (virtual networking)
apt-get install vncviewer
(virtual machine console connection, as Ubuntu server does not have X, qemu
console simulation will not work.)

Ubuntu network config file for bridging to work (/etc/network/interface)
Mapping network interface name, config goes to file /etc/udev/rules.d/70-persistent-net.rules
***************************************
auto lo
iface lo inet loopback


auto mgmt
iface mgmt inet static
      address 192.168.2.100
      network 192.168.2.0
      netmask 255.255.255.0
      gateway 192.168.2.1

auto wbr
iface wbr inet manual
        bridge_ports wan
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

auto lbr
iface lbr inet manual
        bridge_ports lan
        bridge_stp off
        bridge_fd 0
***************************************
Step 2:  Creating Ubuntu server virtual machine for dding wanos disk image
1) creating a Ubuntu server virtual diska (20G);
   qemu-img create -f raw  diska 20G
2) creating a Wanos virtual diskb (70G)
   qemu-img create -f raw  diskb 70G

(Silverpeak emphasizes that virtual disk needs to be on raw format)

Step 3:
1) Install virtual Ubuntu server on diska
   sudo qemu-system-x86_64   -device virtio-net-pci,netdev=net1,mac=DE:AD:BE:EF:35:52 -netdev tap,id=net1,script=qemu-ifup-wan -device virtio-net-pci,netdev=net0,mac=DE:AD:BE:EF:35:51 -netdev tap,id=net0,script=qemu-ifup-lan -vnc :1 -drive file=./diska,if=virtio -m 2048 -boot d -cdrom ./ubuntu-10.04.4-server-amd64.iso

   vncviewer 192.168.2.100 :1
   to connect console for system installation.
qemu-ifup-lan file
#!/bin/sh
nic=$1
ifconfig $nic 0.0.0.0 up
brctl addif lbr $nic

qemu-ifup-wan file
#!/bin/sh
nic=$1
ifconfig $nic 0.0.0.0 up
brctl addif wbr $nic

2) boot after completing server installation
   sudo qemu-system-x86_64   -device virtio-net-pci,netdev=net1,mac=DE:AD:BE:EF:35:52 -netdev tap,id=net1,script=qemu-ifup-wan -device virtio-net-pci,netdev=net0,mac=DE:AD:BE:EF:35:51 -netdev tap,id=net0,script=qemu-ifup-lan -vnc :1 -drive file=./diska,if=virtio,boot=on -drive file=./diskb,if=virtio -m 2048

3) Upload Wanos image and dding to diskb
   kvm calls diska as /dev/vda and diskb as /dev/vdb

4) dump diska and boot virtual machine on diskb
   sudo qemu-system-x86_64   -device virtio-net-pci,netdev=net1,mac=DE:AD:BE:EF:35:52 -netdev tap,id=net1,script=qemu-ifup-wan -device virtio-net-pci,netdev=net0,mac=DE:AD:BE:EF:35:51 -netdev tap,id=net0,script=qemu-ifup-lan -vnc :1 -drive file=./diskb,if=virtio,boot=on  -m 2048


Step 4:
by vncviewer, logging on wanos and doing clean.sh script to restart wanos.
« Last Edit: September 12, 2014, 06:53:43 AM by peter »

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos throughput on kvm
« Reply #7 on: September 12, 2014, 10:17:52 AM »
Excellent, thanks Peter, that is quite comprehensive. I'll add it to the wiki as well.

I am curious what throughput you are getting in KVM? e.g. just a single device without optimization, edge and core. Earlier you mentioned very low numbers in the Kbps range. Has this improved?

Netmap support will be included soon for a little more efficiency. It would be interesting to see whether netmap makes a noticeable difference in the kvm scenario.

Thanks for taking the time to write and submit the kvm howto guide. Appreciate it.
« Last Edit: September 12, 2014, 10:26:26 AM by ahenning »
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

peter

  • Contributor
  • Full Member
  • *****
  • Posts: 51
    • View Profile
Re: wanos throughput on kvm
« Reply #8 on: September 18, 2014, 06:53:19 AM »
Eventually, I caught the break.  The issue with low throughput is because of  NIC
 hardware.  I was using Silicom cards with company provided  drivers for testing.
When I switched to pure Intel NIC cards, the throughput on KVM is good. 
                           
Testing result with wanos on KVM virtual machine

a) 100Mbits network environment;
b) vsftpd for ftp testing run
 
1) with both core and edge bypass (no optimization)
2) with only core bypass
3) with only edge bypass

all above three test cases gave around 10-12MB throughput which is very reasonable.

Thanks again.

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos throughput on kvm
« Reply #9 on: September 23, 2014, 10:56:30 AM »
Thanks for the info Peter,

10-12 MB throughput on 100 Mbps infrastructure is about line rate. I think the kvm platform is something to consider and hopefully down the line make it an officially supported platform.

Thanks for taking the time to submit this info.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs