-
I am trying to get WANOS up and running on a Juniper WXC 590. So far, after some fiddling, I am able to get it to boot and initialize the 'Management' interface on the Juniper. The primary card appears to be stuck in bypass mode. It is represented by lan0 in the attached images. Wan0 is the 'Management' interface on the front of the appliance.
The main NIC is a Silicom Dual Port Gig Ethernet Adapter PXG2BPIG-JP and as you can see in the pics, there appears to be a spot on the board where a 'Bypass Mode' switch could have been soldered. Is this NIC fully supported by WANOS? Or would I need to figure out a way to bypass it or replace it with one that is?
Thanks!
-
It appears that the card is not detected / driver not supported. Perhaps confirm all interfaces with 'ifconfig -a' to see if its a complete driver issue or just a case of switching the bypass on/off.
Silicom supply Linux drivers via their online support. Its potentially something we can look into.
-
tc@wanos:~$ ifconfig -a
dummy0 Link encap:Ethernet HWaddr 8E:01:98:C1:C8:C8
BROADCAST NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
ip_vti0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lan0 Link encap:Ethernet HWaddr 00:30:48:69:10:B9
UP BROADCAST NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:286 errors:0 dropped:0 overruns:0 frame:0
TX packets:286 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:18047 (17.6 KiB) TX bytes:18047 (17.6 KiB)
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:192.168.6.200 P-t-P:192.168.6.200 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:411 errors:0 dropped:0 overruns:0 frame:0
TX packets:813 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:36400 (35.5 KiB) TX bytes:104054 (101.6 KiB)
tunl0 Link encap:UNSPEC HWaddr 00-00-00-00-FF-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:1480 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
wan0 Link encap:Ethernet HWaddr 00:30:48:69:10:B8
UP BROADCAST RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:1264 errors:0 dropped:0 overruns:0 frame:0
TX packets:817 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:151550 (147.9 KiB) TX bytes:115988 (113.2 KiB)
-
It seems that the card is being detected. What is interesting is that lan0 and wan0 are on the same MAC range: 00:30:48:69:10:B8 and 00:30:48:69:10:B9 which is normally the case with dual port cards. Are you 100% sure wan0 is binding on the management port?
When connecting both cables to the dual port card what happens?
Are the silicom card mac addresses visible on the physical card?
Lastly, I don't see any jumpers to switch the bypass as with the Expand Accelerator (http://wanos.co/forum/index.php?topic=142.0), but just maybe its possible to set it in the bios?
-
It seems that the card is being detected. What is interesting is that lan0 and wan0 are on the same MAC range: 00:30:48:69:10:B8 and 00:30:48:69:10:B9 which is normally the case with dual port cards. Are you 100% sure wan0 is binding on the management port?
When connecting both cables to the dual port card what happens?
Are the silicom card mac addresses visible on the physical card?
Lastly, I don't see any jumpers to switch the bypass as with the Expand Accelerator (http://wanos.co/forum/index.php?topic=142.0), but just maybe its possible to set it in the bios?
Yes, I hadn't noticed the similar mac addresses on the one interface, but after a second look I think I see why. It appears the system main board has the potential for two nics. As you can see in the one pic by LAN1 and LAN2, but a second port wasn't added to the board. How's that for confusing?
I've added some more pics for clarification, but to answer your questions:
Yes, I'm sure wan0 is bound to the management port, I can unplug both cables from the Silicom card and still access the device.
Connecting both cables (as in the pic) does nothing. It still shows it is in bypass. If I hook another computer up to the empty switch connected to the Remote Port of the nic, I can get an IP, network access, etc. from that computer.
Yes, both interfaces have MAC address stickers. See attached pics...
It could be possible I suppose, unfortunately this board has no VGA port soldered to it, but one could be added so I could see.
-
Oh ok, yes, looks like Linux is picking up that second NIC, although there is no RJ45 port connected to the motherboard. I guess they did not enable it to 'avoid' confusion.
It seems then my previous post was wrong and the Silicom card is not detected. It seems to be a driver issue then. I have contacted silicom-usa to request the drivers. Lets see if they have drivers available and its simple to get going.
-
That would be fantastic. I did go to their website and have them in a .zip... too large for upload however.
-
Would it be possible to provide remote access to the WXC 590 to see if we can get the bypass card up?
-
If you need to I think that could be arranged... we could do a webex and let you SSH in. Thanks!
-
Ok some partial success. The drivers are ok on the older kernels, which was used in v.1.4.1.
Here is a possible way to get the card going:
Download and extract the VHD version of v.2.0.5 (It uses the older kernel)
Convert the VHD to RAW disk image format. It should create a 64GB disk image.
Write the RAW image to HDD. It will take a long time with DD and Win Disk Imager, so if you have something better use that.
Once the system is up and running, download the bypass e1000bp.ko driver.
Save it in /tce/packages/click/
Edit /opt/bootsync.sh and add the following lines just before the bootlocal line:
install -D -m 644 /mnt/sda1/tce/packages/click/e1000bp.ko /lib/modules/3.0.21-tinycore/kernel/drivers/net/e1000bp/e1000bp.ko
insmod /mnt/sda1/tce/packages/click/e1000bp.ko
modprobe e1000bp BPoff=1
After a reboot or entering these commands in manually as root (sudo), if 'ifconfig -a' then produces some additional interfaces then we are in luck and can then check that the bypass is off and then edit /tce/etc/mactab with the correct lan0/wan0 addresses.
-
Ok, let me give it a try! I'll report back and let you know if we can get this thing a going.
-
Download and extract the VHD version of v.2.0.5 (It uses the older kernel)
Convert the VHD to RAW disk image format. It should create a 64GB disk image.
I'm getting about an 8GB raw image file, is that correct? I'm using qemu-img for Windows to do it.
-
Not familiar with qemu-img, but I think it converted to a sparse raw image. Not sure if it will boot. Taking a look into it.
Edit: Writing the sparse raw disk created by qemu-img to HDD did not work.
-
Not familiar with qemu-img, but I think it converted to a sparse raw image. Not sure if it will boot. Taking a look into it.
Edit: Writing the sparse raw disk created by qemu-img to HDD did not work.
What do you recommend?
-
An image will be ready tomorrow pre-loaded with the silicom e1000bp driver, so the process will be as per normal: write to drive, boot, check interfaces.
It's worth a shot, if the card and bypass toggling works then we'll get a couple to experiment with in the lab and make it officially supported.
-
Preloaded Silicom PxGxBP driver image. Extracted its a bit large at 64GB and will take +-45 min to write to hdd depending on method used.
Link: "dead link"
-
We're close!
Got the image installed and it does see the network interface now. See attached. How do I toggle bypass on the card? It is loading up in bypass mode.
tc@wanos:~$ ifconfig -a
dummy0 Link encap:Ethernet HWaddr 4E:2C:3F:0F:CE:34
BROADCAST NOARP MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth2 Link encap:Ethernet HWaddr 00:E0:ED:0A:95:E8
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
eth3 Link encap:Ethernet HWaddr 00:E0:ED:0A:95:E9
BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lan0 Link encap:Ethernet HWaddr 00:30:48:69:10:B9
UP BROADCAST NOARP MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
inet addr:192.168.6.200 P-t-P:192.168.6.200 Mask:255.255.255.0
UP POINTOPOINT RUNNING NOARP PROMISC MULTICAST MTU:1500 Metric:1
RX packets:73 errors:0 dropped:0 overruns:0 frame:0
TX packets:66 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:10555 (10.3 KiB) TX bytes:11763 (11.4 KiB)
wan0 Link encap:Ethernet HWaddr 00:30:48:69:10:B8
UP BROADCAST RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:158 errors:0 dropped:0 overruns:0 frame:0
TX packets:73 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:30946 (30.2 KiB) TX bytes:12981 (12.6 KiB)
Thanks!
-
Ok great!
We need to do two things:
1) Edit the /tce/etc/mactab file to reflect the silicom card macs. Steps here to change interfaces (http://wanos.co/forum/index.php?topic=157.0)
2) To load the card out of bypass mode edit /opt/bootsync.sh and change the modprobe line to read modprobe e1000bp BPoff=1
Please also test if this command toggles the bypass on/off manually (difficult to test without a card, but I think it should be bypass_eth0):
sudo echo "off" > /proc/net/bypass/bypass_wan0/bypass
sudo echo "on" > /proc/net/bypass/bypass_wan0/bypass
Edit: changed bypass_eth0 to bypass_wan0 for future reference
-
Edited the mactab and bootsync.sh files... restarted... card was in bypass, no connection. pulled drive and changed mactab back to other interface... reinstalled into Juniper and connected again.
Ran the bypass on/off command:
tc@wanos:~$ sudo echo "off" > /proc/net/bypass/bypass_eth0/bypass
-sh: can't create /proc/net/bypass/bypass_eth0/bypass: nonexistent directory
tc@wanos:~$ sudo echo "off" > /proc/net/bypass/bypass_eth0/bypass
-sh: can't create /proc/net/bypass/bypass_eth0/bypass: nonexistent directory
Obviously, nothing happens as a result.
Seems I'm missing something... I double checked and triple checked the MAC address to make sure I entered them right. They are, but when it booted up, the card was still lit up in bypass. Even though the bootsync.sh file shows BPoff=1
-
I got it to bypass... I pulled the card and stuck it in another machine so that I could see what I was doing.
It was bypass_wan0... not bypass_eth0.
Ok, so now that I have it working in a regular computer and I can turn the bypass on and off sending that command. If I turn it off, then run the clean.sh script and send the drive and card back over to the Juniper, will it still function correctly? Or do I need to turn it back off again once it is installed in the Juniper?
-
I think it will need to be switched via the driver with that command (some cards have jumpers). The command can be added to the scripts to disable the bypass once all the wanos processes are running.
After running the clean command, the mactab needs to be edited to match the bypass card macs. I understand there is no vga, so maybe console is worth exploring.
-
Ok, I tried a null modem cable and didn't get anything on serial... Do I need a specific pinout?
-
The pin outs depends on the hardware. It could also be that the juniper console is on a different com port (this can be changed in /tce/etc/inittab) or maybe baud rate (default 9600).
In fact the mactab change can be done as the very last step. We'll be able to toggle the bypass and make the card switch and then once this works correctly update the macs.
Update: To toggle the bypass to off automatically run this one once:
echo "sudo echo 'off' > /proc/net/bypass/bypass_wan0/bypass" >> /tce/etc/wanos/wanos
-
BINGO!
I never could get the serial to function, so I improvised. I actually edited the mactab file to point wan0 at the silicom and lan0 at the integrated nic (mgmnt), this way I could still SSH into it on the network. After that, I ran the echo off script to verify I could disable the bypass, it did... then I ran the other script to add it to wanos for automatically shutting it off. I restarted (sudo reboot) and it stayed in bypass off. Then I added the other MAC on the silicom as lan0, restarted (sudo reboot) and it stayed in bypass off... However, after powering it down (sudo poweroff) and restarting it, it is in bypass again.
-
Nevermind, syntax error on my part... it works! Boots up and turns off bypass automatically when it's finished booting!
That's awesome... Now I'm on to getting these SATA drives in it hooked up so I can use them to store the database on.
Thanks for all of your help!
With some tweaking it does work on a Juniper WXC 590 without a monitor ;-)
-
Awesome! Pleasure, totally worth it.
-
Have you ever had an issue with a random reboot? It's not quite random though, it happens about 5 min after it boots. I hear the fans go to high speed, turn around and notice the card is in bypass. See the hard drive led pulsing like it's booting up and I hear the nic 'click' out of bypass and all is well again for 5-6 min. Is there a log to see what the cause might be?
???
-
No, not random reboots. Linux is usually super stable. Unless the WXC bios / Nic driver has some watchdog / intelligence to try recover with a reboot if some software event does not occur.
Maybe have a look at the output of:
sudo cat /proc/net/bypass/bypass_wan0/bypass_wd
sudo cat /proc/net/bypass/bypass_wan0/wd_autoreset
sudo cat /proc/net/bypass/bypass_wan0/reset_bypass_wd
sudo cat /proc/net/bypass/bypass_wan0/wd_expire_time
sudo cat /proc/net/bypass/bypass_wan0/bypass_info
Got these from the driver readme file.
-
I got it... there's a jumper on the mother board of the Juniper WXC 590 labeled JWD... care to take a guess what WD stands for? ;)
Remove the jumper completely... no more reboots.
It has 2 drives in the front SATA0 and SATA1. I'm using SATA0 as the database drive and inside I'm using the IDE1 interface to boot from. I'm wondering if the BIOS is already has all boot devices enabled and listed in boot order and if I could use SATA0 to boot from and SATA1 as the database drive?
I'm going to play with it some more and see if I can. It's running stable now. I'll get the other one up and running and time for some tests! Thanks!
-
SATA should be fine. Can't see why it won't work.
-
To confirm, it will boot off the SATA drives in the front. I have the one on the left for WANOS and the one on the right for the database. I'll do the other unit and do a bench trial.
Thank you again for all of your help! Great product and support here! 8) 8) 8)
-
Pleasure, looking forward to the bench results. One bonus of the second drive bay is that it looks like a SSD upgrade in the future could be relatively simple to do.
-
Test with 30MB file...
1st Pass:
1MB = 8Mbps
2nd Pass:
3MB = 21Mbps
We have a 20Mbps connection between sites, so not too bad... Our current solution is lucky to achieve 4Mbps. The WXC 590 has 2 sockets, only 1 is filled right now, so there's room for improvement.
-
Mmm, it will be interesting to see what the usable normal operational throughput is going to be. As far as I can tell from docs online, the WXC 590 is a Supermicro board with Xeon CPU, but I am not sure which one to make a guesstimate of peak throughput.
-
I could pop off the heatsink and take a look...
-
Either way it should be capable of a lot more than the Express 6 Mbps up/outbound limit so when you are ready to test at higher rates we can load trial plus keys to remove the limit.
-
FYI: The WXC 590 has one of these processors in it, and 1 empty socket for an additional processor. 4GB of RAM as well.
http://ark.intel.com/products/27091/64-bit-Intel-Xeon-Processor-3_80-GHz-2M-Cache-800-MHz-FSB
It also has [2] 250GB SATA drives and a 1GB CompactFlash Card (mine did) with an IDE adapter inside.
-
Couple of more pics just for clarity. This unit is "supposed" to automatically turn on when power is applied. I have found this to be a bit hit and miss. On the outside of the unit, all you see is a power led, but when you open the case, there's a large block attached to the backside of the face plate. Within this block are more leds for things like network activity, hard drive activity, etc. Above those leds are two switches, one for power, one for restart. When facing the unit, the left one is for power. I drilled a 1/4" hole in the face plate to allow me access to it from the outside.
Pics of the JWD jumper location and pics of the sockets on the board added.
Thanks!
-
Great info, thanks for contributing.
That JWD jumper is a good catch. It took me a while to figure out the Expand Accelerator jumpers to get the bypass to co-operate but in the end the Expand works quite nice in that no software is required. The juniper hardware is better due to the dual core CPU (and possibility of a second CPU) and 4GB RAM. Based on CPU benchmarks of the early Xeon's I suspect the peak rate is going to be 40 Mbps throughput on cold transfers with all features enabled. Best case scenario in low or hot transfers will be in the region of 100 - 150 Mbps.
Realistically a sustainable wan rate of 20 Mbps in both directions might be a bit much. It would be interesting to see the bench results (considering adding something along those lines in the GUI to get an idea of hardware capability).