Author Topic: wanos + ESXi with pfSense NAT  (Read 17165 times)

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
wanos + ESXi with pfSense NAT
« on: December 23, 2014, 04:05:18 PM »
I have a similar setup on pfsense (simulating two sites on one esxi host), but I guess it's not working because local status is READY but Peer Status is Down
Traffic policies are default.
I have verified port connections are correct by matching mac addresses
Promiscuous Mode enabled on ports Lan, Lan_010, Main, Main_010

The setup as shown in attachment is as follows;
Pfsense, WanOs, Win7 (default ports)
Pfsense_010, WanOS_010, Win7_010 (Vlan10)
vSwitch2 is acting as a bridge between Pfsense & WanOs

On Win7, there is a website hosted and it can be accessed locally i.e. http://localhost.
If the website is accessed from Win7_010 (pfsense has nat rule for publishing http), site won't load and
internet & access to pfsense web gui stops for couple of minutes from both desktops.

Please suggest.

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #1 on: December 23, 2014, 06:02:42 PM »
Hi Astar,

Interesting, complicated at first. To be honest I had a bit of a hardtime to understand the traffic flow with the names and vlans, so please bare with me if I got it wrong:

So it seems this lab networks has:
2x Win7 PC
2x Wanos
2x pfSense

WAN is where the two pfSense machines connect back to back. So we can ignore this network, except if the physical vmnic0 connects to the same network as vmnic3.

Vlan 0 simulates one site: Win7, wanos0
Vlan 10 simulates another site: Win7-10, wanos10

The two sites (or vlans) are connected to each other via the WAN network.

LAN and MAIN means site one and site two, basically.

Ok got it :)

The problem is most likely to be the NAT rule if it is based on port 80. Most routers and firewalls can't NAT ip proto 108 IPComp and drops the packets. Traffic stops for a while while the one peer is 'up' and after 60 Seconds they timeout and you can get to the pfSense GUI again.

The methods to resolve it is to use one to one NAT, disable NAT, or set encapsulation under optimization settings to UDP (Encap needs to match on both ends).

Also after all is working you can add a bypass rule for the pfSense ip address so that it is possible to access the pfSense GUI while the peers are up.

Hope it helps, please let me know if we need to investigate further.


« Last Edit: February 23, 2015, 10:13:07 PM by ahenning »
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #2 on: December 23, 2014, 06:45:31 PM »
vmnic0 = WAN. Has it's own physical nic connecting to office network (treating as WAN uplink)
vmnic3 = LAN. Has it's own physical nic connecting to a seperate switch
No adapters(Main) = bridging between PFsense_Lan and WanOS_WAN.

Lan = Site A (vlan0). This has Win7 and wanos
Lan_010 = Site b (vlan10). This has win7_010 and wanos_010

seems UDPENCAP=Enable in /tce/etc/wanos/wanos.conf did not help.

I am afraid as I can't disable nat nor enable 1:1 nat.

 

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #3 on: December 23, 2014, 07:11:35 PM »
After editing /tce/... a soft reset via the interface is also required.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #4 on: December 23, 2014, 07:13:49 PM »
Rebooted WanOS as well as PFSense after enabling UDPENCAP.

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #5 on: December 23, 2014, 07:17:14 PM »
Following entries are in log

Site A
Tue Dec 23 17:48:00 UTC 2014  : Info : Cleared Statistics and Logs
Tue Dec 23 17:48:15 UTC 2014  : Info : Clearing the datastore
Tue Dec 23 17:48:15 UTC 2014  : Info : Reboot Requested
Tue Dec 23 17:48:27 UTC 2014  : Routine : Initializing Startup Scripts
Tue Dec 23 17:48:27 UTC 2014  : Routine : Set Interface Roles
Tue Dec 23 17:48:27 UTC 2014  : Routine : Setting Optional Interface driver flags if supported
Cannot set device flag settings: Operation not supported
Cannot set device flag settings: Operation not supported
Tue Dec 23 17:48:27 UTC 2014  : Routine : Check Configs
Tue Dec 23 17:48:27 UTC 2014  : Routine : Updating Configuration
Tue Dec 23 17:48:27 UTC 2014  : Routine : Initializing Wanos Click
wanos.click:11: While initializing 'wan0 :: FromDevice':
  warning: wan0: no IPv4 address assigned
wanos.click:12: While initializing 'lan0 :: FromDevice':
  warning: lan0: no IPv4 address assigned
Debug : Building datastore: 0
Debug : Prepare Ready: 0
Debug : Probe On
DropBroadcasts: dropped a packet
Tue Dec 23 18:43:40 UTC 2014  : Routine : Initializing Startup Scripts
Tue Dec 23 18:43:40 UTC 2014  : Routine : Set Interface Roles
Tue Dec 23 18:43:40 UTC 2014  : Routine : Setting Optional Interface driver flags if supported
Cannot set device flag settings: Operation not supported
Cannot set device flag settings: Operation not supported
Tue Dec 23 18:43:40 UTC 2014  : Routine : Check Configs
Tue Dec 23 18:43:40 UTC 2014  : Routine : Updating Configuration
Tue Dec 23 18:43:40 UTC 2014  : Routine : Initializing Wanos Click
wanos.click:12: While initializing 'lan0 :: FromDevice':
  warning: lan0: no IPv4 address assigned
wanos.click:11: While initializing 'wan0 :: FromDevice':
  warning: wan0: no IPv4 address assigned
Debug : Building datastore: 0
Debug : Prepare Ready: 0
Debug : Probe On
DropBroadcasts: dropped a packet


Site B
Tue Dec 23 17:47:58 UTC 2014  : Info : Cleared Statistics and Logs
Tue Dec 23 17:48:14 UTC 2014  : Info : Clearing the datastore
Tue Dec 23 17:48:14 UTC 2014  : Info : Reboot Requested
Tue Dec 23 17:48:25 UTC 2014  : Routine : Initializing Startup Scripts
Tue Dec 23 17:48:25 UTC 2014  : Routine : Set Interface Roles
Tue Dec 23 17:48:25 UTC 2014  : Routine : Setting Optional Interface driver flags if supported
Cannot set device flag settings: Operation not supported
Cannot set device flag settings: Operation not supported
Tue Dec 23 17:48:25 UTC 2014  : Routine : Check Configs
Tue Dec 23 17:48:25 UTC 2014  : Routine : Updating Configuration
Tue Dec 23 17:48:25 UTC 2014  : Routine : Initializing Wanos Click
wanos.click:11: While initializing 'wan0 :: FromDevice':
  warning: wan0: no IPv4 address assigned
wanos.click:12: While initializing 'lan0 :: FromDevice':
  warning: lan0: no IPv4 address assigned
Debug : Building datastore: 0
Debug : Prepare Ready: 0
Debug : Probe On
DropBroadcasts: dropped a packet
Tue Dec 23 18:43:29 UTC 2014  : Routine : Initializing Startup Scripts
Tue Dec 23 18:43:29 UTC 2014  : Routine : Set Interface Roles
Tue Dec 23 18:43:29 UTC 2014  : Routine : Setting Optional Interface driver flags if supported
Cannot set device flag settings: Operation not supported
Cannot set device flag settings: Operation not supported
Tue Dec 23 18:43:29 UTC 2014  : Routine : Check Configs
Tue Dec 23 18:43:29 UTC 2014  : Routine : Updating Configuration
Tue Dec 23 18:43:29 UTC 2014  : Routine : Initializing Wanos Click
wanos.click:12: While initializing 'lan0 :: FromDevice':
  warning: lan0: no IPv4 address assigned
wanos.click:11: While initializing 'wan0 :: FromDevice':
  warning: wan0: no IPv4 address assigned
Debug : Building datastore: 0
Debug : Prepare Ready: 0
Debug : Probe On
DropBroadcasts: dropped a packet


ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #6 on: December 23, 2014, 07:20:24 PM »
So far all good.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #7 on: December 23, 2014, 07:31:15 PM »
so peer status will still be down when UDPENCAP is enable ?

My idea of deployment it to optimize vpn traffic in production.
e.g. - client from Site A will establish vpn over Pfsense's internal OpenVPN to DC. (both sides have wanos)

If all looks good, I'll go ahead and implement the same over production network (over actual WAN) and see how it goes.

In case I run into problems, shall I continue on same post or start a new one?

Many thanks for your instant assistance and support  :)



ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #8 on: December 23, 2014, 08:20:04 PM »
No problem, would be great to get it working.

Peers should be up, even with UDPEncap, but enough traffic is needed to trigger peer detection with UDPEncap. So in this case it could go easier in production.

Its fine, create a new thread if the configs, topology info is different.

Good luck.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #9 on: December 24, 2014, 03:22:32 PM »
Hello ahenning,

I deployed same setup on our production wan. UDPENCAP is enabled.
Established openvpn client connection form Site A to Site B and transferred 1GB of data. Peer status displays nothing, even Local Status is blank.

Also, In my local test environment peer status displays nothing, even Local Status is blank.

Kindly suggest.




astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #10 on: December 24, 2014, 04:49:52 PM »
did another quick test by replicating http://wanos.co/wan-optimization/virtual-wan-optimization-esxi-lab-guide.
Created a new switch (no adapters) and added vms. Created traffic by copying files between vms.

Still the peer status is "down" but some data for Reduction. Please refer the attached images.



ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #11 on: December 24, 2014, 04:54:22 PM »
Seems to be working as expected. Set your peer timeout from 60 seconds to say 120 or 180 seconds.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #12 on: December 24, 2014, 05:37:42 PM »
setting timeout to 180 did the job.  :)
Will continue testing by transferring files and then switch back to production wan setup that includes pfsense

ahenning

  • Team Wanos
  • Administrator
  • Full Member
  • *****
  • Posts: 629
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #13 on: December 24, 2014, 06:04:39 PM »
Ok great, we'll introduce an 'idle' state for the peers when there is no traffic between them, but neither side is completely down.
CCIE RS, CCIE SP, Mnet&sys

Note: Forum posts may be outdated. Please see the latest documentation at wanos.co/docs

astar

  • Member
  • ***
  • Posts: 11
    • View Profile
Re: wanos + ESXi with pfSense NAT
« Reply #14 on: December 24, 2014, 07:32:17 PM »
On production wan it's still not working. Following is my config
HOLD=600
UPDENCAP=Enable
UDPPEERRATE=0.1

After connecting to machine at SiteA over RDP, openvpn client is started to access remote machine at SiteB.
Openvpn configuration changed to TCP from UCP and no compression.
Transferred 10GB files over vpn connection, still all columns under peer status page are blank.