We have a lab which we can access by using a VPN (Cisco ASA and Cisco AnyConnect). This setup has a so-called split DNS configuration, which means that only resources in the lab are accessed through the VPN tunnel. Regular Internet traffic uses my local DSL connection.
At my house I (like most folks) rely on DHCP for providing me with IP address, gateway and DNS servers. My local subnet uses 192.168.10.1 for DNS and 192.168.10.254 is my default gateway. So my clients are in the same subnet as my DNS server (directly-connected).
All these things considered I should be able to browse the Internet while I have a VPN running. Well, that's where you're wrong.
As soon as I start the VPN connection, all Internet access is gone. At least everything related to resolving goes wrong. I can ping the Google DNS servers (8.8.8.8), but there's no way of getting a single ping out to e.g. www.google.com.
Some investigation showed that we have the same subnet in our lab, and that traffic destined for my locally connected DNS server is going through the tunnel (and 192.168.10.1 is not a DNS server). That's strange, because according to the VPN policy, access to local resources is allowed, AND split DNS is enabled.
OS X routing table with the VPN enabled.
192.168.10 is my local network and exists in the lab.
Since this all happens on MacBook I thought that it was some sort of OS X thing. I know that Apple iOS is also a bit weird in regards to VPN, DNS servers, etc. until I accessed the lab from my iMac.....
My iMac functioned as one should expect. After enabling the VPN, I was able to access Internet through my own DSL connection (checked by accessing whatismyip.com), and access the lab at the same time. Since the iMac and the MacBook both run OS X El Capitan, it must be some configuration thing.
The basic differences between the MacBook and the iMac is that the iMac is connected by cable, and the MacBook uses wireless, but both are connected to the same IP network.
After a couple of hours searching the Internet in regards to OS X routing tables, Cisco bugs etc. I decided to manually compare all settings of my iMac and MacBook. That's where I found a very small difference that makes no sense at all.
On my iMac I use DHCP for my IP address, but for some reason I added a fixed IP address for my local DNS server. It's the same address I get when using DHCP. Only difference is that I manually entered it. My MacBook gets its DNS server through DHCP.
The following screenshots show the difference between the two.
The screenshot of the network settings on the left is from a DHCP client where the DNS server is configured by DHCP. You cannot select the IP address. The screenshot on the right has the same DNS server configured by hand. This time you can select / highlight the IP address, and on the bottom the minus symbol becomes active so you can remove the manually configured DNS server.
When I statically assign the DNS server everything works fine, but if the DNS server is assigned by DHCP, the resolving goes through the tunnel to 192.168.10.1 in the lab. This occurs on wireless and on the cable, so for some reason the OS X platform (and VPN software combination??) treat DNS by DHCP and statically assigned differently.
I have no idea what's causing this. I'm glad I figured it out eventually, and since these (wireless) settings are SSID related I have no problems with a fixed DNS server when I'm access other SSID's / wireless networks and locations (on my MacBook). And since my iMac doesn't 'travel' that much, so I expect no problem there as well.
So in the end, local / Internet resolving is done by querying my local DNS server, while lab resources are resolved through the DNS servers in the lab.
One thing I need to check is what happens when the 192.168.10.0/24 network is no longer advertised as accessible through the VPN.