Unable to Create File Services Domain Error When Enabling vSAN File Services

Recently I’ve been looking for ways to shrink the footprint of our File Servers and NAS devices at our ROBO sites. So when vSAN 7.0U3 announced access based enumeration for SMB file shares, it sounded like a good time to give it a try.  The set up was pretty straight forward… Enable the service, fill in some IP and AD info and your off. I’m not going to bore any of you with that. Mostly because VMware does a great job explaining it in their documentation (Here’s the Link). What I am  going to talk about is an issue that sent me down a rabbit hole for way longer that it should have and ended up being a very simple fix.

 Cannot complete the operation. See the event log for details. Connection to AD server cannot be established from information in FS domain configuration. this problem can be caused by one of the following issues: 1. Firewall configurations are incorrect. Check the open server port list in the link below. https://support.microsoft.com/en-in/help/179442/how-to-configure-a-firewall-for-domains-and-trusts 2. cannot resolve DNS name for the DNS server and DNS suffix. 3. AD network cannot be reached form the file server network. 

First off that error is a mouthful and looking back at it… I should have been able to pick it apart and figure out the fix faster but as most of us do, the easy solution got over looked for way too long.

Spoiler Alert….  I needed to double check my Sites and Services configuration.

So lets break this error down :

1.  Firewall configuration not correct –  I didn’t think this was the case, but I entrained the idea since this was a ROBO deployment and it wasn’t outside the realm of possibility since the DC was hosted off site at a larger facility.  Eventually after 20-30 minutes testing and teams chatting with the network team I decided the firewall was not the issue.  
2. Cannot resolve DNS server and suffix. –  This was actually the first thing I checked and was able to rule this out pretty quickly with some quick nslookup from the hosts.
3. AD network cannot be reached from the File server network. –  Again this was quick to rule with a simple ping of the DC from the host. 

In the end the problem was an error on my part because I didn’t check what site my ROBO was linked to for Sites and Services.  I had the DNS pointed to the wrong site for AD and because it wasn’t linked in sites and services to that site it timed out every time it tried to create the FS domain.  After disabling the service and re-enabling it with the proper DNS for the site link it worked flawlessly.

One final note: If you do find yourself in this situation make sure you fully disable and try it from scratch. I couldn’t get it to update successfully without doing a disable/enable.