Deploying new solutions in SharePoint 2013 takes too much time

One of my customers is having TEST, QA and PROD environment. Where QA and PROD are equal (same amount of WFE servers (in this case 2 WFE servers), same configuration and settings, in case PROD is down we can switch to QA in a minutes and it will be the same). 

What happened: since a while deploying a new version of the solutions takes too much time, around 1.5 hours, compared with the past and QA - 15 min. It is horrible to stay in front of the monitor and wait the slow deployment to finish. Yes, it finished without errors but waiting was killing me. I wasn't aware of any changes in infrastructure level that can lead to this slowness. 

I have to mention that there wasn't any slowness on the sites and locally on the server everything was working fine, no high memory or CPU consumption, nothing that could tell you way the deployment is so slow. 

In QA environment the deployment time was normal, around 15-20 min. So this also exclude the .wsp's from the suspected root cause.

Start looking for a solutions in internet and I was surprised how many article and post related to SharePoint slowness, sites are loading too slow and etc. but nothing related to my case. That's why I decided to write this post because I was able to find the root cause.

Incidentally I opened two RDP sessions to the WFE servers and size of the windows was different, guess what:

Can you see where is the problem from the picture above?

Yes, that is right. 

Both servers are having different time, the second server was 5 min behind the real time.

OK potentially I thought this could be the reason and went to check the time zones, but on both servers the time zones were correct, but not the time. After a while I checked the services and found out that "Windows Time" services on both machines were set to disable. Check how it is QA and there the services were running. Enabling the services and set them to automatic was the fix I needed.    

In the future if you have a slow deployment make sure you check the "Windows Time" service, set it to automatic and run it.

Hope you will find this post helpful, if so please share it.

Enjoy!