Friday, 7 June 2013

ASP.NET web app’s AppDomain constantly resets/recycles/reloads while hosted in IIS on the development machine with TFS (Team Foundation Server) enabled.

I’m currently developing a large web application at the moment.  Every time it starts, Application_Start fires and I load a few 100MB of data into memory which takes about 20-30 seconds to complete. This would be fine if it happened a few times per day, but for me, it was happening every single time I checked a file out of TFS (to trigger a check-out in TFS all you have to do is start editing the file).

After a while, I realised that making only tiny changes to files, such as static html or javascript files would cause the entire app domain to recycle. This meant that it would take up to 30 seconds to see a change every time I made a change.  It was ridiculous.

I went about trying to discover what was actually happening.  First of all I logged Application_Start and Application_End events into a text file every time it occurred:

   1: public string AppPath { get; set; }



   2:  



   3: protected void Application_Start(object sender, EventArgs e)



   4: {



   5:     AppPath = HttpRuntime.AppDomainAppPath;



   6:     LogAppStart();



   7: }



   8: protected void Application_End(object sender, EventArgs e)



   9: {



  10:     LogAppEnd();



  11: }



  12:  



  13: [Conditional("DEBUG")]



  14: private void LogAppStart()



  15: {



  16:     System.IO.File.AppendAllText(System.IO.Path.Combine(AppPath, "log.txt"), string.Format("{0}\t{1}\r\n", DateTime.UtcNow.ToString(), "APP START"));



  17: }



  18:  



  19: [Conditional("DEBUG")]



  20: private void LogAppEnd()



  21: {



  22:     System.IO.File.AppendAllText(System.IO.Path.Combine(AppPath, "log.txt"), string.Format("{0}\t{1}\r\n", DateTime.UtcNow.ToString(), "APP END due to " + System.Web.Hosting.HostingEnvironment.ShutdownReason));



  23: }




Then I used BareTail to track when and why exactly the AppDomain was being unloaded (you can just use a text editor if you like, but BareTail gives you changes as they happen, so I could keep it on my screen whilst using Visual Studio to see exactly what triggers the unload).



Image1



I found that it unloaded every time I edited a file (but did not save it).  I noticed that the only thing that happened sometimes was that the TFS “lock” icon would change to a “red tick” icon.  So I was leaning toward it being something to do with TFS from this point.



I needed to see exactly what was happening so used procmon from sysinternals.  Unfortunately, it gives a vast amount of information which needs to be filtered and sifted through.  After spending quite a long time going through the logs I discovered that the TFS check out process was creating an “app_offline.htm” file in the root of the web app.  It wasn’t really conclusive from procmon though.  It tends to give quite cryptic Win32 codes.  But it pointed me in the right direction.  I then used Directory Monitor to monitor the root application path.  It gave an audit log which confirmed that “app_offline.htm” was indeed being created and immediately deleted by TFS / Visual Studio when a file is changed that’s under source control.



App_offline.htm is usually used during deployments / publishing to tell IIS to take the application offline gracefully while an upgrade is performed (it shows a nice friendly message to the user, rather than some yellow screen of death error – very useful in the right circumstances).



So it seems the creation and instant deletion of this file may have something to do with it.



So how on earth do you get it to stop doing that?!



The app_offline.htm file is cached in



C:\Users[user]\AppData\Roaming\Microsoft\VisualStudio\11.0\ (replace [user] with your username)



I tried deleting the file… but then discovered Visual Studio will just auto-generate the file back into existence!  So that didn’t work.



However, eventually I found the best way to prevent problem is to prevent the creation of app_offine.htm at all. 



How?



Just delete the app_offline.htm file from C:\Users[user]\AppData\Roaming\Microsoft\VisualStudio\11.0\ , then cunningly create a directory called “app_offline.htm” inside C:\Users[user]\AppData\Roaming\Microsoft\VisualStudio\11.0\  



This causes the creation of app_offline.htm (the file) to silently fail in Visual Studio!



Problem solved! It’s a bit of a hack but it works.