Pages

Friday, November 25, 2011

Sporadic delays in BeginRequest of ASP.NET webpages

I've been investigating why some of our ASP.NET webpages sporadically take 10-20 seconds to load, when on average they take 500-700 ms.


The NewRelic instrumentation shows that the delay always happens due to uninstrumented code in BeginRequest.

13,812
100.00%
System.Web.HttpApplication.BeginRequest()0.000 s
0.0
0.00%
13 fast method calls0.000 s
0.0
0.00%
Aptimize.WebAccelerator.OptimizerModule.OptimizerModule_BeginRequest()0.000 s
0.0
0.00%
ASP.global_asax.Application_BeginRequest()0.000 s
0.0
0.00%
System.Web.Security.FormsAuthenticationModule.OnEnter()0.000 s
0.0
0.00%
System.Web.Handlers.ScriptModule.AuthenticateRequestHandler()0.000 s
0.0
0.00%
System.Web.Security.DefaultAuthenticationModule.OnEnter()0.000 s
0.0
0.00%
System.Web.Security.RoleManagerModule.OnEnter()0.000 s
0.0
0.00%
System.Web.Security.AnonymousIdentificationModule.OnEnter()0.000 s
0.0
0.00%
System.ServiceModel.Activation.HttpModule.ProcessRequest()0.000 s
0.0
0.00%
System.Web.Security.UrlAuthorizationModule.OnEnter()0.000 s
0.0
0.00%
System.Web.Caching.OutputCacheModule.OnEnter()0.000 s
0.0
0.00%
System.Web.Routing.UrlRoutingModule.OnApplicationPostResolveRequestCache()0.000 s
0.0
0.00%
System.Web.HttpApplication+MapHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()0.000 s
0.0
0.00%
System.Web.Routing.UrlRoutingModule.OnApplicationPostMapRequestHandler()0.000 s
13,578
98.31%
Uninstrumented time 0.000 s
0.0
0.00%
4 fast method calls
After some digging around, I found a few articles that hint this could be due to deadlock issues with Microsoft session module implementation. When you have a page, containing several iframes, these iframes can be concurrently requesting the same session ID from a session state resulting in a deadlock and a delay.

There's a good StackOverflow discussion right here which Demis Bellot pointed me to:
What they recommend to do is to reimplement the session module using MSDN's
custom session handler class (http://msdn.microsoft.com/en-us/library/system.web.sessionstate.sessionstateutility.aspx). The only problem is that that class is not thread safe.

So I started tweaking the class and came up with the following source code, which you can download from: http://www.cs.yale.edu/~yampolsk/SessionDemo.rar
. I am planning to put it into Production next week and measure the results.
Simple Apache benchmarking of default MS session handler and this one, result in almost identical performance.





Reactions:

10 comments:

I'm experiencing something similar. What were your results with your code?

The results were positive. It helped.
We had to further tweak the session module to persist the data in a global distributed cache (otherwise it didn't get shared between servers). But the timeouts in beginrequest got eliminated.

Would you mind setting up a small project on e.g. GitHub with the source code so that the community can give coordinated feedback and maybe fix/improve your solution in case of any problems or shortcomings? That would be a great move, I think :-)

Thanks anyway for writing this up. I also ran into this problem only thanks to New Relic, by the way.

Does this work with mode="StateServer"? We don't need cross-server state management but we don't want state disappearing every time we update our code, so that's why we use the built in state server. I looked at the code and I can't see any way this version explicitly supports StateServer, so I was wondering if maybe I'm just missing some implicit support?

Note, AngiesList implemented a custom session module using REDIS distributed cache:
https://github.com/angieslist/AL-Redis

You could try using that for cross-server state management.

Hello Alaxander,

Thanks a lot for the nice post! We also see similar pattern in our New Relic for our ASP.NET application. I need your expert views on following things:

1. Our application is using "System.Web.SessionState.SessionStateModule" -- I think this is the default one for .NET correct? And this also has same issue what you have described in this post? Our config. is as below:


2. What's your inputs fro Apache Benchmarking? As per my understanding with your improved logic Req/Sec should increase and Response time should decrease -- correct? Or any other things to monitor?
3. This issue exists for all ASP.NET applications? Or any specific ASP.NET version? This issue applies to all heavy request loaded ASP.NET application only correct?

Keenly waiting for your expert views!

Thanks again,
Kiran

Tks for the post and code.
I try to use it in my project. i added the relevant code from your download, compiles well, and added:


to httpmodul section.
It works... but when i put breakpoints all over your code, it never hits, so it looks for me, as it is not used in my project, means, the session runs with normal asp.net session inproc implementation. can that be ? do i miss something additional to the webconfig ?

This comment has been removed by the author.

Thanks for article. However it is not clear if this module is going to work with standard MS SQL Session provider. Module should call provider, right? Could you clarify please?

Thanks for your crystal clear explanation.

prathap
dot net training in chennai

Post a Comment