Comments for Jesper On Stuff

Comment on Google App Engine by Alexey Rusakov

Alexey Rusakov — Tue, 08 Apr 2008 10:12:22 +0000

I thought it was something similar to Amazon’s S3 and EC2 services, but after watching the sample app demo it looks its more like online application sandbox.

They only support python and a fairly simple webapp framework, but more is to come.

Comment on Understanding ASP.Net memory by Maximus

Maximus — Thu, 20 Dec 2007 10:54:42 +0000

I would like to see a continuation of the topic

Comment on Understanding ASP.Net memory by jesperfj

jesperfj — Mon, 15 Oct 2007 09:53:16 +0000

Hi James
Interesting observation. Do you run anything else than IIS on these servers? I am curious about what proces is actually using the CPU. I would set up three counters in perfmon on both servers:
1) Total CPU usage
2) CPU usage of w3wp.exe process
3) .Net CLR memory > % time in GC

By comparing 1) and 2) you should get an idea if is Asp.Net using the memory or “other stuff”. As the kernel got its memory reduced to 1 GB, it may be more CPU heavy.

3) Shows how much CPU the .Net garbage collector uses. Now that you have increased the memory for .Net, the garbage collector should be more busy collecting garbage from this memory. You can also set up perfmon counters to see how many GC collects are performed.

If you got rid of the OOM exceptions on the 4GB server, you may find it acceptable that it performs slightly worse than the other server because you never know when OOM will show its ugly face to the users and the result is usually a broken website. If recycling the App pool once a day avoids OOM exceptions, i very much prefer this to recycling at random times of day when memory is full. But I recommend a balanced strategy where you recycle once a day, but also have a memory limit where to recycle just in case you are getting close to a OOM. But it should be the exception that this occurs. I assume you add data to the Asp.Net cachesince you run out of memory. If you have set up any limits in your code where you stop to add more to the cache, you actually have a so called “self inflicted” memory leak. You can therefore avoid the OOM’s if your code stops adding to the cache at some limit.

The Sitecore CMS is heavily relying on caching, but still in general our customer websites run for days, weeks and months without hitting memory limits or experiencing OOM’s. This has been obtained by monitoring the processes “Private bytes” in our code, and stop adding to the cache at configured thresholds.

Notice: I agree that recycling the system service should not be the norm. But your code may keep pointers to the memory, which is e.g. the case if you store something in a (in process) session variable. The garbagfe collector will NEVER be able to collect this memory because it is in use. Therefore, at some point you have to prioritize what data to cache in memory.

Brgds. Jesper

Comment on Understanding ASP.Net memory by James

James — Sat, 29 Sep 2007 06:40:16 +0000

WE have 2 servers w/2GB physical RAM both are idetical.
We were having OutOfMEM exceptions.

We added 2GB to one to make 4GB and added the 3/GB switch.

The servers host ASP.NET xml web services that use IIS / http for transport.

The server with 2GB is slightly outperforming the server with 4GB in terms of response times.

Also, the 4GB server is using much more CPU than the 2GB Server to do the same job.

With using the 3GB switch, the kernel is now limited to 1GB.
Could there be an issue with http.sys queuing up since it runs in kernel mode?

We recycle the App pool on a schedule once a day. All other settings on the App pool are
default.

Should we try tuning with recycling maxed used mem: 800?

Is it true that recyling using this method is standard practice?

Recycling a system service to manage the Application doesnt sound as good as the Application
performing Garbage collection on its own.

Can you give some advice?

Thanks,
James

Comment on Multiple Application Pools impact on Virtual Memory and Pagefile Size by jesperfj

jesperfj — Mon, 24 Sep 2007 12:18:28 +0000

Hi Christian
Yes, these are Sitecore sites. However the focus of the article is more on what happens to the server environment at extreme memory loads rather than going into details of Sitecore. One should’t put too much into the actual numbers of sites that we cound run in this test as several conditions are artificial. E.g. the number of hits/site/minute was taken out of blue air, and as you also point to, we didn’t stress the server with requests to css, javsacript and image files during these tests.

But if you are going to perform similar tests on your own, you should be able to find some gold nuggets that are not too obvious at first look.

/Jesper

Comment on Multiple Application Pools impact on Virtual Memory and Pagefile Size by Christian

Christian — Mon, 24 Sep 2007 08:57:26 +0000

Interesting post

You do not write what kind of asp.net website you are doing test against – is it Sitecore sites?

Most websites use image, css, javascript etc. is this also including in your test?
Most files will be cached on the client, but the will still make a hit on the server with the E-tag, so each of your *.aspx request should make 15 extra hits to static files

Comment on Understanding ASP.Net memory by jesperfj

jesperfj — Sat, 11 Aug 2007 16:10:58 +0000

Hi Raj
You should check if it is “OutOfMemory” (OOM ) OR “memorylimit Exceeded” exceptions that is thrown when you get to 1.8GB. Assuming it is “memorylimit Exceeded”, you could increase the memoryLimit. That would be in IIS manager on IIS 6.0, where you should find the “Maximum Memory…” and adjust it to whatever you find right.

Notice however that 1.8GB is the recommended memoryLimit when using the 3GB switch. Without the switch the recommendation is only 800MB. But you can of course increase the limit above 1,8GB and see what happens. But the risk of receiving OOM exceptions will increase.

You could also see if you can decrease the memory usage by storing it outside the ASP.Net process. If you start new IIS website with its own application pool on the same server, this application pool will get its own 3GB virtual memory – giving you 3+3 = 6GB virtual memeory on one machine. However there is no such thing as a free meal, and boosting virtual memory like this will increase disk swapping by Windows memory management.

What kind of exception is it you receive, and what is the memoryLimit set to?

Comment on Understanding ASP.Net memory by Raj

Raj — Sun, 08 Jul 2007 14:23:29 +0000

Hi jasperfj,
Thank you very much for the information. Its been a very useful one. The myths about /3GB are now unravelled.
I have a typical problem facing. I have a web server farm of about 36 servers (32 bit) and in each server the RAM size is 4GB. As you said earlier, since the 3GB switch is not enabled the kernel uses 2GB and other applications can use another 2GB.
The problem is that my ASP application running on the IIS 6.0 is reaching its peak of 1.8 GB memory utilisation very frequently. i thought if i could change the boot.ini file by adding 3GB switch, the application will be able to use more virtual memory. Hoping that this may increase maximum limit for memory utilisation. But there wasn’t any success. I just rebooted the machine after adding 3GB switch. Should i be following any other steps to achieve what i am looking for ??
Can you please suggest me how do i increase the virtual memory limit from 1.8GB to more. (P.S The application is an ASP application)

Comment on Understanding ASP.Net memory by jesperfj

jesperfj — Fri, 15 Jun 2007 08:41:56 +0000

Hi Adam
If you start up perfmon to show “Process\Private Bytes” as well as Virtual Bytes you should see that private bytes stay below the 800Mb you stated in Max Used Memory. What are these values when you experience OOM’s? First of all you should double check that it really is OOM exceptions you experience.
The “memoryLimit exceeded” (MLE) exception could also cause problems, and it is more or less so that whenever you decrease the MaxUsedMemory, you also decrease the risk of OOM’s but INCREASE the risk of MLE’s and vice versa. Therefore it is important to be certain what exception you actually experience.
With your setup (MaxUsedMemory=800 and 3GB switch) you have 2.2 GB memory free for the tasks that usually cause OOM’s. So if your code uses up this 2.2GB it is a bit extreme. You can download a free trial of a memory profiler at memprofiler.com that can tell you about anything about the current memory status of your process. It should enable you to find the cause of the OOM’s.
When you set Max Used Memory to 400Mb, I am almost certain that process recycles do to memoryLimit Exceeded. If you observer private bytes in perfmon you should be able to see it increase up to the 400MB limit, and then suddenly drop as the process is recycled. What is the highest “Private Bytes” you observe when you set Max Used Memory to 800MB?
It sounds like you may have a “memory leak” somewhere in the code, that needs to be fixed in order to get rid of the recycles. If you have such a leak, it is only a question about time before you run into trouble.
Feel free to write again.

Comment on Understanding ASP.Net memory by Adam

Adam — Fri, 15 Jun 2007 06:42:30 +0000

Im running iis6 with win server 2003 with /3gb switch in boot.ini and 4gb ram.
I have set the Max Used Memory to 800mb
and im getting OOM exceptions fairly frequently.

Ive tried changing it to 400mb, but now i get the process recycling every couple of minutes..

The above looks and sounds great.. but for the life of me i cant get the process to recycle at a resonable frequency per day.

any tips?