MSCluster: Low free PTEs caused cluster service to disconnect

Back in the days when a lot of us are not sure what the 3GB switch really does and thought its must be set so that Windows can recognise 4GB RAM and above, a number of our application servers has been set with 3GB switches in their servers. This is one of the servers.

The other day one of the application cluster suddenly failed over. A quick check on the servers’ eventlogs show not issues with low non-paged pool memory or memory issues nor any network issues. The application logs was rather clean, other than a strange repetitive event from the MOM agent. We have a keep alive event which the MOM agent runs once a day, but I was seeing the same event running 2 twice every minute. On another server when had the same program we could see that the idling node had cpu running at 20% or more. Once we stopped the MOM agent, the cpu dropped to almost idle.

Windebug: 3GB and PAE switches

Caught a few interesting blog talk about myths and misunderstanding of the 3GB and PAE switches.

Myth: PAE increases the virtual address space beyond 4GB

This is another non sequitur. PAE increases the amount of physical memory that can be addressed by the processor, but that is unrelated to virtual address space. (Remember that PAE stands for Physical Address Extensions.)

PAE increases the physical address space (the address space that the CPU can use to access the memory chips on your computer) from 32 bits to 36 on a Pentium 2, for a theoretical maximum physical memory capacity of 64GB. However, the size of a pointer variable hasn’t changed. It’s still 32 bits (for a 32-bit processor), which means that the virtual address space is still 4GB.

The oft-misunderstood /3GB switch

It’s simple to explain what it does, but people often misunderstand.

The /3GB switch changes the way the 4GB virtual address space is split up. Instead of splitting it as 2GB of user mode virtual address space and 2GB of kernel mode virtual address space, the split is 3GB of user mode virtual address space and 1GB of kernel mode virtual address space.

Consequences of running 3GB and PAE together

As of Windows 2003 SP1 you should more than likely use /userva with /3GB if you need to use /3GB at all. I would choose a values around 2970 that would give you over 20,000 sysPTES’s free, which should allow the system to function without tipping over, but this is something that you’ll need to “tune” for your particular app. You also have to keep in mind that you’re reducing the space for the pools as well.

Really what you should do is reevaluate why you’re running 3GB in the first place, perhaps what you’re really looking for is PAE/AWE and just have 3GB in place by mistake. Also you should ask yourself, why am I NOT running an x64 hardware and OS?