Expanding on the Exchange to DC processor Ratio's post

This post focuses on two aspects of Active Directory (AD) design for Exchange 2007: 

• The recommended ratio of Exchange 2007 servers to Active Directory global catalog (GC) servers
• When to use dedicated Active Directory sites for your Exchange 2007 servers

While most of the existing guidance about AD design for Exchange 2003 applies to Exchange 2007, there are several new factors that must be considered:

• Exchange 2007's new 64-bit architecture
• Availability and mix of both 32-bit and 64-bit AD servers
• New (or upgraded) services and clients (Unified Messaging, anti-virus/anti-spam, Outlook 2007, etc.)
• New role separation and architecture for Exchange 2007 servers (Mailbox, Hub, Client Access Server, Edge, UM.)
• More use of advanced features by typical users (mobile devices, calendaring, etc.)
• Increasing average mail traffic and storage allowances
• Additional message handling (anti-spam/antivirus, compliance features) requires more GC network bandwidth and processor utilization per user/message

How Many Global Catalog Servers Do I Need for My Exchange 2007 Servers?

The previous recommendations for Exchange 2003 are available at:

Global catalog server placement and ratios in an Exchange 2000 Server organization or in an Exchange Server 2003 organization

The primary recommendation in the above article is that you should have 1 Global Catalog processor core per 4 Exchange server processor cores. It is important to note that this is not a 1:4 ratio of servers or even processors, but of processor cores: a dual core processor counts as 2 when doing the ratio calculations.

With Exchange Server 2007, Microsoft recommends that you deploy one 32-bit GC CPU core for each four Exchange 2007 mailbox server CPU cores.  While the other server roles will influence the number of GC cores required, the Mailbox servers that are deployed influences the deployment of each of the other roles, so basing the number of GC cores on mailbox server cores will suffice.

For example, suppose you have an 8 processor (single core) Exchange 2003 mailbox server. This Exchange server's Active Directory needs can be satisfied by two single-processor/single core GC machines, by one dual-processor/single core machine, or by a dual-core single-processor GC.

The Exchange 2003 recommendations have been based on domain controllers running 32-bit Windows 2003. Something wonderful happens when you upgrade your domain controllers to 64-bit Windows 2003: efficiency can double!

Instead of a 1:4 ratio, you can have a 1:8 ratio between Global Catalog processor cores and Exchange server cores. This assumes that you fulfill one critical requirement: You must have enough RAM installed on the 64-bit server to cache the entire Active Directory database in memory.

To check the size of your Active Directory database, look on a global catalog server for the NTDS.DIT file. This file is installed by default in %WINDIR%\NTDS.

On a 32-bit machine, no matter how much RAM you install, you're unlikely to be able to cache the whole database unless it is less than 1.5-1.7 GB in size. This is because of inherent memory addressing limitations in the 32-bit platform. For more details about why this is so, see the article links below.

On 64-bit Windows, with its massively larger address space compared to 32-bit, you can cache even very large Active Directory databases entirely in RAM. This gets you a significant speed boost in responding to queries along with a dramatic reduction in expensive disk I/O.

(If you're thinking, well, my AD database is a couple of gigabytes in size, but what if I were to use the /3GB switch to get some extra 32-bit address space for cache? Theoretically, yes, you can increase the amount of cache this way. As a practical matter, you would likely only get about 2.5-2.7 GB consistently in cache, and that would come at the price of system stability. Use of the /3GB switch on domain controllers is discouraged because it is likely to starve the machine of critical kernel resources. When this happens, the typical result is that after several days or weeks of operation, or after a peak in client demand, the server becomes unstable or unresponsive.)

When you upgrade to 64-bit domain controllers, be sure you install enough RAM to cache your entire Active Directory database with some room to spare. Remember: if you don't install enough RAM, your performance will be more like a 32-bit domain controller.

Keep in mind that these ratio recommendations are rules- of-thumb, not guaranteed numbers. Your environment may have unusual loads or configurations.

Also, if you keep using 32-bit domain controllers with 64-bit Exchange Mailbox servers, you may skew the standard 1:4 recommendation. The ratio recommendation assumes that the DCs and the Exchange Mailbox servers are both on roughly equivalent hardware. But a 64-bit machine is likely to have more advanced processors and other components (busses, memory, etc.). If using older domain controller machines, you may need to lower the ratio. Standard benchmarks, such as the SPEC benchmarks (http://www.spec.org/) , can give you a more realistic basis for comparing older to newer machines.

The Performance Tools included in the Toolbox section of Exchange 2007's Exchange Management Console application can be very useful in reporting actual performance and finding bottlenecks. For more detailed recommendations, including lists of performance counters to monitor, take a look at these blog posts and white papers:

Measuring AD Performance (Exchange Team Blog post)
Active Directory Performance for 64-bit Versions of Windows Server 2003 (Windows white paper)
Exchange 2007 Processor and Memory Recommendations (Exchange Team Blog post)
Microsoft Windows Kernel Memory Management and Microsoft Exchange Server (Exchange Team Blog post)
Best Practice Active Directory Design for Exchange 2000 (Exchange white paper still highly applicable to Exchange 2007)
Planning a Microsoft Exchange Server 2003 Messaging System (Exchange white paper also a good resource that mostly applies to Exchange 2007)

Regardless of your GC processor platform, you should expect some increase in network traffic between GCs and Exchange 2007 servers. This increase may be sizeable if you are taking full advantage of all the new features and roles in Exchange 2007 (antivirus, anti-spam, legal compliance features, unified messaging and so on). At Microsoft, when all of these features are heavily used, the network traffic between AD and Exchange nearly doubles. While this is a substantial increase, it has not had a large practical effect. Historically, network bandwidth has not been a common bottleneck between Exchange and Active Directory. Nonetheless, you should check your current utilization and be sure you have enough network bandwidth between Exchange servers and domain controllers as you deploy additional Exchange 2007 features.

Should I Use Dedicated Active Directory Sites for My Exchange 2007 Servers?

The short answer to that question is, No, not if you're creating a dedicated site only for the purpose of domain controller load balancing. This is a different answer than for Exchange 2003, so it needs some explaining.

In Exchange 2003, message routing and Active Directory site topology were independent of each other. The primary reason to dedicate an Active Directory site to Exchange was to prevent other services and applications from hogging domain controllers needed by Exchange. (Or, to take a less Exchange-centric view of the universe, to prevent Exchange from hogging the DCs needed by other services.)

In Exchange 2007, message routing maps directly to Active Directory site topology. Your AD site topology should therefore make sense for message routing, not just for domain controller load balancing. You should avoid creating an Exchange-dedicated Active Directory site unless doing so makes routing better too.

Fortunately, the extra headroom available with 64-bit GC servers makes it easier for demanding applications to coexist in the same site, further reducing the motivation to use dedicated sites for Exchange 2007.

Let's define a "demanding application" as one that:

• Continuously generates a large number of LDAP queries
• Requires ANR (Ambiguous Name Resolution) to resolve a good percentage of those queries, and
• Makes frequent authentication requests against domain controllers

If you have multiple demanding applications running in a site, you must think not only about their average loads, but also about their peak loads, and the possibility that all of those applications may all at once decide to pick on a single domain controller. As a rule-of-thumb, if you are using 32-bit domain controllers, then Exchange is likely to coexist happily with other demanding applications in a site with up to 10,000 Exchange users. If you are using 64-bit directory servers, you can support up to 20,000 Exchange users in the same site with other demanding applications.

Exchange does a pretty good job of load balancing its requests across the domain controllers available to it, including taking into account how busy the DCs are with other work. So Exchange is not likely to be the site bully. But it is a demanding application and will present steady, relatively high load across all the GCs available in the site.

If you are above the peak limits described above, you should look at ways to dedicate directory resources to Exchange. That does not necessarily mean isolating Exchange to another site:

• You can try to reserve domain controllers for Exchange use by being clever with DNS. By appropriately configuring SRV record priorities and weights, you can often get nearly the same effect as a dedicated site.

More about how to do this can be found here:

• Creating an Active Directory Site for Exchange Server (Ignore the first part of this paper and scroll down to the part about how to "Isolate Client Authentication Traffic from Exchange Facing Domain Controllers.")
• You can define a static set of domain controllers for use by a particular Exchange server (using the set-ExchangeServer cmdlet). This lets you keep Exchange from using domain controllers dedicated to other applications. While this is preferable to creating a dedicated site, it is not optimal from the perspective of ongoing management. You have to remember which GCs belong to whom and manually tune and maintain lists instead of letting Exchange automatically optimize the load balancing. 

If another application cannot be prevented from overwhelming the domain controllers Exchange needs, then investigate your options for restricting or isolating that application before deciding to protect Exchange in a dedicated site.

If you have more than five Active Directory sites in your environment try very hard not to put Exchange 2007 in a dedicated site. In an environment more complex than five sites, it's unlikely that you will be able to design a dedicated site topology that works well for routing. Yes, you can then wrestle the Exchange 2007 routing topology into the shape you want without regard to the AD site topology. But you don't want the ongoing headache of managing and maintaining a message routing topology and an AD site topology that don't mesh together.

Exchange and Active Directory are deeply entwined. Exchange 2007 takes advantage of this synergy to simplify message routing and make it more deterministic (and easier to troubleshoot). Let that work in your favor. If your Exchange routing needs are completely at odds with your AD site topology, it may be time to rethink both together.

To summarize all this:

• 64-bit Active Directory servers are great news for Exchange and can double the performance of 32-bit servers.
• Take advantage of the doubling in performance you can get with 64-bit Active Directory servers. You can reduce the number of AD servers required to support Exchange and simplify both your Active Directory site and Exchange message routing topologies.