Friday, February 25, 2011

Basic Powershell script to show appointments from a calendar using the EWS Managed API

One of the more common tasks you might turn to use one of the Exchange API's for is to enumerate appointments in one or more users calendars. This can be useful for a number of reasons and applications and scripts, the EWS Managed API makes getting calendar appointment pretty easy using a script once you understand a few of the fundamentals around how calendar appointment's are stored.

Appointments unlike messages are more complex objects eg where an email would only represent one object in a folder an appointment if its recurring or a recurring exception or a deleted occurrence could represent one more calendar appointments. So its important not just to do a finditems query like you would with Email messages as this will return just the base instances of each appointment object. What you should do when querying for calendar appointments is its important to use a CalendarView, what this means is that when you query for appointments you specify a date range (start and end) for the appointments you what to retrieve and EWS will handle expanding any recurring appointments,deleted occurrences etc and return you what is essentially what you would see in Outlook (which does the same thing). This is a rather simplified explanation if you want to read more check out this sample chapter from Inside Exchange Web Services which is available on MSDN.

So when it comes to putting all this together in Powershell script that will pull all the appointments from a specific calendar we first need to specify the SMTP address of a mailbox you want to access and a Date range you want to check in a few variables

$MailboxName = "gscales@domain.com"
$StartDate = new-object System.DateTime(2009, 08, 01)
$EndDate = new-object System.DateTime(2009, 11, 01)

or maybe you want to look at the appointment for next week

$StartDate = new-object [System.DateTime]::Now
$EndDate = new-object [System.DateTime]::Now.AddDays(7)

Connect to EWS using the currently logged on Credentials the assumes that this account has access to the mailbox is question.

$dllpath = "C:\Program Files\Microsoft\Exchange\Web Services\1.0\Microsoft.Exchange.WebServices.dll"
[void][Reflection.Assembly]::LoadFile($dllpath)
$service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService([Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2007_SP1)

$windowsIdentity = [System.Security.Principal.WindowsIdentity]::GetCurrent()
$sidbind = "ldap://<SID/=" + $windowsIdentity.user.Value.ToString() + ">"
$aceuser = [ADSI]$sidbind

$service.AutodiscoverUrl($aceuser.mail.ToString())

Then you need to specify the calendarview object to use

$folderid = new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::Calendar,$MailboxName)
$CalendarFolder = [Microsoft.Exchange.WebServices.Data.CalendarFolder]::Bind($service,$folderid)
$cvCalendarview = new-object Microsoft.Exchange.WebServices.Data.CalendarView($StartDate,$EndDate,2000)

and finally query the calendar and return the appointments

$cvCalendarview.PropertySet = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)
$frCalendarResult = $CalendarFolder.FindAppointments($cvCalendarview)
foreach ($apApointment in $frCalendarResult.Items){
$psPropset = new-object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::FirstClassProperties)
$apApointment.load($psPropset)
"Appointment : " + $apApointment.Subject.ToString()
"Start : " + $apApointment.Start.ToString()
"End : " + $apApointment.End.ToString()
"Organizer : " + $apApointment.Organizer.ToString()
"Required Attendees :"
foreach($attendee in $apApointment.RequiredAttendees){
" " + $attendee.Address
}
"Optional Attendees :"
foreach($attendee in $apApointment.OptionalAttendees){
" " + $attendee.Address
}
"Resources :"
foreach($attendee in $apApointment.Resources){
" " + $attendee.Address
}
" "
}

I've put a download of the script here.

Outlook Anywhere logon report powershell GUI

One of all the more useful reasons to learn scripting is the ability to turn information that is recorded in one format in a seemless useless rable of bits, into a format this is more useful to ordinary humans in everyday situations. One the examples of this are the IIS logs which in Exchange contain information about users accessing OWA, Accessing ActiveSync and Outlook Anywhere. Like everything in IT there are a few ways of tackling how you go about turing log information into something useful, one of the more popular ways to do this is using the Log parser which is a brilliant tool for those that aren't comfortable doing a lot of coding. You cant beat this tool for speed and efficiency and if your parsing logs often you should learn to use it. More recently the exLogAnalyser has been released which is interesting and it looks like a really great piece of coding that lacks a very very important ingredients for a tool like this. The documentation is spartan and it was written to solve a problem rather then help other people solve their own problems. (It has great potential you just need think about how people will use it!).

If you want to do something a lot more creative with the information your parsing out of log files this is when you need to do some serious number crunching and processing. The first really big problem when parsing a log file is that these files can get really large and reading a very large raw log file that contains millions of lines can be very slow if you want to process the information in everyline in a specific way. If done poorly it can mean your script will start consuming a lot of memory or maybe just take a lot of time. This is where using LogParser generally has a large advantage but the cost is what you can creatively do with the result so the trade off with using Powershell for very large log files is you get a lot of functionality but you pay a big cost in performance when you do it. The good thing however is that not everybody has that problem of very large log files (its really only if you have a large number of users) so for those that don't or dont mind spending some CPU cycles on processing then I've got a few scripts that you might find usefull. The parse engine is built on a previous post which works well in parsing the log file in a more reliable way. One goal I had for this script was to create something that would track logons so it would show me the logon/logoff time the duration and how many time a users logs on. To do this the script uses a hashtable and looks for the contiuned appearance of a certain entry from a user in a log file. If there is a gap of more then 30 mintues between the last entry it track this as a logon and if the users reappears in the sequence this becomes a new logon.

The script shows summaries or raw information and does time conversation to local time as well. I've posted a download of this script here the script iteself looks like the following.

[System.Reflection.Assembly]::LoadWithPartialName("System.Drawing")
[System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")
function openLog{
$exFileName = new-object System.Windows.Forms.openFileDialog
$exFileName.ShowHelp = $true
$exFileName.ShowDialog()
$fnFileNamelableBox.Text = $exFileName.FileName
Populatetable
}
#
function Populatetable{
$logTable.Clear()
$sumTable.Clear()
$fname = $fnFileNamelableBox.Text
$mbcombCollection = @()
$FldHash = @{}
$usHash = @{}
$sumHash = @{}
$fieldsline = (Get-Content $fname)[3]
$fldarray = $fieldsline.Split(" ")
$fnum = -1
foreach ($fld in $fldarray){
$FldHash.add($fld,$fnum)
$fnum++
}
get-content $fname | Where-Object -FilterScript { $_ -ilike “*MSRPC*” } | %{
$lnum ++
if ($lnum -eq $rnma){ Write-Progress -Activity "Read Lines" -Status $lnum
$rnma = $rnma + 1000
}
$linarr = $_.split(" ")
$uid = $linarr[$FldHash["cs-username"]] + $linarr[$FldHash["c-ip"]]
if ($linarr[$FldHash["cs-username"]].length -gt 2){
if ($usHash.Containskey($uid) -eq $false){
if ($linarr.Length -gt 0){$lfDate = $linarr[$FldHash["date"]]}
if ($linarr.Length -gt 1){$lfTime = $linarr[$FldHash["time"]]}
$ldLogDatecombine = $lfDate + " " + $lfTime
if ($ltimeconv.Checked -eq $true){
$LogDate = Get-Date($ldLogDatecombine)
$LocalLogDate = $LogDate.tolocaltime()
$ldLogDatecombine = $LocalLogDate.ToString("yyyy-MM-dd HH:mm:ss")
}
$usrobj = "" | select UserName,IpAddress,LogonTime,LogOffTime,Duration,NumberofRequests,BytesSent,BytesRecieved
$usrobj.UserName = $linarr[$FldHash["cs-username"]]
$usrobj.IpAddress = $linarr[$FldHash["c-ip"]]
$usrobj.LogonTime = $ldLogDatecombine
$usrobj.LogOffTime = $ldLogDatecombine
$usrobj.Duration = 0
$usrobj.NumberofRequests = 0
$usrobj.BytesSent = $linarr[$FldHash["sc-bytes"]]
$usrobj.BytesRecieved = $linarr[$FldHash["cs-bytes"]]
$usHash.add($uid,$usrobj)
}
else{
if ($linarr.Length -gt 0){$lfDate = $linarr[$FldHash["date"]]}
if ($linarr.Length -gt 1){$lfTime = $linarr[$FldHash["time"]]}
$ldLogDatecombine = $lfDate + " " + $lfTime
if ($ltimeconv.Checked -eq $true){
$LogDate = Get-Date($ldLogDatecombine)
$LocalLogDate = $LogDate.tolocaltime()
$ldLogDatecombine = $LocalLogDate.ToString("yyyy-MM-dd HH:mm:ss")
}
$duration = New-TimeSpan $usHash[$uid].LogOffTime $ldLogDatecombine
if ([INT]$duration.Totalminutes -gt 30){
$mbcombCollection += $usHash[$uid]
$usHash.remove($uid)
$usrobj = "" | select UserName,IpAddress,LogonTime,LogOffTime,Duration,NumberofRequests,BytesSent,BytesRecieved
$usrobj.UserName = $linarr[$FldHash["cs-username"]]
$usrobj.IpAddress = $linarr[$FldHash["c-ip"]]
$usrobj.LogonTime = $ldLogDatecombine
$usrobj.LogOffTime = $ldLogDatecombine
$usrobj.BytesSent = $linarr[$FldHash["sc-bytes"]]
$usrobj.BytesRecieved = $linarr[$FldHash["cs-bytes"]]
$usrobj.Duration = 0
$usrobj.NumberofRequests = 0
$usHash.add($uid,$usrobj)
}
else{
$usHash[$uid].LogOffTime = $ldLogDatecombine
$lgduration = New-TimeSpan $usHash[$uid].LogonTime $ldLogDatecombine
$usHash[$uid].Duration = [INT]$lgduration.Totalminutes
$usrobj.NumberofRequests = [INT]$usrobj.NumberofRequests + 1
$usrobj.BytesSent = [INT]$usrobj.BytesSent + [INT]$linarr[$FldHash["sc-bytes"]]
$usrobj.BytesRecieved = [INT]$usrobj.BytesRecieved + [INT]$linarr[$FldHash["cs-bytes"]]
}

}
}
}
$usHash.GetEnumerator() | sort LogonTime -descending | foreach-object {
$logTable.Rows.Add($_.value.UserName,$_.value.IpAddress,$_.value.LogonTime,$_.value.LogOffTime,$_.value.Duration,$_.value.NumberofRequests,$_.value.BytesSent,$_.value.BytesRecieved)
if($sumHash.contains($_.value.UserName)){
$sumHash[$_.value.UserName].NumberofLogons = $sumHash[$_.value.UserName].NumberofLogons + 1
$sumHash[$_.value.UserName].Duration = [INT]$sumHash[$_.value.UserName].Duration + [INT]$_.value.Duration
$sumHash[$_.value.UserName].BytesSent = [INT]$sumHash[$_.value.UserName].BytesSent + [INT]$_.value.BytesSent
$sumHash[$_.value.UserName].BytesRecieved = [INT]$sumHash[$_.value.UserName].BytesRecieved + [INT]$_.value.BytesRecieved
}
else{
$usrobj = "" | select UserName,NumberofLogons,Duration,BytesSent,BytesRecieved
$usrobj.UserName = $_.value.UserName
$usrobj.NumberofLogons = 1
$usrobj.Duration = $_.value.Duration
$usrobj.BytesSent = $_.value.BytesSent
$usrobj.BytesRecieved = $_.value.BytesRecieved
$sumHash.add($_.value.UserName,$usrobj)
}
}
$sumHash.GetEnumerator() | sort LogonTime -descending | foreach-object {
$sumTable.Rows.Add($_.value.UserName,$_.value.NumberofLogons,$_.value.Duration,$_.value.BytesSent,$_.value.BytesRecieved)
}
$dgDataGrid.DataSource = $sumTable
}
$Dataset = New-Object System.Data.DataSet
$logTable = New-Object System.Data.DataTable
$logTable.TableName = "RCPLogons"
$logTable.Columns.Add("UserName");
$logTable.Columns.Add("IpAddress");
$logTable.Columns.Add("LogonTime");
$logTable.Columns.Add("LogOffTime");
$logTable.Columns.Add("Duration",[INT64]);
$logTable.Columns.Add("NumberofRequests",[INT64]);
$logTable.Columns.Add("Sent",[INT64]);
$logTable.Columns.Add("Recieved",[INT64]);
$sumTable = New-Object System.Data.DataTable
$sumTable.TableName = "RpcSummary"
$sumTable.Columns.Add("UserName");
$sumTable.Columns.Add("NumberofLogons",[INT64]);
$sumTable.Columns.Add("Duration",[INT64]);
$sumTable.Columns.Add("Sent",[INT64]);
$sumTable.Columns.Add("Recieved",[INT64]);
$form = new-object System.Windows.Forms.form
$form.Text = "Outlook Anywhere Log Tool"

# Add DataGrid View
# Local Time Converstion CheckBox
$ltimeconv = new-object System.Windows.Forms.CheckBox
$ltimeconv.Location = new-object System.Drawing.Size(310,7)
$ltimeconv.Size = new-object System.Drawing.Size(150,20)
$ltimeconv.Checked = $true
$ltimeconv.Text = "Convert to Local Time"
$form.Controls.Add($ltimeconv)
# Content
$cmClickMenu = new-object System.Windows.Forms.ContextMenuStrip
$cmClickMenu.Items.add("test122")
# Add Open Log file Button
$olButton = new-object System.Windows.Forms.Button
$olButton.Location = new-object System.Drawing.Size(20,19)
$olButton.Size = new-object System.Drawing.Size(75,23)
$olButton.Text = "Select file"
$olButton.Add_Click({openLog})
$form.Controls.Add($olButton)
# Add FileName Lable
$fnFileNamelableBox = new-object System.Windows.Forms.Label
$fnFileNamelableBox.Location = new-object System.Drawing.Size(110,25)
$fnFileNamelableBox.forecolor = "MenuHighlight"
$fnFileNamelableBox.size = new-object System.Drawing.Size(200,20)
$form.Controls.Add($fnFileNamelableBox)
# Add Refresh Log file Button
$refreshButton = new-object System.Windows.Forms.Button
$refreshButton.Location = new-object System.Drawing.Size(390,29)
$refreshButton.Size = new-object System.Drawing.Size(75,23)
$refreshButton.Text = "Refresh"
$refreshButton.Add_Click({Populatetable})
$form.Controls.Add($refreshButton)
# Show Summary CheckBox
$SsumBox = new-object System.Windows.Forms.CheckBox
$SsumBox.Location = new-object System.Drawing.Size(510,7)
$SsumBox.Size = new-object System.Drawing.Size(200,20)
$SsumBox.Checked = $true
$SsumBox.Add_Click({if ($SsumBox.Checked -eq $true){$dgDataGrid.DataSource = $sumTable}
else {$dgDataGrid.DataSource = $logTable}})
$SsumBox.Text = "Show Summary"
$form.Controls.Add($SsumBox)
$dgDataGrid = new-object System.windows.forms.DataGrid
$dgDataGrid.AllowSorting = $True
$dgDataGrid.Location = new-object System.Drawing.Size(12,81)
$dgDataGrid.size = new-object System.Drawing.Size(1024,750)
$form.Controls.Add($dgDataGrid)

$form.topmost = $true
$form.Add_Shown({$form.Activate()})
$form.ShowDialog()

Expanding on the Exchange to DC processor Ratio's post

This post focuses on two aspects of Active Directory (AD) design for Exchange 2007: 
  • The recommended ratio of Exchange 2007 servers to Active Directory global catalog (GC) servers
  • When to use dedicated Active Directory sites for your Exchange 2007 servers
While most of the existing guidance about AD design for Exchange 2003 applies to Exchange 2007, there are several new factors that must be considered:
  • Exchange 2007's new 64-bit architecture
  • Availability and mix of both 32-bit and 64-bit AD servers
  • New (or upgraded) services and clients (Unified Messaging, anti-virus/anti-spam, Outlook 2007, etc.)
  • New role separation and architecture for Exchange 2007 servers (Mailbox, Hub, Client Access Server, Edge, UM.)
  • More use of advanced features by typical users (mobile devices, calendaring, etc.)
  • Increasing average mail traffic and storage allowances
  • Additional message handling (anti-spam/antivirus, compliance features) requires more GC network bandwidth and processor utilization per user/message
How Many Global Catalog Servers Do I Need for My Exchange 2007 Servers?
The previous recommendations for Exchange 2003 are available at:

Global catalog server placement and ratios in an Exchange 2000 Server organization or in an Exchange Server 2003 organization

The primary recommendation in the above article is that you should have 1 Global Catalog processor core per 4 Exchange server processor cores. It is important to note that this is not a 1:4 ratio of servers or even processors, but of processor cores: a dual core processor counts as 2 when doing the ratio calculations.

With Exchange Server 2007, Microsoft recommends that you deploy one 32-bit GC CPU core for each four Exchange 2007 mailbox server CPU cores.  While the other server roles will influence the number of GC cores required, the Mailbox servers that are deployed influences the deployment of each of the other roles, so basing the number of GC cores on mailbox server cores will suffice.

For example, suppose you have an 8 processor (single core) Exchange 2003 mailbox server. This Exchange server's Active Directory needs can be satisfied by two single-processor/single core GC machines, by one dual-processor/single core machine, or by a dual-core single-processor GC.

The Exchange 2003 recommendations have been based on domain controllers running 32-bit Windows 2003. Something wonderful happens when you upgrade your domain controllers to 64-bit Windows 2003: efficiency can double!

Instead of a 1:4 ratio, you can have a 1:8 ratio between Global Catalog processor cores and Exchange server cores. This assumes that you fulfill one critical requirement: You must have enough RAM installed on the 64-bit server to cache the entire Active Directory database in memory.

To check the size of your Active Directory database, look on a global catalog server for the NTDS.DIT file. This file is installed by default in %WINDIR%\NTDS.

On a 32-bit machine, no matter how much RAM you install, you're unlikely to be able to cache the whole database unless it is less than 1.5-1.7 GB in size. This is because of inherent memory addressing limitations in the 32-bit platform. For more details about why this is so, see the article links below.

On 64-bit Windows, with its massively larger address space compared to 32-bit, you can cache even very large Active Directory databases entirely in RAM. This gets you a significant speed boost in responding to queries along with a dramatic reduction in expensive disk I/O.

(If you're thinking, well, my AD database is a couple of gigabytes in size, but what if I were to use the /3GB switch to get some extra 32-bit address space for cache? Theoretically, yes, you can increase the amount of cache this way. As a practical matter, you would likely only get about 2.5-2.7 GB consistently in cache, and that would come at the price of system stability. Use of the /3GB switch on domain controllers is discouraged because it is likely to starve the machine of critical kernel resources. When this happens, the typical result is that after several days or weeks of operation, or after a peak in client demand, the server becomes unstable or unresponsive.)

When you upgrade to 64-bit domain controllers, be sure you install enough RAM to cache your entire Active Directory database with some room to spare. Remember: if you don't install enough RAM, your performance will be more like a 32-bit domain controller.

Keep in mind that these ratio recommendations are rules- of-thumb, not guaranteed numbers. Your environment may have unusual loads or configurations.

Also, if you keep using 32-bit domain controllers with 64-bit Exchange Mailbox servers, you may skew the standard 1:4 recommendation. The ratio recommendation assumes that the DCs and the Exchange Mailbox servers are both on roughly equivalent hardware. But a 64-bit machine is likely to have more advanced processors and other components (busses, memory, etc.). If using older domain controller machines, you may need to lower the ratio. Standard benchmarks, such as the SPEC benchmarks (http://www.spec.org/) , can give you a more realistic basis for comparing older to newer machines.

The Performance Tools included in the Toolbox section of Exchange 2007's Exchange Management Console application can be very useful in reporting actual performance and finding bottlenecks. For more detailed recommendations, including lists of performance counters to monitor, take a look at these blog posts and white papers:

Measuring AD Performance (Exchange Team Blog post)
Active Directory Performance for 64-bit Versions of Windows Server 2003 (Windows white paper)
Exchange 2007 Processor and Memory Recommendations (Exchange Team Blog post)
Microsoft Windows Kernel Memory Management and Microsoft Exchange Server (Exchange Team Blog post)
Best Practice Active Directory Design for Exchange 2000 (Exchange white paper still highly applicable to Exchange 2007)
Planning a Microsoft Exchange Server 2003 Messaging System (Exchange white paper also a good resource that mostly applies to Exchange 2007)

Regardless of your GC processor platform, you should expect some increase in network traffic between GCs and Exchange 2007 servers. This increase may be sizeable if you are taking full advantage of all the new features and roles in Exchange 2007 (antivirus, anti-spam, legal compliance features, unified messaging and so on). At Microsoft, when all of these features are heavily used, the network traffic between AD and Exchange nearly doubles. While this is a substantial increase, it has not had a large practical effect. Historically, network bandwidth has not been a common bottleneck between Exchange and Active Directory. Nonetheless, you should check your current utilization and be sure you have enough network bandwidth between Exchange servers and domain controllers as you deploy additional Exchange 2007 features.

Should I Use Dedicated Active Directory Sites for My Exchange 2007 Servers?

The short answer to that question is, No, not if you're creating a dedicated site only for the purpose of domain controller load balancing. This is a different answer than for Exchange 2003, so it needs some explaining.

In Exchange 2003, message routing and Active Directory site topology were independent of each other. The primary reason to dedicate an Active Directory site to Exchange was to prevent other services and applications from hogging domain controllers needed by Exchange. (Or, to take a less Exchange-centric view of the universe, to prevent Exchange from hogging the DCs needed by other services.)

In Exchange 2007, message routing maps directly to Active Directory site topology. Your AD site topology should therefore make sense for message routing, not just for domain controller load balancing. You should avoid creating an Exchange-dedicated Active Directory site unless doing so makes routing better too.

Fortunately, the extra headroom available with 64-bit GC servers makes it easier for demanding applications to coexist in the same site, further reducing the motivation to use dedicated sites for Exchange 2007.

Let's define a "demanding application" as one that:
  • Continuously generates a large number of LDAP queries
  • Requires ANR (Ambiguous Name Resolution) to resolve a good percentage of those queries, and
  • Makes frequent authentication requests against domain controllers
If you have multiple demanding applications running in a site, you must think not only about their average loads, but also about their peak loads, and the possibility that all of those applications may all at once decide to pick on a single domain controller. As a rule-of-thumb, if you are using 32-bit domain controllers, then Exchange is likely to coexist happily with other demanding applications in a site with up to 10,000 Exchange users. If you are using 64-bit directory servers, you can support up to 20,000 Exchange users in the same site with other demanding applications.

Exchange does a pretty good job of load balancing its requests across the domain controllers available to it, including taking into account how busy the DCs are with other work. So Exchange is not likely to be the site bully. But it is a demanding application and will present steady, relatively high load across all the GCs available in the site.

If you are above the peak limits described above, you should look at ways to dedicate directory resources to Exchange. That does not necessarily mean isolating Exchange to another site:

  • You can try to reserve domain controllers for Exchange use by being clever with DNS. By appropriately configuring SRV record priorities and weights, you can often get nearly the same effect as a dedicated site.
More about how to do this can be found here:
  • Creating an Active Directory Site for Exchange Server (Ignore the first part of this paper and scroll down to the part about how to "Isolate Client Authentication Traffic from Exchange Facing Domain Controllers.")
  • You can define a static set of domain controllers for use by a particular Exchange server (using the set-ExchangeServer cmdlet). This lets you keep Exchange from using domain controllers dedicated to other applications. While this is preferable to creating a dedicated site, it is not optimal from the perspective of ongoing management. You have to remember which GCs belong to whom and manually tune and maintain lists instead of letting Exchange automatically optimize the load balancing. 

If another application cannot be prevented from overwhelming the domain controllers Exchange needs, then investigate your options for restricting or isolating that application before deciding to protect Exchange in a dedicated site.

If you have more than five Active Directory sites in your environment try very hard not to put Exchange 2007 in a dedicated site. In an environment more complex than five sites, it's unlikely that you will be able to design a dedicated site topology that works well for routing. Yes, you can then wrestle the Exchange 2007 routing topology into the shape you want without regard to the AD site topology. But you don't want the ongoing headache of managing and maintaining a message routing topology and an AD site topology that don't mesh together.

Exchange and Active Directory are deeply entwined. Exchange 2007 takes advantage of this synergy to simplify message routing and make it more deterministic (and easier to troubleshoot). Let that work in your favor. If your Exchange routing needs are completely at odds with your AD site topology, it may be time to rethink both together.

To summarize all this:
  • 64-bit Active Directory servers are great news for Exchange and can double the performance of 32-bit servers.
  • Take advantage of the doubling in performance you can get with 64-bit Active Directory servers. You can reduce the number of AD servers required to support Exchange and simplify both your Active Directory site and Exchange message routing topologies.

Exchange 2010 SP1 Auto-Mapping

Here is an article describing one of the new features of Exchange 2010 SP1, Auto-Mapping.  I ran into an issue today where a user’s mailbox was extremely slow.  He had full access permissions to many mailboxes and even though he would close them auto-mapping would open those mailboxes again.  Once the full access permissions were removed Outlook was back to normal functionality.

http://www.stevieg.org/2010/08/auto-mapping-shared-mailboxes-in-exchange-2010-sp1-with-outlook-2010/

Exchange to DC processor Ratio's

This has become a big topic and I wanted to put this out. 
  • If Active Directory is running on the x86 platform (32-bit), the recommended ratio of Active Directory directory server processor cores to Exchange 2010 Mailbox server processor cores is 1:4.
  • If Active Directory is running on the x64 platform (64-bit), the recommended ratio of Active Directory directory server processor cores to Exchange 2010 Mailbox server processor cores is 1:8. To achieve the 1:8 ratio, you must have enough memory installed on the directory server to cache the entire Active Directory database in memory. To check the size of your Active Directory database, examine the NTDS.DIT file on a global catalog server. By default, this file is located in %WINDIR%\NTDS.
Please make note of the highlighted section this is important for Virtualized Environments. DC’s must have enough memory to cace the entire NTDS.Dit and maintain the 2 GB of system memory.

Thursday, February 24, 2011

Move Mailboxes from Exchange 2007 to Exchange 2010

Moving mailboxes from Microsoft Exchange Server 2007 Service Pack 2 (SP2) to Exchange Server 2010? Consider the following:

1. The move process is performed online, and end-users will be able to access their mailboxes during the move.

2. You can't move mailboxes from Exchange 2007 SP1 or earlier. The source Mailbox server must be running Exchange 2007 SP2 or later.

3. Perform the move from a server running Exchange 2010 by using the Exchange Management Console or the move request cmdlets in the Exchange Management Shell. You can't use the Move-Mailbox cmdlets in Exchange 2007 to move mailboxes to Exchange 2010 servers.

Exchange 2010 Backups

Backup-less Support: The Exchange Server Database Availability Group architecture allows log file replay to be lagged, enabling administrators to perform point-in-time database restores without the need for tapes. Organizations can rely on their high availability infrastructure rather than tape backups to recover from failures, and substantially decrease their operating costs.

Lagged mailbox database copy
A passive mailbox database copy that has a log replay lag time greater than zero.

Test Outlook Web Services (AutoDiscover)

Use the Test-OutlookWebServices cmdlet to verify the Autodiscover service settings for Microsoft Outlook on a computer running Microsoft Exchange Server 2010 that has the Client Access server role installed.
Syntax

Code:
Test-OutlookWebServices [-Identity <RecipientIdParameter>] [-ClientAccessServer <ClientAccessServerIdParameter>] [-Confirm [<SwitchParameter>]] [-DomainController <Fqdn>] [-MonitoringContext <$true | $false>] [-TargetAddress <RecipientIdParameter[]>] [-WhatIf [<SwitchParameter>]]


This example tests for a connection to each service. This example also submits a request to the Availability service for the user name@domain.com to determine whether the user's free/busy information is being returned correctly from the Client Access server to the Outlook client.
Code:
Test-OutlookWebServices -Identity:holly@contoso.com

Other Examples:
Code:
Test-OutlookWebServices -ClientAccessServer "Server"

Set-ClientAccessServer -id mss150 -AutoDiscoverServiceInternalUr


Important update about Exchange 2010 Personal Archives support in Outlook 2007

Quick post to inform you that we've found an issue in Exchange 2010 Personal Archives support for Outlook 2007 in the December 2010 Cumulative Update for Office 2007, which may result in users being unable to access their archive mailbox.

Our friends in the Outlook team have been working hard to fix this. The fix has been checked-in in the February 2011 Cumulative Update for Office 2007, which will be available later this month. As with all updates, we recommend that you test them in a non-production environment before deploying them in production. Sorry for any inconvenience this may have caused.

View article...

Exchange ActiveSync and Windows Phone 7 "phantom data" issue

Some of you may have seen stories about higher than expected data usage for a very small percentage of Windows Phone 7 users being caused by an inefficiency between Windows Phone 7 client and Yahoo! Mail. You may have also seen that these stories mention that there is a separate Exchange ActiveSync (EAS) issue that is impacting a small percentage of users.

Unfortunately, like a game of telephone, many of the stories I've seen are combining these two issues into one, or mentioning both of these issues as if they have equal impact on users. For the record, Yahoo! Mail is not connected to Windows Phone 7 using EAS, and the Windows Phone 7 EAS implementation issue is much smaller. In some of the most egregious cases, some have called this a protocol level bug in EAS. This is untrue — this is a client issue.

We on the Exchange team know that you all work very hard to make sure your Exchange servers are running well, so we wanted to get you some good information (straight from us here at Microsoft) about these issues.
First off; for administrators, there's no need for any changes to your Exchange servers — the issues here are client-side and thus do not impact your servers nor increase the load on your servers.

That said, we know that many of you work closely with your users and always try to help them out with good support information. If you have users who are contacting you about this issue or may be impacted, there are mitigations available for each of these issues while the Windows Phone 7 team continues to work with Yahoo! on a full software fix that is expected in the near term.

For Windows Phone 7 EAS issue, users might see increased data usage when a message is "stuck" in their outbox. A user might encounter this state if they send email to an invalid email address or if they send a message that's larger than the allowed message size (server side policy) from their phone. In these cases, users can go to their Outbox folder and delete the stuck message and the phone's data use will return to normal. Below are the steps a user can take to see if there is a message stuck in their outbox and how to delete it.

Note: This impacts only a very small percentage of Windows Phone 7 users and does not impact EAS users on other phone platforms.

1. Click the folder icon  at the bottom of the EAS-enabled inbox
2. Select the Show all folders option
3. Select the folder called Outbox (if you have a lot of folders, you may have to scroll down to see this folder)
4. Delete  any messages in this folder

The Yahoo! Mail issue is not connected to EAS, but users can lessen the impact by changing the frequency that the Yahoo! Mail account is checked for new messages. Below are the steps to make this change:

1. On the Start screen of Windows Phone 7, click on the arrow at the top right
2. Choose Settings from the app list
3. Choose email & accounts
4. Choose Yahoo! Mail
5. Click on the setting under Download new content
6. Select a less frequent setting. If you are using the default setting (every 2 hours), change this setting to manually
7. Click on the setting under Download email from
8. Select a shorter time range. If you are using the default setting (the last 2 weeks), change this setting to the last 7 days

View Article...

Exchange 2010 SP1: The Troubleshooters

When managing an Exchange 2010 server organization in a production environment, an Exchange Administrator many times wants to keep track of things like Database Latency, Disk Space problems, search issues, user needs, and more. PowerShell was created to give us more power in our admin efforts, and many books have already been written to assist us with mastering the ins and outs of PowerShell Programming. What if I told you that you already had some examples of PowerShell programming, right on your Exchange server? And they are pretty helpful on top of that. Beginning with RTM, Exchange 2010 has shipped with 3 PowerShell Troubleshooters out of the box. Written in the PowerShell scripting language, they are ready for you to use:
  • Content Index Troubleshooter(Troubleshoot-CI.ps1)
  • Database Latency Troubleshooter(Troubleshoot-DatabaseLatancy.ps1)
  • Database Disk Space troubleshooter(Troubleshoot-DatabaseSpace.ps1)
These PowerShell scripts are located in the \Program Files\Microsoft\Exchange Server\V14\Scripts folder.

Let's go over them.

The Content Index Troubleshooter:The CI Troubleshooter (Troubleshoot-CI.ps1) is designed to monitor and perform troubleshooting on Content Index catalogs. It detects and resolves the following symptoms:
  • Deadlock: Exchange Search deadlocks waiting on threads from MSSearch
  • Corruption: One or more search indices are corrupted
  • Stall: Similar to deadlock in that the indices are not getting updated
  • Backlog: The Search catalog is backlogged resulting in missing index searches
This troubleshooter can be used against the server or just a database, and can be used to detect as well as resolve problems. It can also be used in just a monitoring context to log warnings and failure events in the app log.
 
Parameters

Server: (optional) The NETBIOS name of mailbox server on which troubleshooting should be attempted for CI catalogs. If this optional parameter is not specified, the local server is assumed.

Database: (optional) This is the name of database to troubleshoot. If this parameter is not specified, catalogs for all databases on the server specified by the Server parameter are used.

Symptom: (optional) Specifies the symptom to detect and troubleshoot.
Possible values are:
  • Deadlock
  • Corruption
  • Stall
  • Backlog
  • All(default)
When 'All' is specified, all symptoms are tested.

Action: (optional) This specifies the action to be performed to resolve a symptom. Possible values are:
  • Detect (default)
  • DetectAndResolve
  • Resolve
MonitoringContext: (optional) This specifies that the troubleshooter is being run in a monitoring context. The possible values are $true and $false. Default is $false. If the value is $true, warning/failure events are logged to the application event log.

FailureCountBeforeAlert: (optional) This specifies the number of failures the troubleshooter will allow before raising an Error in the event log, leading to a SCOM alert. The allowed range for this parameter is [1,100], default is 3. No alerts are raised if MonitoringContext is $false.

FailureTimeSpanMinutes: (optional) This specifies the number of minutes in the time span during which the troubleshooter will check the history of failures to count the failures and alert. If the failure count during this time span exceeds the value for FailureCountBeforeAlert, an alert is raised. No alerts are raised if MonitoringContext is $false. The default value for this parameter is 600 minutes.

Examples
C:\PS> .\Troubleshoot-CI.ps1 -database DB01

Detects and reports if there's any problem with catalog for

database DB01. Does not attempt any Resolution.

C:\PS> .\Troubleshoot-CI.ps1 -database DB01 -symptom Stall

Detects if indexing on catalog for database DB01 is stalled. Does not attempt any Resolution.

C:\PS> .\Troubleshoot-CI.ps1 -Server <S001>

Detects and reports problems with all catalogs on server S001, if any. Does not attempt any Resolution.

C:\PS> .\Troubleshoot-CI.ps1 -database DB01 -Action DetectAndResolve

Detects and reports if there's any problem with catalog for database DB01.

Attempts a Resolution of the problem.

Event Log
Events logged to the Microsoft-Exchange-Troubleshooters/Operational event log:

5000 Informational The troubleshooter started successfully
5001 Informational The troubleshooter finished successfully
5002 Informational The troubleshooter didn't find any issues for any catalog
5003 Informational The troubleshooter didn't find any catalog issues for the specified database
5004 Informational Restart of search services succeeded
5005 Informational Reseeding succeeded for the catalog of the specified database
5300 Warning Detected search service deadlock
5301 Warning Detected catalog corruption for the specified database
5302 Warning Detected indexing stall for the specified database
5600 Error The troubleshooter failed with the specified exception
5601 Error The troubleshooter detected the symptom %1 %2 times in the past %3 hours for catalog %4. This exceeded the allowed limit for failures.
5602 Error Search services failed to restart.
5603 Error Reseeding failed for the content index catalog of mailbox database %1. Reason: %2
5604 Error Indexing backlog reached a critical limit of %2 hours or more for the specified database
5605 Error Another instance of the troubleshooter is already running on this machine. Two or more instances cannot be run simultaneously.
6000 Informational The troubleshooter started detection.
6001 Informational The troubleshooter finished detection
6002 Informational The troubleshooter started resolution.
6003 Informational The troubleshooter finished resolution.
6600 Error The troubleshooter failed during detection.
6601 Error The troubleshooter failed during resolution.

The Database Latency Troubleshooter:The Database Latency Troubleshooter (Troubleshoot-DatabaseLatency.ps1) is designed to monitor and perform troubleshooting on database latency. It detects and resolves the following symptoms:
  • Disk latency
  • Active Directory Latency
  • RPC Latency
  • Top CPU user
This troubleshooter will run again the local mailbox database only.
 
Checks Disk Latency:
The troubleshooter detects disk latency by checking the following counters on the mailbox server database it is running against:

"\MSExchange Database ==> Instances($database)\I/O Database Reads Average Latency"
"\MSExchange Database ==> Instances($database)\I/O Database Reads/sec"
"\MSExchange Database ==> Instances($database)\I/O Database Writes Average Latency"
"\MSExchange Database ==> Instances($database)\I/O Database Writes/sec"

Note: The trouble shooter will take into account that the disk latencies are not caused by a heavily loaded disk subsystem.

It uses the following default thresholds:
  • Maximum latency threshold for disk read before it is deemed as bad is 200
  • Minimum read rate for disk before it is deemed as bad is 20
  • Minimum write rate for disk before it is deemed as bad is 20
Checks RPC Latency:
The troubleshooter detects the RPC latency by checking the following performance counters on the mailbox server for the database it is running against

"\MSExchangeIS Mailbox($database)\rpc average latency"
"\MSExchangeIS Mailbox($database)\RPC Operations/sec"
The maximum RPC average latency by default is set to 70
The maximum RPC operations/sec threshold by default is set to 50

Checks the Top CPU User:
The troubleshooter detects the top CPU user by generating a descending list of the users that are using up the most time on server for a given database. It uses the output of Get-StoreUsageStatistics to get the MailboxGuid and the time in the server used during the sampling period (10 min). If Quarantine has been enabled, the troubleshooter will log an event and quarantine the top CPU user.

Parameters:
The following parameters are provided as a part of the Database Latency troubleshooter:
MailBoxDatabaseName: (required) The Mailbox database the troubleshooter will run against.

Possible values are:
  • GUID
  • Distinguished name (DN)
  • Database name
LatencyThreshold: (optional) The maximum RPC average latency the server should be experiencing. The valid range for LatencyThreshold is from 1 to 200 with default as 70.
 
MonitoringContext: (optional) Specifies whether or not the troubleshooter is to write monitoring events to the Application and the Operations log in the Event Viewer.

TimeInServerThreshold: (optional) The TimeInServerThreshold sets the threshold for the top users that are causing the CPU starvation. The valid range is from 1 to 600000 with default as 60000.

Quarantine: (optional) Specifies whether or not to quarantine heavy users. By default it does not quarantine the heaviest user.

Example
Troubleshoot-Databaselatency.ps1 -MailboxDatabaseName <DatabaseID> [-latencyThreshold <1-200>] [-TimeinServerThreshold <1-600000>] [-Quarantine <switch>] [-MonitoringContext <switch>]

Event Log
Events logged to the Microsoft-Exchange-Troubleshooters/Operational event log:
5110 Informational The troubleshooter started successfully
5111 Informational The troubleshooter detected latency
5411 Warning The troubleshooter quarantined a user
5412 Warning Unusual activity found in a mailbox, but quarantine was not specified
5710 Error Disk latencies are abnormal for the specified database
5711 Error DSAccess latencies are abnormal for the specified database
5712 Error High RPC Average Latencies were detected for the specified database, but the troubleshooter was unable to determine the cause.

The Database Disk Space Troubleshooter:The Database Disk Space Troubleshooter (Troubleshoot-DatabaseSpace.ps1) is designed to monitor and perform troubleshooting on database disk space issues. It detects the amount of database disk space, and it detects the cause for log generation:
  • It tracks the top users that are generating logs
  • It tracks the available disk space for both logs and database
Based on a disk space and time threshold, the troubleshooter has the option to Quarantine the top users.

Parameters:
The following parameters are provided as a part of the Database Disk Space Troubleshooter:
Server: (required) Specifies the mailbox server on which you are monitoring the log growth for all mailbox databases.

Note   You can't use this parameter in conjunction with the MailboxDatabaseName parameter.
MailboxDatabaseName: (required) Specifies the mailbox database on which you are monitoring the log growth.

Possible values are:
  • GUID
  • Distinguished name (DN)
  • Database name
Note:   You can't use this parameter in conjunction with the Server parameter.

PercentEdbFreeSpaceThreshold: (optional) Specifies the percentage of disk space for the EDB file at which Exchange should begin quarantining users. The valid value is from 1-99 and the default is 25 percent.

PercentLogFreeSpaceThreshold: (optional) Specifies the percentage of disk space for the log files at which Exchange should begin quarantining users. The valid value is from 1-99 and the default is 25 percent.

HourThreshold: (optional) Specifies the number of hours that you can wait until running out of space. The default value is 12 hours. The valid range is 1 to 1000000000

MonitoringContext: (optional) Specifies whether the results of the command include monitoring events to be written in the regular application logs in Event Viewer and in the Operations log.

Quarantine: (optional) Specifies that heavy users will be quarantined

Examples
Troubleshoot-DatabaseSpace.ps1 -MailboxDatabaseName <DatabaseID> [-PercentEdbFreeSpaceThreshold <1-99>] [-PercentLogFreeSpaceThreshold <1-99>] [-HourThreshold <1- 1000000000>] [-Quarantine <switch>] [-MonitoringContext <switch>]
Troubleshoot-DatabaseSpace.ps1 -Server <ServerID> [-PercentEdbFreeSpaceThreshold <1-99>] [-PercentLogFreeSpaceThreshold <1-99>] [-HourThreshold <1- 1000000000>] [-Quarantine <switch>] [-MonitoringContext <switch>]

Event Log
Events logged to the Microsoft-Exchange-Troubleshooters/Operational event log:
5100 Informational The troubleshooter started successfully
5101 Informational The troubleshooter finished. No problems detected.
5400 Warning The database is over the expected threshold.
5401 Warning Over the threshold, but the database is not growing. No action taken.
5410 Warning The troubleshooter quarantined the specified mailbox.
5700 Error The database is over the expected threshold and continues to grow. Manual intervention is required.

To watch for:
Even though you start them from the Exchange Management Shell, for the most part these troubleshooters give very little to no output to the window itself. You will need to go to the event log to see the reports. The exception is if you run the troubleshooters in Verbose Mode.

But, on the good side, you will be able to filter the event log for warnings and errors. So you will be able to periodically check the server and then take a quick glance at the event log to see if there are any problems. And speaking of periodically checking the server, that brings me to...

Bonus:
You can call the troubleshooters from within other PowerShell scripts as part of an overall server health monitoring plan. Here's a simple example:

$servers= get-mailboxserver
while($true)
{
     foreach ($server in $servers)
     {
        .\Troubleshoot-CI.ps1 -verbose -server $server.Name -Action:DetectAndResolve -Symptom:Stall
     }
     sleep 14440
}

Calling get-mailboxserver with no parameters will return a complete list of the mailbox servers in the entire Exchange organization. The above script will poll each Exchange Server in the org, sleep, and then poll them again.

View Article....

Whtie Space in Exchange 2010, Where did 1221 Go?

One important Exchange Server application task & process has always been the white space and how we deal with it as Exchange administrators. White space has always been one of the most asked, case & specific area in Exchange Server. If you remember in previous versions of Exchange event log 1221 would be logged on the application logs and we would sum these up to find out the total white space in Exchange databases.

Now, as far as basic goes, dealing with Exchange white space was simple. We wouldn't take the database "off line" and try to run defrag on it, we would simply create new DB and move Mail Boxes into the newly created DB. Once we are done moving all mail boxes, we would delete the DB with all the white space.
The big reason we would do this would be for backup considerations, as we would not want to backup white space , otherwise Exchange was smart enough to find the white space each night after online maintenance and mark the white space as "re-usable" area for the related DB.

Now in Exchange 2010 there are no more 1221 event logs, because Exchange 2010 is dealing with the white space in a bit more advance approach. Background database maintenance and 24/7 online defragmentation process is explained in details as fallows.

If you would like to see the available space on the database you can run fallowing

Get-MailboxDatabase Database1 -Status | FL AvailableNewMailboxSpace

Database White Space

The database size on the physical disk isn't just the number of users multiplied by the mailbox storage quota. When the majority of users aren't approaching their mailbox storage quota, the databases consume less space and white space isn't a capacity concern. The database itself will always have free pages, or white space, spread throughout. During background database maintenance, items marked for removal from the database are removed, which frees these pages. The percentage of white space is constantly changing due to the efforts of the 24x7 online defragmentation process.

You can estimate the amount of white space in the database by knowing the amount of mail sent and received by the users with mailboxes in the database. For example, if you have 100 2-GB mailboxes (total of 200 GB) in a database where users send and receive an average of 10 MB of mail per day, the amount of white space is approximately 1 GB (100 mailboxes × 10 MB per mailbox). The amount of white space can exceed this approximation if background database maintenance isn't able to complete a full pass.

Click here to read more ....