Tech Blog

Hello,

Preface this post by saying a man from Exchange support said “This is the most complicated Exchange environment I’ve ever seen”.

That said this issue is pretty common and hopefully this post will help someone else.

We have an Exchange 2010 to 365 hybrid environment that look a bit like this:

We had an issue where users on our 365 tenancy couldn’t see on the Exchange 2010 on-premises free/busy info for users in Group2.contoso.com.

Now I know what your thinking, just compare the settings on Group1 to Group2, well due to company rules and politics I can’t….I can only troubleshoot group2 and the servers there.

So first things first, check users permissions and setup a test user and find the error in Outlook:

“No Information. No free/busy information could be retrieved”

The recipient’s server could not be contacted. Contact your administrator.

Thanks to Babunski and his post I found this really good troubleshooting guide and everything looked ok:

https://support.microsoft.com/en-us/help/10092/troubleshooting-free-busy-issues-in-exchange-hybrid-environment

• Firewall is fine,
• Network is ok,
• DNS surprising is working,
• Check Exchange online tool:

https://www.testexchangeconnectivity.com

• 365 to prem relationship is ok:

OrganizationRelationship -Identity "Exchange Online to On Premises Organization Relationship" | FL

• IIS Logs look ok, %SystemDrive%\inetpub\logs\LogFiles

• EWS logs look ok, %SystemDrive%\inetpub\logs\LogFiles

• Checked the external URL – seems ok.

Get-WebServicesVirtualDirectory | FL Name,Server,externalURL

• Check the IIS permissions with – this looked ok

Get-WebServicesVirtualDirectory | fl name,server,externalURL,ExternalAuthenticationMethods

• Checked IIS EWS and Autodiscover:

• Checked more relationship stuff – all ok

Get-OrganizationReationship -Identity "On Premises to Exchange Online Organization Relationship”

Next – contact support!  🙁

Before I contact support I did find one more URL that suggests to check the certs and import the cert you used to setup the federation onto the CAS server which unfortunately didn’t work for us:

https://support.microsoft.com/en-gb/help/3057905/exchange-online-users-cannot-access-free-busy-information-of-users-in

Soooo here I am, time to contact support.

The first thing they checked was the local url on the client access server:

https://ClientAccessGroup2Server1.group.contoso.com/ews/exchange.asmx

So there’s an issue, basically we didn’t add the server to our wild card cert.  So added the server names as Subject alternative names and imported it using PowerShell onto both Client access servers and then rebooted:

Enable-ExchangeCertificate -Thumbprint 011111111111111111111111111111111111 -Services SMTP,IIS

Fixed:

Checked the URLS set in Exchange:

Get-WebServicesVirtualDirectory | FL

Our internal URL was actually set to Client Access array for Contoso rather than group2.contoso.com so we changed this:

set-WebServicesVirtualDirectory –identity ClientAccessGroup2Server1 –internalurl https://CASARRAY.Group2.contoso.com/EWS/exchange.asmx

And rebooted again.

Next we disabled and re-enabled ISS security (this broke OOF for a while, we had to run this twice):

Set-WebServicesVirtualDirectory -Identity 'ClientAccessGroup2Server1\ews (Default Web Site)' -BasicAuthentication $false -WindowsAuthentication $false -WSSecurityAuthentication $false

Set-WebServicesVirtualDirectory -Identity ' ClientAccessGroup2Server1\ews (Default Web Site)' -BasicAuthentication $true -WindowsAuthentication $true -WSSecurityAuthentication $true

So here are stuck…..

MS ran some traces using Extra:

And went away for a while and came back with:

Internet facing Site Conotso.com is able to look up the user and send a request to Group2 servers.

Testy1@contoso.mail.onmicrosoft.com: Request for Testy2@group2.contoso.com is being proxied to https://CASARRAY.Group2.contoso.com/ews/exchange.asmx

Testy1@contoso.mail.onmicrosoft.com: Setting exception to all queries: Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException: System.InvalidOperationException: Client found response content type of '', but expected 'text/xml'.

The request failed with an empty response.

at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling(). The request information is ProxyWebRequest type = CrossSite, url = https://ClientAccessGroup2Server1.group2.contoso.com:443/ews/exchange.asmxMailbox list = <Bert Test>SMTP: Testy2@group2.contoso.com, Parameters: windowStart = 30/07/2018 00:00:00, windowEnd = 10/09/2018 00:00:00, MergedFBInterval = 30, RequestedView = MergedOnly. ---> System.InvalidOperationException: Client found response content type of '', but expected 'text/xml'.

On group2 server we notice below error,

<TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Error"><TraceIdentifier>http://msdn.microsoft.com/en-GB/library/System.ServiceModel.Diagnostics.EventLog.aspx</TraceIdentifier><Description>Wrote to the EventLog.</Description><AppDomain>/LM/W3SVC/1/ROOT/EWS-1-131788827699531225</AppDomain><ExtendedData xmlns="http://schemas.microsoft.com/2006/08/ServiceModel/DictionaryTraceRecord"><CategoryID.Name>WebHost</CategoryID.Name><CategoryID.Value>5</CategoryID.Value><InstanceID.Name>WebHostFailedToProcessRequest</InstanceID.Name><InstanceID.Value>3221356547</InstanceID.Value><Value0>System.ServiceModel.ServiceHostingEnvironment+HostingManager/39086322</Value0><Value1>System.ServiceModel.ServiceActivationException: The service '/EWS/Exchange.asmx' cannot be activated due to an exception during compilation.  The exception message is: This collection already contains an address with scheme http.  There can be at most one address per scheme in this collection.

So the long and short of it is they think IIS is broken. The traffic is being passed to the Group2 services but these services are not passing the information back up the stream.

MS decided they wanted swap out the EWS web.config with a new one from:

c:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\exchweb\EWS

The reason being in the config file it was referencing:

<assemblyIdentity name="Microsoft.Exchange.Common.IL" publicKeyToken="31bf3856ad364e35" culture="neutral" />

<codeBase version="0.0.0.0" href="file:///%ExchangeInstallDir%bin\Microsoft.Exchange.Common.IL.dll"/>

And it should be referencing (or where ever you install of Exchange is):

<assemblyIdentity name="Microsoft.Exchange.Common.IL" publicKeyToken="31bf3856ad364e35" culture="neutral" />

 <codeBase version="0.0.0.0" href="file:///C:\Program Files\Microsoft\Exchange Server\V14\bin\Microsoft.Exchange.Common.IL.dll"/>

And another reboot.

Next we checked the logging from Outlook:

Which dumps files too: %Temp%\outlook logging

And they found this error:

Exception Type Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequestProcessingException

Response Code ErrorProxyRequestProcessingFailed

This prompted MS to check the IIS bindings which were wrong

So we added some missing bindings using these command:

C:\Windows\system32>cd inetsrv

appcmd set site /site.name:"Default Web Site" /+bindings.[protocol='net.tcp',bindingInformation='808:*'] SITE object "Default Web Site" changed

appcmd set site /site.name:"Default Web Site" /+bindings.[protocol='net.pipe',bindingInformation='*'] SITE object "Default Web Site" changed

appcmd set site /site.name:"Default Web Site" /+bindings.[protocol='net.msmq',bindingInformation='localhost'] SITE object "Default Web Site" changed

appcmd set site /site.name:"Default Web Site" /+bindings.[protocol='msmq.formatname',bindingInformation='localhost'] SITE object "Default Web Site" changed

No change and still the same error: Response Code ErrorProxyRequestProcessingFailed

So we checked windows services and would you believe it but these dot net services were not installed:

net.tcp lisener adapter

net.pipe listener adapter

So we installed the missing features:

Rebooted both.

And Boom we are working!!!!!!!!!!!!!!!!!!!!!

If you ever get this annoying message when opening an Email Address book policy:

The specified e-mail address policy couldn't be edited. E-mail address polices created with legacy versions of Exchange must be upgraded using the 'Set-EmailAddressPolicy' task, with the Exchange 2010 Recipient Filter specified.

This means the policies were created with old versions of Exchange, in my case Exchange 2003, you can find them all, looking for the word Legacy:

Get-EmailAddressPolicy | Select Identity, RecipientFilterType, RecipientFilter, LDAPRecipientFilter | FL

As you can this will show the ldap query too, the problem is if you run the Set-EmailAddressPolicy against this then you’ll break it, all custom filters (LDAP queries) will be reset to “mailnickname=*” which can result significant email outages.

So how to fix it.

Download this PS script: https://gallery.technet.microsoft.com/office/7c04b866-f83d-4b34-98ec-f944811dd48d

Choose a policy to convert, copy the query from above and run the following:

.\ConvertFrom-LdapFilter.ps1 "(&(mailNickname=*)(objectCategory=person)(objectClass=user)(objectClass=user)(objectCategory=person)(mailNickname=*)(msExchHomeServerName=*)(objectCategory=user)(mail=rob.D@EMAIL.com))"

So now we have this:

( ( Alias -ne $null ) -and ( ObjectCategory -like 'person' ) -and ( ObjectClass -eq'user' ) -and ( ObjectClass -eq 'user' ) -and ( ObjectCategory -like 'person' ) -and ( Alias -ne $null ) -and ( ServerLegacyDN -ne $null ) -and ( ObjectCategory -like 'user' ) -and ( WindowsEmailAddress -eq 'rob.D@EMAIL.com' ) )

Now finially lets run the Set-EmailAddressPolicy (NOTE THE {} at the beginning and end, not a ” :

Set-EmailAddressPolicy -Identity "migrate-test" -RecipientFilter {( ( ( Alias -ne $null ) -and ( ( ObjectCategory -like 'person' ) -and ( ObjectClass -eq 'user' ) -and ( recipientType-eq 'UserMailbox' ) ) ) -and ( ObjectCategory -like 'user' ) -and ( WindowsEmailAddress -eq 'rob.D@EMAIL.com' ) )

Done, F5 it in Exchange console and see if you can open it!!

Had a  crazy problem recently, a company Contoso.com was using Exchange 2010 (CAS, Hubs and Mailbox servers) with all the latest patches and roll-ups and had users on their domain using Outlook Anywhere perfectly.

The problem was their sub domain SubDomain.Contoso.com is also big company, they have their own Exchange 2010 servers  (CAS, Hubs and Mailbox servers) with all the same patches and roll-ups except they’re “downstream” of Contoso.com.  The problem is all users in SubDomain.Contoso.com couldn’t use Outlook Anyhwere, i.e. RPC proxy redirection wasn’t working:

The setup:

So all in all it was weird, we checked everything, firewall blocks, routing problems etc etc, even https://testconnectivity.microsoft.com/

So finally we logged a call with MS and after some lengthy troubleshooting they found the underlying issue:

https://support.microsoft.com/en-us/kb/2725008,

Although the article relates to Exchange 2010 sp2 RU3 it was still applicable in this scenario (SP3 RU15).

Applying the fix documented in method 2 resolved the issue.

“Disable preferred site enforcement
To resolve this issue, disable preferred site enforcement. To do this, create the following DWORD registry value:

Key:

HKLM\System\CurrentControlSet\Services\MSExchangeRPC\ParametersSystem

Value Name: EnablePreferredSiteEnforcement
Data Type: REG_DWORD
Value: 0”

Annoying but at least its fixed.

To add send as permissions from the existing permissions on a mailbox you can use this script:

All you need to do is specify the username twice:

$users = Get-MailboxPermission -identity "USERNAME" | where {$_.user -notlike "*SELF" -and $_.isinherited -eq $false}
foreach ($user in $users)

{
    $mailboxuser = get-aduser  "USERNAME"
    Add-ADPermission -Identity $mailboxuser.DistinguishedName -ExtendedRights Send-As -User $user.user
}

So We had a shared mailbox that was originally opened in Finland and as such the inbox was named postilaatikkoon and sent items was named Lähetetyt.

So I thought I’d jump into OWA and change the language there, well it turns out that has nothing to do with Folder names.

So after some research I found I could use:

Outlook.exe /resetFolderNames

But the mailbox is shared so doesn’t have a enabled user to logon with so I found this Exchange PowerShell command:

set-MailboxRegionalConfiguration -id "Mailbox Name" -LocalizeDefaultFolderName:$true -Language en-GB

Job done.

To connect to Exchange and use powershell use this command:

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri http://ExchangeServer.DOMAIN.LOCAL/PowerShell/ -Authentication Kerberos
Import-PSSession $Session

A friend came across this issue while migrating from Exchange 2010 to 2013.  Users were unable to receive/send email from their mobile device after migration.  This continues for up to 4 hours or so then starts working out of the blue.

It’s actually a bug with certain versions of Exchange 2013.

To fix, simply recycle the following App Pools in IIS on the CAS server:

• MSExchangeAutodiscoverAppPool
• MSExchangeSyncAppPool

A fix is in place for CU5, so if you are on that version or higher this fix “shouldn’t” apply…….

Hi,

When you install Microsoft Skype on a Windows-based computer that has Microsoft Lync installed, you experience one or more of the following issues:

Microsoft Outlook does not display presence information.
The Microsoft SharePoint client does not display presence information.
Other Microsoft Office applications that support presence features do not display presence information

This behavior may occur if the value of the DefaultIMApp registry entry in the following registry key is incorrect:
HKEY_CURRENT_USER\Software\IM Providers

Note The DefaultIMApp registry entry specifies which IM application provides IM and presence information in Office applications. When you install the Lync 2010 client together with Microsoft Office, the value of the DefaultIMApp registry entry is updated to “Communicator.”  Or, if you install the Lync 2013 client together with Microsoft Office, the value of the DefaultIMApp registry entry is updated to “Lync.”
To work around this issue, follow the steps that are appropriate for the version of Windows that you are running.

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs.

Windows XP, Windows Vista, or Windows 7

Exit the Skype client and all Office applications.
Click Start, click Run, type regedit.exe, and then click OK.
Locate the following registry entry:
HKEY_CURRENT_USER\Software\IM Providers\DefaultIMApp registry value
Right-click DefaultIMApp, and then click Modify.
Type Skype in the Value data box, and then click OK.
Exit Registry Editor, and then restart  the Skype client and the Office applications.

Windows 8

On the Start screen, click the Search tile.
Type regedit in the Search window and then double-click regedit.exe.
Locate the following registry entry:
HKEY_CURRENT_USER\Software\IM Providers\DefaultIMApp registry value
Right-click DefaultIMApp, and then click Modify.
Type Skype in the Value data box, and then click OK.
Exit Registry Editor, and then restart  the Skype client and the Office applications

Hope this helps

So today I was trying to running some cross domain PowerShell commands on Exchange but kept getting the following error:

The operation couldn't be performed because object 'user' couldn't be found on 'Server_Name.Domain'

Which basically means the Domain Controller your referencing can only see your sub domain and nothing higher.  So to resolve run this before the command:

Set-AdServerSettings -ViewEntireForest $True

Want to record exactly what happens when you’re using the Exchange Management Shell? Use the Start-Transcript cmdlet. Anything that you do after you run this cmdlet will be recorded to a text file that you specify. To stop recording your session, use the Stop-Transcript cmdlet.

Notice that the Start-Transcript cmdlet overwrites the destination text file by default. If you want to append your session to an existing file, use the Append parameter:

Start-Transcript c:\MySession.txt -Append