Tech Blog

WSUS console kept crashing with the reset console error:

The WSUS administration console was unable to connect to the WSUS Server via the remote API. 

Verify that the Update Services service, IIS and SQL are running on the server. If the problem persists, try restarting IIS, SQL, and the Update Services Service.

System.Net.WebException -- The operation has timed out

Source
System.Web.Services

Stack Trace:
   at System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(WebRequest request)
   at Microsoft.UpdateServices.Internal.DatabaseAccess.ApiRemotingCompressionProxy.GetWebResponse(WebRequest webRequest)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.UpdateServices.Internal.ApiRemoting.ExecuteSPGetUpdateServerStatus(Int32 updateSources, Boolean includeDownstreamComputers, String updateScopeXml, String computerTargetScopeXml, String preferredCulture, Int32 publicationState, Int32 propertiesToGet)
   at Microsoft.UpdateServices.Internal.DatabaseAccess.AdminDataAccessProxy.ExecuteSPGetUpdateServerStatus(UpdateSources updateSources, Boolean includeDownstreamComputers, String updateScopeXml, String computerTargetScopeXml, String preferredCulture, ExtendedPublicationState publicationState, UpdateServerStatusPropertiesToGet propertiesToGet)
   at Microsoft.UpdateServices.Internal.BaseApi.UpdateServer.GetStatus(UpdateSources updateSources, Boolean includeDownstreamComputers, UpdateScope updatesToInclude, ComputerTargetScope computersToInclude, UpdateServerStatusPropertiesToGet propertiesToGet)
   at Microsoft.UpdateServices.Internal.BaseApi.UpdateServer.GetReplicaStatus(UpdateSources updateSources)
   at Microsoft.UpdateServices.UI.AdminApiAccess.CachedObject.RefreshCache()
   at Microsoft.UpdateServices.UI.AdminApiAccess.CachedObject.GetFromCache()
   at Microsoft.UpdateServices.UI.SnapIn.Pages.ServerSummaryPage.backgroundWorker_DoWork(Object sender, DoWorkEventArgs e)

So after doing all the normal IIS stuff, got Microsoft involved and here’s what they did to fix it:

• We checked and ensured that none of the services crashed or stopped after console crashes,
• We checked and ensured that WSUSPool was also running,
• We checked and ensured that SQL service was also running on the remote server,
• We increased the WSUSPool PrivateMemoryLimit to unlimited by setting it to ‘0’ on the IIS console
• We also increased the Maximum queue length from 1000 to 3000,
• We stopped WSUSPool and Restarted the IIS service,

We ran below mentioned on SQL to check and find if there are Obsoletes update to clean up but found none

exec spGetObsoleteUpdatesToCleanup

• We ran SQL query to Re-index SUSDB and tried to connect to the console https://gallery.technet.microsoft.com/scriptcenter/6f8cde49-5c52-4abd-9820-f1d270ddea61#content
• We changed WSUS to use port 80 using command below

• WSUSUtil.exe usecustomewebsite false

• WSUS was successfully changed to port 80
• We tried to connect to the console but t still fails the same way

• We found that console was failing to connect with below error as well
• We removed the WSUS mmc from folder %appdata%\Microsoft\MMC\
• We tried to connect again to WSUS console but with no avail

• We further investigated and ensured that WSUS s failing to connect with SUSDB as it is timing out
• We checked and found that SUSDB size was more than 6.5 GB
• We restarted the SQL Service for instance hosting SUSDB
• We tried to connect to WSUS console after restarting the service and it connected successfully
• We see that there were more than 13,000 approved updates on WSUS console
• This could cause an issue for WSUS to connect as it would take a long time to connect with SUSDB
• We decided to run PowerShell script to decline superseded updates from WSUS to enhance the performance of WSUS while connecting to SUSDB
• We ran PowerShell script from below mentioned link to decline superseded updates
  https://blogs.technet.microsoft.com/configurationmgr/2015/04/15/support-tip-configmgr-2012-update-scan-fails-and-causes-incorrect-compliance-status/
• Superseded updates were declined successfully

PS C:\Scripts> cd e:
PS E:\> cd .\ms\
PS E:\ms> .\sus.ps1 -updateserver WSUS_SERVER -port 80
Connecting to WSUS server WSUS_Server on Port 80... Connected.
Getting a list of all updates... Done
Parsing the list of updates... Done.
List of superseded updates: E:\SupersededUpdates.csv
Summary:
========
All Updates = 17101
Any except Declined = 15718
All Superseded Updates = 9621
    Superseded Updates (Intermediate) = 6801
    Superseded Updates (Last Level) = 2820

SkipDecline flag is set to False. Continuing with declining updates
 DeclineLastLevel is set to False. Declining all superseded updates.
 Declined 9621 updates.
 Backed up list of superseded updates to E:\SupersededUpdatesBackup.csv

• We changed the WSUS to use customwebsite port 8530 again

• WSUSUtil.exe usecustomewebsite true

   

• WSUS was successfully changed to port 8530
• We tried to open WSUS console and it connected successfully
• We ran synchronization on WSUS server and it completed successfully
• BOOOM WOOP

All our users kept getting:

There was a problem, but sync will try again.

(0x80c80317)

On the server we kept getting the following event:

The Windows Sync Share service failed to setup a new sync partnership with a device. Database: \\?\S:\users\SyncShareState\WorkFolders\Metadata; User folder name: \\?\S:\WorkFolder\WORKFOLDERS_ROOT\USER.TEST; Error code: (0x8e5e0408) Unable to read from or write to the database.

To fix it:

• I failed the roles over and rebooted both nodes of the cluster, nothing.
• I disabled restarted the sync share through the server admin console,
• I’ve tried to rename the metadata on the client:

C:\Users\User\AppData\Local\Microsoft\Windows\WorkFolders\Metadata

• I’ve tried repairing a user which seems broken:

Repair-SyncShare -name workfolders -user Domain\test

Repair-SyncShare : (0x80c80317) There was a problem, but sync will try again.

At line:1 char:1

+ Repair-SyncShare -name WorkFolders -user Domain\test

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

 + CategoryInfo: NotSpecified: 
(Msft_SyncShare (Name = "MA_Userfiles"):Root/Microsoft/.../Msft_SyncShare) [Repair-SyncShare], CimException    
+ FullyQualifiedErrorId : HRESULT 0x80c80317,Repair-SyncShare

So here’s the weird thing, I tried the following which seemed to fix it (I’ve no idea why):

Get-SyncUserStatus -User domain\test -syncshare WorkFolders

So to apply to all users (which also worked), first I gained the users from the AD group I used (I dont have the AD functions on my work folders server):

Get-ADGroupMember -Identity "Workfolders Users" | 
select SamAccountName | Export-Csv c:\temp\WorkFolders.csv

Then used the CSV to apply the Get-SyncuserStatus to all users:

$List = Import-Csv C:\temp\WorkFolders.csv
foreach ($user in $list) {
Get-SyncUserStatus -User $user.SamAccountName -syncshare WorkFolders}

Recently our WSUS console kept crashing either when opening certain computer folders or All Computers.

Error: Unexpected error

So after lots of frustration I rebuild WSUS on Server 2016 and using a proper SQL instance:

Uninstall WSUS PowerShell:

Uninstall-WindowsFeature -Name UpdateServices,Windows-Internal-Database -Restart

Then re-install with powershell:

Install-WindowsFeature -Name UpdateServices-Services,UpdateServices-DB –IncludeManagementTools

Then set the SQL instance using a elevated CMD:

WsusUtil.exe PostInstall SQL_INSTANCE_NAME="SQLSERVER\INSTANCE" CONTENT_DIR=E:\WSUS

The servers all started populating, great.

Then FAIL again.

The WSUS administration console has encountered an unexpected error. This may be a transient error; try restarting the administration console. If this error persists, 

Try removing the persisted preferences for the console by deleting the wsus file under %appdata%\Microsoft\MMC\.


The WSUS administration console has encountered an unexpected error. This may be a transient error; try restarting the administration console. If this error persists, 

Try removing the persisted preferences for the console by deleting the wsus file under %appdata%\Microsoft\MMC\.


System.Xml.XmlException -- '', hexadecimal value 0x16, is an invalid character. Line 1, position 779977.

Source
System.Xml

Stack Trace:
   at System.Xml.XmlTextReaderImpl.Throw(Exception e)
   at System.Xml.XmlTextReaderImpl.ParseNumericCharRefInline(Int32 startPos, Boolean expand, StringBuilder internalSubsetBuilder, Int32& charCount, EntityType& entityType)
   at System.Xml.XmlTextReaderImpl.ParseCharRefInline(Int32 startPos, Int32& charCount, EntityType& entityType)
   at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
   at System.Xml.XmlTextReaderImpl.ParseText()
   at System.Xml.XmlTextReaderImpl.ParseElementContent()
   at System.Xml.XmlReader.ReadStartElement()
   at System.Xml.Serialization.XmlSerializationReader.ReadStringValue()
   at System.Xml.Serialization.XmlSerializationReader.ReadTypedPrimitive(XmlQualifiedName type, Boolean elementCanBeType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderApiRemotingCompressionProxy.Read1_Object(Boolean isNullable, Boolean checkType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderApiRemotingCompressionProxy.Read2_GenericReadableRow(Boolean isNullable, Boolean checkType)
   at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReaderApiRemotingCompressionProxy.Read249_Item()
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
** this exception was nested inside of the following exception **


System.InvalidOperationException -- There is an error in XML document (1, 779977).

Source
System.Xml

Stack Trace:
   at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at Microsoft.UpdateServices.Internal.ApiRemoting.ExecuteSPSearchComputers(String computerTargetScopeXml)
   at Microsoft.UpdateServices.Internal.DatabaseAccess.AdminDataAccessProxy.ExecuteSPSearchComputers(String computerTargetScopeXml)
   at Microsoft.UpdateServices.Internal.BaseApi.ComputerTarget.SearchComputerTargets(ComputerTargetScope searchScope, UpdateServer updateServer)
   at Microsoft.UpdateServices.UI.AdminApiAccess.ComputerTargetManager.GetComputerTargets(ComputerTargetScope searchScope)
   at Microsoft.UpdateServices.UI.AdminApiAccess.BulkComputerPropertiesCache.GetAndCacheComputers(ExtendedUpdateScope updateScope, ComputerTargetScope computerTargetScope)
   at Microsoft.UpdateServices.UI.SnapIn.Pages.ComputersListPage.GetListRows()

After some Googling I found the problem seems to be with a corrupt PC checking in, i.e. the PC checks in, then via WMI all the info is gathered on the PC and if that info is corrupt then WSUS crashes!!

To fix,

Copy the WSUS console error into Notepad++ (don’t use anything else) and look for:

System.Xml.XmlException -- '', hexadecimal value 0x16

Basically this means that the weird square symbol or SYM symbol is present in the WSUS database and causing WSUS to shit itself:

So we need to find it, open SQL Management Studio and open the SUSDB

Note: if you’re not using proper SQL then open the DB via:

\\.\pipe\MICROSOFT##WID\tsql\query

Changed the default view setting by going to the Tools > Options > SQL Server Object Explorer settings, then adjusted the ‘Value for Edit Top <n> Rows command’ from 200 to 0, making the option display all rows.

Navigate down within the SUSDB database, to the dbo.tbComputerTargetDetail table, right-clicked it, and selected ‘Edit All Rows’.

Click the top left corner of the column/rows to select everything, then copy-pasted it into NotePad++

Then copy the SYM from the error and search for it in the info you’ve just copied:

As you can see the ID on the left is 525:

Lets now cross reference this against another table in the SUSDB database via SQL script:

Select * from dbo.tbComputerTarget

Look for the ID:

Thats the PC!!!

Find it and update the BIOS and anything else you can, then get it to report in again to WSUS:

wuauclt /scannow
wuauclt /detectnow
wuauclt /r /ReportNow

Credit to SterlingT, he’s a wonderful human being:

https://social.technet.microsoft.com/Forums/windowsserver/en-US/90dc15d3-c498-42b8-b36a-bd29be35cf99/wsus-console-unexpected-error-when-choosing-all-computers-folder?forum=winserverwsus