SQL 2008 Patch Setup Fails if Patch Source is on a Shared Disk

I recently worked on a case where Service Pack 2 for SQL 2008 clustered instance failed with this error (Summary.txt)

Overall summary:
  Final result:                  The patch installer has failed to update the shared features. To determine the reason for failure, review the log files.
  Exit code (Decimal):           -2068709375
  Exit facility code:            1202
  Exit error code:               1
  Exit message:                  The directory ‘S:\Bits\SP2\’ doesn’t exist.
  Start time:                    2012-02-03 08:02:17
  End time:                      2012-02-03 08:02:55
  Requested action:              Patch

Here is the failure stack from Detail.txt

2012-02-03 08:02:53 Slp: Copying S:\Bits\SP2\ to C:\Program Files\Microsoft SQL Server\100\Setup Bootstrap\Update Cache\KB2285068\ServicePack …
2012-02-03 08:02:53 Slp: Error: Action "CachePatchPage" threw an exception during execution.
2012-02-03 08:02:53 Slp: Microsoft.SqlServer.Setup.Chainer.Workflow.ActionExecutionException: The directory ‘S:\Bits\SP2\’ doesn’t exist. —> Microsoft.SqlServer.Chainer.Infrastructure.ChainerInvalidOperationException: The directory ‘S:\Bits\SP2\’ doesn’t exist.
2012-02-03 08:02:53 Slp:    at Microsoft.SqlServer.Configuration.MsiExtension.CachePatchPackageAction.CachePatchPackage(String patchPackagePath, ProductProperty patchSummary)
2012-02-03 08:02:53 Slp:    at Microsoft.SqlServer.Configuration.MsiExtension.CachePatchPackageAction.ExecuteAction(String actionId)
2012-02-03 08:02:53 Slp:    at Microsoft.SqlServer.Chainer.Infrastructure.Action.Execute(String actionId, TextWriter errorStream)
2012-02-03 08:02:53 Slp:    at Microsoft.SqlServer.Setup.Chainer.Workflow.ActionInvocation.InvokeAction(WorkflowObject metabase, TextWriter statusStream)
2012-02-03 08:02:53 Slp:    at Microsoft.SqlServer.Setup.Chainer.Workflow.PendingActions.InvokeActions(WorkflowObject metaDb, TextWriter loggingStream)
2012-02-03 08:02:53 Slp:    — End of inner exception stack trace —
2012-02-03 08:02:53 Slp:    at Microsoft.SqlServer.Setup.Chainer.Workflow.PendingActions.InvokeActions(WorkflowObject metaDb, TextWriter loggingStream)
2012-02-03 08:02:53 Slp: Received request to add the following file to Watson reporting: C:\Users\dblogon\AppData\Local\Temp\2\tmpE186.tmp
2012-02-03 08:02:53 Slp: The following is an exception stack listing the exceptions in outermost to innermost order
2012-02-03 08:02:53 Slp: Inner exceptions are being indented
2012-02-03 08:02:53 Slp:
2012-02-03 08:02:53 Slp: Exception type: Microsoft.SqlServer.Chainer.Infrastructure.ChainerInvalidOperationException
2012-02-03 08:02:53 Slp:     Message:
2012-02-03 08:02:53 Slp:         The directory ‘S:\Bits\SP2\’ doesn’t exist.
2012-02-03 08:02:53 Slp:     Data:
2012-02-03 08:02:53 Slp:       ChainerInvalidOperationExceptionData = S:\Bits\SP2\
2012-02-03 08:02:53 Slp:     Stack:
2012-02-03 08:02:53 Slp:         at Microsoft.SqlServer.Configuration.MsiExtension.CachePatchPackageAction.CachePatchPackage(String patchPackagePath, ProductProperty patchSummary)
2012-02-03 08:02:53 Slp:         at Microsoft.SqlServer.Configuration.MsiExtension.CachePatchPackageAction.ExecuteAction(String actionId)
2012-02-03 08:02:53 Slp:         at Microsoft.SqlServer.Chainer.Infrastructure.Action.Execute(String actionId, TextWriter errorStream)
2012-02-03 08:02:53 Slp:         at Microsoft.SqlServer.Setup.Chainer.Workflow.ActionInvocation.InvokeAction(WorkflowObject metabase, TextWriter statusStream)
2012-02-03 08:02:53 Slp:         at Microsoft.SqlServer.Setup.Chainer.Workflow.PendingActions.InvokeActions(WorkflowObject metaDb, TextWriter loggingStream)
2012-02-03 08:02:53 Slp: Watson Bucket 1
 Original Parameter Values
2012-02-03 08:02:53 Slp: Parameter 0 : SQL2008@RTM@KB2285068
2012-02-03 08:02:53 Slp: Parameter 1 : Microsoft.SqlServer.Configuration.MsiExtension.CachePatchPackageAction.CachePatchPackage
2012-02-03 08:02:53 Slp: Parameter 2 : Microsoft.SqlServer.Configuration.MsiExtension.CachePatchPackageAction.CachePatchPackage
2012-02-03 08:02:53 Slp: Parameter 3 : Microsoft.SqlServer.Chainer.Infrastructure.ChainerInvalidOperationException@1202@1
2012-02-03 08:02:53 Slp: Parameter 4 : Microsoft.SqlServer.Chainer.Infrastructure.ChainerInvalidOperationException@1202@1
2012-02-03 08:02:53 Slp: Parameter 5 : CachePatchPage

Service pack installer is an exe and can be launched in two ways:

i. Run the exe – In this case, exe would be extracted into some random directory and would be launched from there.
ii. Extract the exe manually (use /x or /extract command line parameters). From the extracted path, launch setup.exe. This method can be used to avoid MSI installer to extract the exe into some random drive/directory.

I had used second method to launch SP2 setup, hence the failure was guaranteed.

The problem here is, SP2 source files are located at S:\Bits\SP2\ which is a shared disk owned by SQL cluster group. By design, SQL Server patch setup takes all the cluster resources for SQL cluster group offline and then brings online few times during the patch installation. After we launched SP2 setup from S:\Bits\SP2\ it took U: itself offline and was further not able to find the SP2 installer files hence the error.

The resolution (or workaround I would say) is simple, use a local disk to launch SP2 setup.

Though there are few caveats:

1. I believe SQL Server patch setup should not update anything which would require disk resources to be taken offline. Hence such design can be improved.

2. If there is still a need for the disk resources to be taken offline, setup can had additional rule check to determine whether the patch source is a shared disk owned by SQL Server cluster group and if so, do not allow path installation to proceed.

3. Even if patch setup exe (without extracting it) is launched from a local disk, it “may” extract itself into a random drive/directory on one of the shared disks owned by SQL Server group and we hit the error.

I have filed a bug for this issue on connect. So far, MS has responded saying it is a “By Design” issue and recommends to run the patch from a local disk. I will update the blog if MS responds back on the other suggestions.

Stay tuned!