We wanted to let you know about a recent issue that affected some Windows machines following a security fix we deployed.
The work was intended to fix a known Windows configuration weakness called an unquoted service path. In plain English, this is where Windows stores the startup location for a service in a way that can be misread if the path contains spaces. Correcting this helps reduce a potential local security risk.
Information on this can be found here https://isgovern.com/blog/how-to-fix-the-windows-unquoted-service-path-vulnerability/
As part of our normal security maintenance, we prepared and tested a script to find affected services and update their paths.
The security fix itself was valid, but an older version of our remediation script was selected and deployed in error. That older script had a very similar name to the current approved version and had not been removed from our script vault.
On some machines, the script updated the main service path but did not preserve all of the additional startup information required by certain Windows services.
As a result, affected machines could appear normal until they were restarted. After rebooting, Windows tried to start using incomplete service details, which caused some machines to fail to boot correctly.
Once we identified the issue, we took two actions:
- For machines that had not yet restarted, we deployed a corrective update remotely before the next reboot.
- For machines that had already restarted and failed to boot, we recovered them using System Restore where possible. (3 machines did not have system restore for reasons we are still looking into)
This was not caused by a virus, external attack, or user action. It was caused by an internal deployment error, and we are sorry for the disruption caused.
We have reviewed what happened and will be making changes to our script management process, including removing outdated scripts, improving script naming, marking superseded scripts more clearly, and adding extra checks before deploying scripts that affect Windows startup or service configuration.
Please contact us if you have any questions or notice anything unusual following this work.