Service not reliably starting under Windows 10

[acronce]

We're evaluating Viscosity as a replacement client for our current VPN. Unfortunately we're seeing an issue where the service is not starting reliably after a reboot.

Basically the Viscosity service fails to start up quickly enough and Windows 10 tells it to shut down. We've tried using the “Automatic (delayed start)” configuration for the service but by the time the service starts, the application has given up trying to connect.

It could be that this problem only happens on slow systems, where starting the service takes longer than some arbitrary OS defined timeout.

This is an issue that we've seen with our own product's Windows service, so it's very familiar. We tried all sorts of things to work around it, including changing our service to return to the OS quickly and initializing separately in a thread. None of these changes mattered. We still had reported cases of our service not starting for some users on slow systems.

In the end the only work around was to register our service so that when a client tries to access one of our named pipes, our service is notified and started. So even on slow systems where the service fails to start at boot, the first time a client tries to access our service it will get started.

Since making that change, we have no reported issues from end users of our service not starting.

It's unclear to us if any of this is relevant to your Viscosity implementation. But if it is, you might want to consider making this change in a future version.

[Eric]

Hi acronce,

Unfortunately our service is privileged so a standard user can't start the service nor do any automated methods work for us. If the service is detected as not running when a user tries to use it, they will be prompted to click Yes to elevate and start the service then, so we do have a method in place.

We do testing on some pretty archaic hardware, the only scenario's we've found where the service times out or doesn't start is actually when it's delayed by AV software trying to inject and scan the process. On slower hardware if AV software can't keep up it will queue software start procedures which is where most of the delay comes from. Viscosity's workload is quite low on boot, so a normal, healthy system should not have any issues.

Regards,
Eric

[acronce]

Thanks for getting back to me.

I realize that this problem is probably rare. But it affected the very first Windows offsite employee of ours (a Windows developer) who tried to use Viscosity. My hope is that it does not affect our other Windows users.

Our service is also privileged. Our work around was not to have the client start the service (which isn't possible without asking the user for permission). Our solution was to register the service to be started if a client tries to access one of the service's named pipes.

All the client has to do is try to use one of the named pipes (as it normally would) and the OS starts the service if it wasn't already started. This all happens automatically and under the hood without needing any user interaction.

Since making this change, we have not had a single support incident about our service not starting.

I don't know how your client communicates with your service. If it uses named pipes, then this solution would be directly applicable. If you use some other mechanism, there might be a similar way for the OS to sense when the client is trying to talk to your service, starting it as needed.

Hope this helps...

[Eric]

Hi acronce,

Are you referring to WAS? If so this isn't something we can use. We're certainly open to suggestions if you know of another way.

Regards,
Eric

[acronce]

Hi Eric,

Thanks for your continued support.

No, we didn't use WAS. The technique we used is called "Service Trigger Events". For our service this amounted to adding a sub-key in the registry at install time named "TriggerInfo" contains the setup parameters needed by "Service Trigger Events".

I think that trigger events require Windows 7 or newer. That was no problem for us because we don't support earlier OS versions anyway.

As an aside, I've got a bit more information from our offsite developer that keeps seeing the problem. When he manually starts the Viscosity app, you are right that it also starts the service. What he can’t do is reliably automatically run Viscosity at Windows startup/login because the service doesn't start.

Note that his machine is a 64-bit Intel i7-5500U (4 logical processors) at 2.4 GHz with 8GB RAM running Windows 10. Doesn't seem like that machine would be particularly slow.

Here's the event log, in case it's helpful:

Log Name: System
Source: Service Control Manager
Date: 9/20/2017 10:26:15 AM
Event ID: 7000
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: FHP-Inspiron17
Description:
The ViscosityService service failed to start due to the following error:
The service did not respond to the start or control request in a timely fashion.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" />
<EventID Qualifiers="49152">7000</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8080000000000000</Keywords>
<TimeCreated SystemTime="2017-09-20T14:26:15.329500500Z" />
<EventRecordID>19564</EventRecordID>
<Correlation />
<Execution ProcessID="912" ThreadID="2596" />
<Channel>System</Channel>
<Computer>FHP-Inspiron17</Computer>
<Security />
</System>
<EventData>
<Data Name="param1">ViscosityService</Data>
<Data Name="param2">%%1053</Data>
<Binary>56006900730063006F00730069007400790053006500720076006900630065000000</Binary>
</EventData>
</Event>

[Eric]

Hi acronce,

Thanks for the information, we'll take a look when we can. We played with manual triggered starts in the past but we had security concerns and dropped the idea early. We'll take another look.

Ideally we need the service running at boot for a variety of reasons, mainly to recover the network if the computer crashes or blue screens while connected. If the network is taking too long to come up this could also be an issue. We've had issues putting dependencies on other services to start first in the past, though this was mainly on Vista and we have since dropped support.

As a work around for now, we recommend you simply increase the timeout for the service start in registry to something like 5 minutes. If you're seeing this regularly on your fleet I would assume the underlying problem is another piece of software or hardware which is delaying the machine at boot.

Regards,
Eric

[acronce]

Hopefully the triggers angle pans out, when you guys get a chance to look at it. If you revisit it I doubt you'll find any security related issues. If it's OK to start your service automatically at boot, then it's should be OK to start it when your client tries to use your named pipe (assuming you have one).

FYI, our service does depend on the network. It's the only dependency that our service has currently. Specifically the dependency is:

"TCP/IP Protocol Driver"

We added that dependency about two years ago and have seen no problems with it. But again, we require Windows 7 and above. Note that we have a very large user base. I'm pretty sure that the above dependency is safe on modern OS's.

I'll let our offsite developer know about the 5 minute trick. Although it's just as simple for him to add the dependency on TCP/IP. Maybe I'll ask him to try that first and report back.

Thanks for your continued help.