“They’ve all gone home for the weekend” responds the janitor.
It can feel like that in Second Life. There is no telling when the problem of people not being able to log in due to “stuck presence” actually started (best guess is inventory maintenance on Thursday). It was posted to Grid Status at 10:15 am on Saturday. I am writing this over 24 hours later.
The number of people in second life has understandably always been highest on the weekends. At Linden Lab that appears to be when support of all types is weakest, especially the kind of technical support needed to fix things that are broken in the code that makes SL work.
In the beginning (Linden World beta) that was an advantage. Stellar Sunshine built the beanstalk when LL went home for the weekend and left the servers on. We have all come a long way since those days. Second Life is 60 to 70 thousand or more residents on the weekends these days. Service should be at it’s best on the weekends and if there is an issue, all staff should be on 24 hour recall if they are needed. The Scouts in customer support (some of my favorite people) need to be especially well manned when the Lindens are away.
The thing Linden Lab finds hardest to change is transitioning from their comfortable geekiness of software development to the reality of customer service. It is NOT OKAY to let a problem that is preventing residents from logging in languish until Monday when everyone comes back to work!
Note: for now, if you see someone, or know of someone stuck in “presence limbo” pay them L$1, that usually will kick them to logoff.
Customers don’t expect you to be perfect. They do expect you to fix things when they go wrong.
Donald Porter, V.P. British Airways
UPDATE 1/31/2012: Timeline:
- Oskar Linden acknowledges the issue; 26/Jan/2012 3:27 PM
- Problem initially posted to SL jira; 27/Jan/2012 6:40 AM
- Posted by Status Desk; 28/Jan/2012 10:15 AM
- Rolling restarts begun on main channel; 31/Jan/2012 5:04 AM (completed 31/Jan/2012 10:24 AM)
- Hopefully the issue is resolved.
To me, taking 5+ days to resolve an issue that is preventing your customers from accessing your product is unacceptable! Not to mention the consequences of pissing off many more people who’s friends are unable to get into SL on the weekend. A better short term fix would have been a rollback as soon as the issue was understood.