Optimizely no longer blocks page loading.
Existing Optimizely configuration settings do not change.
Initial value for the async wait timeout should be 0.5 seconds.
This is just the code change. Creating the experiment to verify this on staging is a separate ticket.
Background Info (from an email):
I've been staring at a lot of waterfall page load views from China and India lately, trying to figure out why Segment seems to be causing slower load times. The conclusion I eventually came to was that while Segment does contribute somewhat to longer page loads, it seems to have a fairly low impact on the actual user experience. Sometimes an integration like Qualaroo will give us an exaggerated page load time because it does something to alter the DOM thirty seconds after the page has loaded and been fully functional – but that's an instrumentation issue that we can address separately.
One thing that is actively hurting the user experience is Optimizely. In order to prevent a flickering effect where users see the original version before the experiment code kicks in, Optimizely will block page rendering altogether until it can load itself. While this works well in the U.S., it gets much slower internationally, particularly for initial loads. In a number of my traces, Optimizely added 1-4 seconds from China and India for the first page load (representing anywhere between 10-30%).
I originally thought that we might be more selective about when and where we include Optimizely on the server side, adjusting it for country of origin and active experiments. But I think I've found a simpler way:
Basically, we can load Optimizely asynchronously, and then prevent very late flickers by putting a timeout (the suggestion is one second). This should address the performance issue, but it will have a few consequences:
1. Users will notice a flicker on pages when an experiment is running. It will be brief but in many cases perceivable.
2. Users who can't download the script in time won't be included in the experiment.
3. Users who see one version of an experiment but then return with a much slower connection later (e.g. their phone), might get an inconsistent user experience.
This would be a lot more robust than any server side configuration code we could add, since it addresses the root problem directly. I believe that the tradeoff is worthwhile, and I would like to move forward with the change as soon as possible.
Becky, Patrick: As the main users and gatekeepers of Optimizely, I would really appreciate getting your thoughts on this. It's a simple enough change where I'd like to try to get it into next week's LMS release if you agree.