Examples

Performance Split A-B Testing with PageSpeed

Run an experiment on 30% of visitors where half get the default configuration and half get no filters. Use Google Analytics ID UA-XXXXXXX-Y to run the experiment, and default to logging the experiment information into custom variable slot 1:


                pagespeed RunExperiment on;
                pagespeed AnalyticsID UA-XXXXXXX-Y
                pagespeed ExperimentSpec "id=1;percent=15;default"
                pagespeed ExperimentSpec "id=2;percent=15

Run an experiment on 30% of visitors where:

One tenth of visitors get the default configuration.
One tenth get a new configuration which has the core filters plus inline_preview_images and remove_comments without rewrite_javascript, with AvoidRenamingIntrospectiveJavascript disabled, and a CSS inlining limit of 4kB.
One tenth get no filters.
The remaining 70% aren't included in the experiment.

All the other visitors will be unaffected. For measurement it logs to custom variable slot 3 of Google Analytics account UA-XXXXXXX-Y.


                pagespeed RunExperiment on
                pagespeed AnalyticsID UA-XXXXXXX-Y
                pagespeed ExperimentVariable 3
                pagespeed ExperimentSpec "id=3;percent=10;default"
                pagespeed ExperimentSpec "id=4;percent=10;level=CoreFilters;
                enabled=inline_preview_images,remove_comments;disabled=rewrite_javascript;
                options=AvoidRenamingIntrospectiveJavascript=off,CssInlineMaxBytes=4096"
                pagespeed ExperimentSpec "id=5;percent=10"

While the RunExperiment feature supports testing many changes, as shown above, the discipline of changing only one variable at a time helps you see the effect and know where it's coming from. The only drawback is that sometimes two filters are more useful in combination, such as minification and inlining, where minifying brings resources down to the inlining threshold.

There is also a tradeoff with the experiment percentages: a larger percentage means you will collect sufficient data for meaningful results more quickly but it also affects more of your users. If you're testing manually with query parameters to be sure your settings aren't making things worse for you, the downside of running with large experiment percentages is low. While you may not have determined the ideal settings right away, you're unlikely to have made things much worse.