An oldie but a goodie. If you think you’re getting statistically significant results from A/B testing, you should probably consider doing some A/A testing.
In an A/A test, you run a test using the exact same options for both “variants” in your test.
This could give a big boost to web performance!
I feel like my problem with design in general today is that folks want to burn everything to the ground and start again all the time. Whether that’s with a website, or a new web standard, or a political policy. They don’t want to fix what’s wrong with things bit by bit, everyone wants Thing 2.0 whilst jumping over all the small improvements that are required to get there.