Testing Methodology

In order to ensure fair comparisons, we follow a very strict methodology when testing hosts. This details the methodology and tools we use.

We will never:

  • Modify or edit results, images, or data
  • Unfairly compare two hosts (ie, Australia Server vs USA Server)
  • Fabricate or lie about testing results, plans, and process
  • Perform optimizations on a site for only one host
  • Use or add external, non-included services for only one host (ie, SpeedKit, Cloudflare, etc)
  • Perform testing without actually having access to a hosting plan

When possible, we will:

  • Archive speed tests on archive.org or archive.is and provide full report links on request.
  • Host both plans on the same domain and DNS resolver service (StackPath Managed DNS)
  • Disable/enable similar functionality at the same time (caching, CDN, etc)
  • Conduct speed tests at nearly identical times

We use the following tools to benchmark and speed test a website:

While we generally do not use Pingdom, GTMetrix, or WebPageTest.org due to the lack of multi-region capabilities, we may occasionally use such results under special circumstances, ie, tiebreaker.

When purchasing or acquiring hosting plans from other sites, we will do our best to choose the plan that is most similar to CynderHost High-Performance Basic in terms of pricing, and storage and bandwidth. A server in the US region will always be chosen.

Our testing consists of two separate sites running WordPress 5.4.2:

  • Default WordPress TwentyTwenty theme with no plugins installed and three posts
  • WordPress Astra Theme with Woocommerce and Elementor + 20 other addons/plugins with 4 products

Sites are migrated and fully cloned (database, wordpress install, wp-content files) using the WPVivid Backup plugins. The plugin is left activated throughout testing.

If possible, PHP version 7.3 is used. If this version is not available, we fall back to 7.4, then 7.2, then 7.1.

Host-installed plugins for site management, login, and connection are left installed, as are must-use plugins from the host. If the host provides separate, custom plugins for speed optimization, we will conduct tests with the plugin activated and deactivated and include the results of such tests. If they recommend the use of a non-custom plugin, such as WP Super Cache, we will not use this plugin.

We will first conduct a speed test on the first site’s homepage using Dotcom-Tools from all 10 USA locations. Should any region yield “unavailable” or “error” in dotcom-tools, or any 1 region yield a significantly higher load time that is abnormal, we will try again, up to 3 retries, after which we will use the latest speed test.

For the second website’s homepage, we will use FastOrSlow, as Dotcom-Tools limits the number of tests you can run per domain. We test a website in a similar fashion to above. Each site is allowed up to 3 retries should there be any error or abnormal load times.

For both tests, if the host has a server-side caching function enabled, we will test the site with the cache on and off. To test cache hits, we will run the respective testing tool twice. Once to prime the cache, and once to test cache-hit speeds. Only the second result is used.

If the host provides some sort of included CDN, we will also test it with the CDN and without the CDN. The only exception to this is if the host uses Cloudflare as a CDN, as it requires a full update of our nameservers. If this is the case, we will also set our CDN to full pass-through mode, where nothing is cached and our CDN is not active.

In all tests, urls and resources are loaded through HTTPS, not http.

We are dedicated to being fully transparent and fair in our testing. If you have any issues or questions, please feel free to leave a comment in the article in question and we will publicly respond, or you can email us at feedback@cynderhost.com