Perception And Performance Measurements

1. PERCEPTION AND PERFORMANCE MEASUREMENTS

2. As web pages continue to grow in size and users expect ever-faster-loading pages, measurements that accurately reflect the user’s experience are critical to understanding web performance. This had led to what some may call “the search for the elusive unicorn metric.” Many people with an interest in web performance are looking for a metric that will accurately measure page performance and user experience. This simply doesn’t exist today, and it might never exist, but that won’t stop us from searching for it. • It isn’t only about how fast all the elements on the page load, but rather how the human user perceives the page load. I don’t care how long it takes all the analytics and tracking widgets to load, I care about when the information I am looking for is available. • According to Jakob Nielsen, there are three main time limits to consider when it comes to web performance and user perception. • 0.1 second – limit for users to feel that user interface items in the page can be manipulated • 1 second – limit for users to feel they can navigate without having to wait for the computer • 10 seconds – limit to keep a user’s attention • Users need to feel something is happening on the page to stay engaged. Delays over one second are noticeable. A page has to have some sort of noticeable change in less than a second for a user to not be frustrated. In a previous post I wrote about objective vs subjective time – these thresholds need to be considered more for subjective rather than objective timings, but for the most part we focus on objective measures.

3. Traditional web performance metrics such as time to first byte (TTFB) or page load time (PLT) are commonly used to create baselines for determining how fast a page loads. Unfortunately these metrics do not provide insight into the user’s perception. TTFB is the time it takes between issuing a request for a resource and receiving the first byte of content. When the first byte of content is received by the browser, the user is still looking at a blank screen (or the previous page they were on). This metric does not help determine how the user is actually perceiving the page, as they still don’t even know that data is starting to be received. On the opposite end of the spectrum, PLT measures how long it takes for every element on the page to load – even if they are “below the fold” or otherwise invisible to the end user. Users may navigate away from a page before it has fully loaded, so this isn’t helpful in determining if a user perceives a page as being slow. Whenever I read a paper or report on how page load times are improved by a certain percentage, I immediately look to see what metric is being reported. In the majority of instances, it is page load time. This disappoints me. Luckily, user-centric measurements have emerged and are becoming more the norm than the exception in the web performance community. Instead of measuring PLT or TTFB, measurements such as start render, Speed Index, critical resources index, and Perceptual Speed Index have come to the forefront. These measurements aim to give us insight into how users perceive page load, which is more important than the actual milliseconds or seconds it took for the document to complete. Start Render Start render measures the moment when the first item was displayed on the screen – its size doesn’t matter, nor does its relevance to the end user. This is when the user is no longer staring at a blank white screen or at the previous screen. Start render is good, but this does not take into account whether the content displayed is relevant to the end user. There may be something on the page, but the page may still not be usable to the end user. Referring back to Nielsen’s considerations, the user still cannot manipulate the UI.

4. Speed Index Speed Index was added to WebPagetest in April 2012, as a way of measuring how quickly content above the fold is populated. Speed Index goes beyond tracking a specific timing milestone in the web application loading process and focuses on items that are most critical to the end user. Speed Index computes the visual completion of the application-loading process based on pairwise comparisons of pixel histograms from time lapse video snapshots, such as the snapshots in the strip below. Essentially each frame of the video is compared with the end frame to estimate how much visual completion has been achieved with respect to the fully-loaded frame. Critical Resources Index Critical resources index, developed by Steve Souders, combines two of the main blockers to rendering, style sheets and synchronous scripts, into a single metric. Measuring these critical resources as a single metric makes it easier to measure the combined impact on performance. Critical resources index is reported in HTTP Archive, as one of the metrics in the “Highest Correlation to Render Time” charts. Look for the CSS and Sync JS label:

5. Perceptual Speed Index Perceptual Speed Index (PSI) is a complementary measurement to Speed Index that accounts for layout stability. Speed Index answers the question of “how fast did above-the-fold content load?” while PSI answers the question “how fast did above-the-fold content load without jitter?” Stability includes performance concerns like whether there are multiple reflow and repaints, and whether there is visual interference by pop-up ads, but can also penalize intended page changes like animated GIFs and auto-running carousels (which some may argue should be penalized). You can’t improve what you can’t measure. But, you also need to make sure you are measuring the right thing. To learn more about this, download the Measuring Web Performance white paper from ZK Research. Thus far we haven’t found the unicorn metric/index of web performance measurements. Maybe one of these metrics can bring us closer to ending the search. Next Steps Taking human visual perception into account is a critical factor in estimating the quality of experience of a web application. There is still work to be done to find measurements that accurately measure user perception of when a page becomes responsive. I look forward to the day when one of these metrics or possibly another one replace TTFB and page load time.

6. Instart Logic is the world’s first endpoint-aware application delivery solution that makes websites and applications fast, secure, and easy to operate. Interested in learning more? Preview our image optimization capabilities in the Playground Contact Sales