How RUM Works
Real User Measurement (also known by its acronym, RUM) collects performance (and other) information about the real user experience of a web page load or other navigation on your site. It does this by asking your visitors’ own web browsers to report on their experiences by sending a beacon of data to a collection repository where it can be aggregated and analyzed. All RUM systems accomplish this through JavaScript, and there are four primary steps involved.
First, a JavaScript loader snippet or tag is placed on every page that you want to have measured. This small chunk of code executes during the page load to kick things off. Next, this loader snippet will request a core JavaScript library that contains the logic to interface with the browser through W3C standard APIs for data collection. For most RUM platforms, this core JavaScript library is a variant of the open source Boomerang software. Next, a configuration file (typically in JSON format) loads in the browser that has site and/or page-specific data collection config details. Finally, when the page load or navigation is complete, the JavaScript that has loaded interacts with the browser through W3C APIs and other standards to assemble a “beacon” of data that is delivered to a data repository and visualization system.
Where the Data Comes From
RUM systems collect data from the browsers using a variety of industry standard JavaScript APIs. The core data collection relies upon four standard APIs defined by the World Wide Web Consortium (W3C) These standards have been adopted by all major browsers:
- Navigation Timing API
- Resource Timing API
- Paint Timing API
- User Timing API
You should encounter very, very few real users who are still on browsers that predate these standards. Google browsers have been the earliest adopters of these standards. Microsoft browsers have supported these standards wince 2012. The final major browser to adopt Navigation Timing was Apple Mobile Safari, which included support beginning with version 8 in 2014. For details on browser adoption: https://www.caniuse.com/.
What Kind of Timers to Measure?
“Page Load Time” is just the starting point when it comes to Real User Measurements. Depending on the RUM platform, data collection may include some or all of the following timers and metrics. Which ones you focus on may change depending on what problems you are trying to solve with the data. Highly popular at the moment are the “core web vitals” measurements of Largest Contentful Paint, Cumulative Layout Shirt, and First Input Delay. These three measurements speak more directly to the user experience than traditional timers that speak to the network or infrastructure performance.
- Time to First Byte
- First Contentful Paint
- Largest Contentful Paint
- DOM Loading
- Time to Visually Ready
- Page Load Time
- Time to Interactive
- Time to First Interaction
- First Input Delay
- Cumulative Layout Shift
- DNS Lookup Time
- TCP Connection Time
- SSL Connection Time
- Resource Load Time
- CDN Edge Delivery Time
- Origin Time
- User Timers
- Rage Clicks
What Kind of Dimensions to Capture?
In addition to capture timers and other performance-related data, any effective RUM solution will capture a wide variety of dimensional data as well. Performance is almost never uniform across all kinds of site content or the wide variety of site visitors. Dimensional breakdowns are critical to finding those “soft spots” in the user experience on your site. Depending on the RUM platform, data collection may include some or all of the following dimensional data:
- Geographic location
- Browser
- Operating System
- Device Details
- Page Group, Page Name, or Page Type
- IP Version
- TLS Version
- ISP
- Network Connection Type
- Multivariate Test and Test Segment
- Custom Dimensions
- Visibility State
RUM and Business Intelligence
While all RUM platforms offer real-time dashboards, basic reports, and alerts, one of the most powerful uses of a RUM system is the ability to connect your visitors’ page load or navigational experiences with the business outcome. You can use RUM data to answer questions like “How fast do I need to be to maximize my primary business metric?” or “What pages on the site are most impactful to my primary business objective?”.
We have the ability to answer these questions because RUM system collect performance data from visitors at the exact same time that it can collect business metric information. For example, if your site is an online retail service, then your business objective is to convert a visitor into a purchaser, and you can measure the success of that objective for each visit by whether the visit includes an order confirmation page view or not. By looking at the web performance experienced by visitors who convert, and comparing it to those who do not, you can gain significant insights to help your business.
The most advanced RUM services and consultants can even prepare data models to let you extrapolate from current site experiences to possible future results. We call this “What If” modeling, and it can be a powerful tool for planning, or even simply justifying, your investment in web performance