UA-Radar: Exploring the Impact of User Agents on the Web

What is it about?

In his presentation at the Workshop on Privacy in the Electronic Society during ACM CCS 2023 in Copenhagen on November 26, 2023, Jean Luc Intumwayase introduced his paper, "UA-Radar: Exploring the Impact of User Agents on the Web." The paper, available on arXiv, examines the relevance of User Agents (UAs) on the web today. When you visit a website, your web browser sends a small piece of information called the User Agent (UA) to the web server. This line of text identifies the browser you're using, its version, and your device's operating system. In the early days of the web, UAs were vital for the functioning of websites. Different browsers displayed web pages in their own ways, often causing compatibility issues. A site that looked fine in one browser might break or display incorrectly in another. The UA helped websites identify the browser and adjust their content, ensuring the webpage appeared as intended, regardless of the browser used. Nowadays, with advanced web standards, most websites look and function similarly across different browsers. This has made the original purpose of UAs — ensuring web pages display correctly — largely unnecessary. Instead, concerns are growing about privacy, particularly how UAs are used in browser fingerprinting. Browser fingerprinting is a web tracking technique that websites use to uniquely identify and track users across different sites. It combines various details of your browser, such as installed plugins, fonts, the user agent, and other settings, to form a unique 'fingerprint'. This all happens without the user’s knowledge, making the technique difficult to block or control and raising significant privacy concerns. Jean Luc and his team set out to explore the relevance of UAs on the web. They replaced all identifiable information in standard browsers with the word ‘None’, naming the resulting browsers ‘None-browsers’. The browsers modified were Chromium, Firefox, and WebKit. Information changed in these browsers included the browser’s name, version, operating system, and any other information not common among the three browsers. They then developed a web similarity tool to measure the similarity of a website when visited by standard browsers versus None-browsers. This tool evaluates web pages by comparing various components, such as HTML structure, content, JavaScript, and CSS, as well as their visual rendering. They then crawled 270,000 web pages from 11,252 websites and found that websites no longer use UAs to serve content, as had been their original purpose. Instead, scripts serving ads and content delivery networks are mostly responsible for the 8.4% changes observed when JavaScript runs in the None-Browsers. They examined the impact of these changes and classified most as unintended and easily fixable, while the few intentional changes could be achieved by other means, such as feature detection. This discovery leads to an important conclusion: we could consider retiring UAs or using them differently to enhance web privacy.

Featured Image