The project “Scraping the Web for Evidence of Price Personalization” was conducted by Maria Garcia da Fonseca Veiga and Tomás Henrique Rijo Mendes, under the supervision of professors Qiwei Han and Fabrizio Esposito. It aimed to study the practice of price personalization – meaning, the act of using specific characteristics of the customer to advertise the most relevant products (price steering) and the most adequate prices in terms of consumer’s willingness to pay (price discrimination) – on websites.
For this project, eleven websites were analyzed, namely: Nike, Adidas, New Balance, Farfetch, Worten, Decathlon, Pingo Doce, Booking, Expedia, TripAdvisor and Airbnb. Moreover, the “specific characteristics” simulated were the device the consumer was on (phone or computer), the operating system they were using – meaning, the software the device, in question, ran (MacOS or Windows for computer; IOS or Android for phone) –, the type of browser used – the “main” website used to access to the specific websites analyzed (Safari, Google Chrome, Firefox or Microsoft Edge) –, and the location of the visiting user (Portugal, Spain, the Netherlands, and the United Kingdom region). These features represent the possible data that websites can gather on users interacting with them, in order to enhance and adapt what they display in terms of products and prices, with the objective of maximizing the possibility of selling something.
Such analysis was possible using the method of web scraping: meaning, extracting the data of the website that is shown to a user whenever they emit any sort of command, like clicking on a page or searching for a product. Such data was, however, extracted before it was translated into a visual form (meaning, the form a usual user sees), directly from the server (the “brain”) of the website. In fact, it is through this command that the website collects the data of the user, to accommodate its response to it. Therefore, for the purposes of this investigation, seven different “user profiles” were created, with different combinations of the features described, who then “accessed” the selected websites to study the response given to the same command (meaning, the same search).
In terms of price discrimination (conducted by Tomás), it was concluded that only three websites – Expedia, TripAdvisor, and Booking – showed different prices to different “user profiles”. However, although Booking was found to have the most significant number of products with different prices between users, it also showed the smallest amount of price variation within said different prices. The custom pricing algorithm of Expedia, on the other hand, was found to be more aggressive, having a higher price variation between users for a lower difference in the number of different prices shown. Furthermore, users using computers received higher prices, especially those using the combination of Windows-Google Chrome. However, users using iPhones (and, therefore, the combination of IOS-Safari) saw higher prices than users with the combination Android-Google Chrome. The product being searched was also found, in some degree, to have an impact on the price being shown to each user – with TripAdvisor and Expedia showing different prices for hotels in Rome and Istanbul, respectively. Finally, the varying location of users seemed not to have an impact on the price of products, with differences being close to none.
In terms of price steering (conducted by Maria), the first analysis showed that Expedia, TripAdvisor, Airbnb, Booking and Farfetch showed differences in the type of products shown for the same search – with Farfetch showing different products to phone users as opposed to computer users. Nike also revealed differences in products shown, but only to users from the Netherlands. The second analysis, however, concluded that, in terms of the ranking of the products shown within each website, the difference was close to zero for all users, with the features used not influencing the order the website choose to present them in any significant way.
Although these conclusions were a first step in proving such practice, much deeper research, with a larger data range collected regarding more variable features, is needed – an objective that is currently being developed in the project “Empowering Personalized Prices: Informing Consumers to Support a Fair and Sustainable Digital Transition”.