Background. Understanding the relations between user perception and aesthetics is crucial for web design. But it is frequent in today’s graphic and media design that rules, established by practitioners even before the advent of Internet and still untested empirically, are taught at design schools and widely used for online interface design. So far, there is no well-established linkage between the in-class recommendations and our empirical knowledge on usability, for which design plays a role just as crucial as web projecting. Will webpages that are better from the designers’ viewpoint perform better in terms of usability? And can one have a list of recommendations tested empirically?

This is especially important for large-scale organizational web spaces where design plays a huge role in brand recognition and visual unity. Large web spaces need complex ergonomic assessment both on the level of selected nodes and on that of architecture/navigation. Of many large web spaces, university portals suit best for elaboration and pre-testing of such a methodology, as they serve various publics, contain sub-domains, and often face criticism for their user-unfriendly design and messy structure.

Objectives. We aim at creating a two-level usability expert test for a large web space that would be based on design recommendations tested empirically, thus eliminating the necessity of tech-based assessment of newcoming products. In this paper, we elaborate the node-level methodology. For this, basing on leading design literature, we create a page usability index (U-index) for ‘good’ design that provides quantitative measurement for traditional design decisions on the micro- and macro-level of a web page. Then, we test by eye tracking whether ‘better’ design (corresponding to higher U-index values) favors a particular pattern of content consumption – not ‘random search’ but more efficient ‘reading’.

Research design. To check whether web design measured qualitatively correlates with perception of web pages as tested by eye tracking, we first define target nodes by collecting the hyperlink structure and constructing web graphs for three web spaces of the biggest universities in the USA and Russia (Harvard University, Moscow State University, St. Petersburg State University). For this, we combine web crawling and web analytics. Second, we construct the U-index with the maximum value of 22. Third, to assess user perception of the target web pages, we create a series of tasks on information search and measure three test parameters (number of eye fixations, duration of fixations, and saccade length) and their derivatives, as well as heat maps. To avoid bias in quantitative measurement, we use two eye trackers (one head-fixed, one stationary) to test the results in parallel. Fourth, for finding correlations between U-index and eye-tracking results, descriptive statistics (Spearman’s rho and Cramer’s V) is used.

Results. First of all, our results suggest that various types of eye tracking hardware produce very different test results; this implies that eye tracking research always needs pre-testing. Second, we see that heat maps may be very suitable in express assessment of the web design quality, which speaks in favor of preserving some eye tracking tests in the final methodology. Third, we see substantial difference between Russian universities and Harvard: the latter, indeed, shows that features of web design correlate with eye tracking experience of the assessors, while for the Russian university websites, even after their repeated attempts of redesign, it remains unclear whether web design contributes to better user experience. For Harvard, the web pages with a higher usability index tend to facilitate ‘reading’ instead of ‘search’. Fourth, micro-level elements of the layout seem to contribute more to the general index and, thus, may deserve bigger attention of web designers.