Page tree
Skip to end of metadata
Go to start of metadata

Task performance measurement is a way to benchmark how easily and how quickly our audiences can complete what they came to do (their tasks) on the Commission's websites. The tests result in an indicator known as the task performance indicator (TPI).

Using task performance measurement as a continuous-improvement management model offers:

  • easily-understood indicators of success, failure and time on task completion.
  • insight into both the current situation and how to improve it. For example, where testing shows that a particular task has a success rate of 60%, the data will also show what caused the 40% of failures. Addressing and fixing the causes of failure should result in a clear improvement in the success rate when the task is tested again.
  • evidence-based action plan. Between iterations, and based on the findings of the tests, an action plan can be made to improve content, usability and navigation, focusing on the most urgent issues first.

Methodology

Task performance measurement involves recording and analysing 15-20 one-on-one remote sessions (about 60 minutes) in which representative users try to complete a prioritised set of tasks.

Preparation: task instructions for testing

TPI testing involves asking participants to complete specific tasks that have been identified as most important. The instructions are real task scenarios and framing them correctly is essential to the success of the measurement process. The following detailed criteria are used to develop instructions: 

  1. Repeatable
  2. Representative and typical for the task
  3. Universal—everyone can do it
  4. One unique answer
  5. Does not contain hidden clues
  6. Not search-engine optimised
  7. Emotionally neutral
  8. Not confidential
  9. Clearly different from other instructions
  10. Independent of other instructions (i.e. not linked to another instruction in the test)
  11. Can be performed directly on website (does't require prior log-in)
  12. Short – 30 words or less
  13. Stable – no change within the testing period

Examples:

Task referenceTest instruction

Top task 1 (EU law, rules, treaties, judgments)

Find the current EU law governing investigations by the European Anti-Fraud Office.
Top task 2 (Research and innovation)Find the page listing potential research and innovation project partners in the field of robotics in order to share knowledge.
Top task 3 (Funding, grants, subsidies)

Find the opening date for the current opportunity for small businesses (SMEs) to submit their proposals in the field of healthcare biotechnology in Horizon 2020.

Top task 5 (EU strategy, political priorities)How many infrastructure and innovation projects were approved for Estonia as of [choose recent date] under the European Fund for Strategic Investments?

In the Commission, task instructions are developed with the relevant DGs. Once a task instruction is agreed, each step in the navigation path that leads to the correct answer is documented in the form of URLs and screen shots.

Although the instructions from the baseline should be used for subsequent iterations, they are nevertheless reviewed with stakeholders before each iteration.


Remote tests

Remote testing is both representative and real because participants are in their own environment and on their own computers. Screen interactions and speech are recorded (with the participant's consent). The time it takes for the test participant to complete each task is measured from the moment they understand it and start working on it until they come up with a definitive answer or give up. 

Each participant starts the test from the same starting page. To ensure that results remain comparable over successive iterations, the starting page needs to be repeatable. In the Commission, therefore, the agreed starting page is the Commission homepage (ec.europa.eu/index_en.htm). Test participants are explicitly told that, although the test starts from this page, they are free to use the internet as they would usually do (Google, Bing, etc.).

Analysis and reporting

Analysis involves identifying:

  • the main causes of success and failure
  • time for task completion and where time is lost
  • recurring patterns

Reporting involves compiling actionable findings on navigation, search, design, content for the class and per instruction, including annotated screen shots.

Video recordings are available per participant and per task instruction.

TPI test reports
Baseline reports are available for the following content classes:







  • No labels

The Europa Web Guide is the official rulebook for the European Commission's web presence, covering editorial, legal, technical, visual and contractual aspects.
All European Commission web sites must observe the rules and guidelines it contains.
Web practitioners are invited to observe its contents and keep abreast of updates. Read more.