Time on Task in Usability Testing: Timing Rules, Calculations, and Common Traps

A short attempt is not always a successful one

A participant who gives up after 15 seconds can be faster than someone who finishes correctly in 50 seconds. If a dashboard calls both durations 'completion time,' the faster result can reward a worse experience. Time on task needs an outcome and a defined clock before it becomes interpretable.

MeasuringU's task-metrics guide separates time for successful completions, time spent by all participants, and time until failure. Its research on summary statistics also shows why a few long attempts can pull the arithmetic mean away from the middle of a distribution. Those distinctions are more useful than treating one average as the answer to every efficiency question.

The worked example here follows a fictional expense-report export task. All times and observations are invented for teaching. You will see the timing protocol, every attempt, three different summaries of successful times, the treatment of abandonment and timeout, and a comparison where a lower median is misleading.

Specify when the clock starts and what stops it

The participant's task is to create a CSV containing only approved expenses from September. The test account is reset to the report landing page before each attempt. Independent success means the resulting file contains exactly the expected records, without moderator guidance. Agree this outcome rule before measuring speed.

For this example, the clock starts when the participant finishes reading the task, says they are ready, and the researcher releases the starting page. It stops when the correct file is available, when the participant gives up or declares a wrong result complete, or when the four-minute session cap is reached. Record which stopping event occurred. The four-minute cap is an operational choice for this invented study, not a universal usability threshold.

Record real elapsed time from the same event source or reviewed recording for every attempt. Do not start at the first click for some participants and at task presentation for others. If task reading is part of the question you want to measure, include it consistently under a different declared protocol.

The fictional protocol includes page loading, reading interface help, and correcting mistakes because these occur during the attempted workflow. A moderator's later discussion is outside the timed task. If an unrelated interruption occurs, mark its start and end and retain both the original duration and any adjusted duration under a predeclared rule; do not silently remove time because it looks large.

Inspect the individual records before calculating an average

Eight participants start the task. Six finish independently. P07 gives up at 15 seconds, and P08 is still trying when the 240-second cap is reached. P06 takes 180 seconds but produces the correct file after reading the help and correcting the status filter. That long valid attempt stays in the data.

The 240-second record says that this attempt did not achieve success before the cap. It is not a completed-task duration of 240 seconds, and it does not establish when the person would have finished if allowed to continue. A study designed to analyze time to completion with censored observations needs an appropriate method; assigning the cap as a successful time is not one.

Look at the recording before deciding that a long duration is an error. A logging failure, accidental idle session, genuine confusion, and careful reading have different meanings. If a value is excluded because the measurement itself is invalid, document the evidence and report the exclusion. A slow but valid task cannot be deleted merely to improve the average.

Calculate the successful-task summaries explicitly

The successful durations are 30, 40, 45, 50, 55, and 180 seconds. Their total is 400 seconds. The arithmetic mean is 400/6 = 66.7 seconds. The median is the average of the third and fourth sorted values: (45 + 50)/2 = 47.5 seconds. These are different summaries of the same six observations, not conflicting measurements.

The geometric mean is exp((ln 30 + ln 40 + ln 45 + ln 50 + ln 55 + ln 180)/6), approximately 54.7 seconds. It summarizes multiplicative variation and requires positive values. A zero duration in a task like this should trigger a check of the timer and start/stop definition; do not add an arbitrary constant just to make a log calculation run.

MeasuringU's 2010 simulation study compared estimates of the center of positively skewed task-time distributions and favored the geometric mean in its small-sample setting. That does not make the arithmetic mean mathematically wrong or establish a universal rule for every task. If your question concerns total elapsed burden across completed attempts, the arithmetic mean and total may be directly relevant. If you describe the middle of this observed sample, label the median as a sample median.

For this small teaching example, report the individual values, count, and both the median and arithmetic mean, with the geometric mean as a reproducible alternative. Avoid declaring 47.5 seconds to be a precise population benchmark. A formal estimate or comparison requires an uncertainty analysis suitable for the sample, distribution, and study design.

Keep failed and capped attempts visible beside the success times

If you average all eight observed elapsed durations, the total is 655 seconds and the mean is 655/8 = 81.875, or 81.9 seconds. This answers a question about time spent in the observed attempts, including giving up and the cap. It is not the mean successful completion time of 66.7 seconds.

The two nonsuccess records should remain distinguishable. P07 may have quickly found no plausible route. P08 may have invested four minutes in repeated attempts. Combining them into a single failure average of 127.5 seconds would be arithmetically possible, but it would hide the difference between abandonment and a researcher-imposed stopping point. With only two cases, report the individual observations.

Never replace failed attempts with zero when calculating completion time. That would make the experience look faster as more people fail. Likewise, do not report only the six successful times without the 6/8 completion count; readers need to know which people the time summary excludes.

A useful chart or table separates successful, abandoned, and capped attempts and keeps the same time unit. If the study has many tasks, calculate each task's summary before considering an overall measure. Averaging a ten-second lookup with a ten-minute report task produces a number whose practical meaning may be unclear.

Do not call a lower median an improvement when completion falls

Imagine a second fictional design tested with eight different participants. Only three complete, with times of 20, 25, and 30 seconds; five do not complete. The successful-task median drops from 47.5 to 25 seconds, while independent completion drops from 6/8 to 3/8. Reporting only the lower median would conceal the larger number of people who could not finish.

This comparison does not establish that the second design caused either change. The people may differ, the sample is small, and the example is invented. It demonstrates a selection problem: successful-task times describe the people who succeeded in each condition, and those subsets may not be comparable.

For an actual comparison, hold the task, success criterion, timing rule, product data, assistance policy, and relevant device conditions consistent. If the same people try both designs, learning and order matter; plan the sequence and analyze the paired observations appropriately. If different people try them, recruitment and assignment matter. Decide the analysis before choosing the more favorable summary.

Read speed together with correctness and the user's goal. Some tasks legitimately take longer when a clearer design prompts people to check important information. A faster export that contains the wrong records is not a better outcome. The decision should follow the task's purpose, not the direction of a single metric.

Write a report someone else can reconstruct

For the first invented design, a concise report is: 'Eight participants attempted the same approved-expense export. Six completed independently. Their observed times were 30, 40, 45, 50, 55, and 180 seconds; median 47.5 seconds and arithmetic mean 66.7 seconds. One participant gave up at 15 seconds and one reached the 240-second cap without success. Timing began after task reading when the starting page was released. Loading, interface reading, and correction time were included; post-task discussion was excluded. These descriptive results identify workflow issues to investigate and are not a precise estimate for all users.'

Attach the event definitions and evidence locations to the raw log. A reviewer should be able to reproduce the duration from the recording or event stream and understand why each attempt entered a summary. Keep a note of any assistance or invalid measurement rather than forcing every record into the same column.

The next design action in this example is to inspect what made P06 read help and change the status filter, what led P07 to stop, and where P08 spent time without reaching the required output. Those observations turn the metric into a design question. A lower average becomes meaningful only when the resulting workflow still helps the intended people complete the right task.

• Define the exact starting event and successful end state.
• Preserve each attempt’s duration, outcome, assistance, and stopping reason.
• Label the statistic and the subset it summarizes.
• Report completion alongside time, including abandoned and capped attempts.
• Keep comparison conditions and analysis choices consistent before claiming improvement.

References

• MeasuringU: A guide to task-based UX metrics
• MeasuringU: Average task times — what to report