information Society Technology

Introduction

The Unified Web Evaluation Methodology should ensure that evaluation tools and methods developed for large scale monitoring or for local evaluation, are compatible and coherent among themselves and with W3C/WAI. This document is the result of a joint effort of three European projects with 23 organisations collaborating in the WAB Cluster to develop UWEM.

The UWEM describes a methodology for evaluation of conformance with the W3C Web Content Accessibility Guidelines 1.0 . For practical reasons, this version of the methodology focuses on the current WCAG 1.0 guidelines. These WCAG 1.0 guidelines are broadly accepted and form a stable factor in accessibility since May 1999. Already in 2002, the EU recommended that they should be adopted by the public sector in Member States. In some countries they found their way into legislation. In the later version, UWEM will be synchronized with the foreseen migration from WCAG 1.0 to WCAG 2.0.

The purpose of the UWEM is to provide a basis for evaluating the methodology with regard to the intended types of testing: expert and automatic evaluation of Web resources 1. The evaluation of the UWEM is also planned to provide feedback and contribute to W3C/WAI for future guidelines or versions of guidelines. W3C/WAI staff have reviewed and provided input into previous drafts of this document in order to minimize potential fragmentation of technical content. This does not imply W3C or WAI endorsement of any part of this document in any way.

Part of the materials presented in this document in Tests for conformance evaluation are annotations of W3C documents. In particular, we are targeting the following two documents:

According to the Intellectual Rights FAQ from W3C 2, Tests for conformance evaluation of UWEM falls under an annotation ... that does not require the copying and modification of the document being annotated.3 Therefore, all references to guidelines and checkpoints are duly quoted, and the URL to the original document is included. W3C is not responsible for any content not found at the original URL, and the annotations in this document are non-normative.

Methodology definition

The Unified Web Evaluation Methodology provides an evaluation procedure offering a system of principles and practices for expert and automatic evaluation of Web accessibility for humans and machine interfaces. The Methodology is designed to be conformant with WCAG 1.0 priority 1 and 2 checkpoints with regard to technical criteria.

The UWEM aims to increase the value of evaluations by basing them on a shared interpretation of WCAG 1.0 and a set of tests that are sufficiently robust to give stakeholders confidence in results. Web content producers may also wish to evaluate their own content and UWEM aims to also be suitable to these users.

The methodology is designed to meet the following requirements:

In the methodology we have included information about:

Target audience of the document

The target audiences for this document include, but are not limited to:

The European Commission, national governments and other organisations who wish to benchmark Web accessibility will be able to use the UWEM to carry out the evaluations and compare their results in a meaningful way.

The UWEM is an evaluation methodology and is not intended to provide information for Web content producers wishing to produce content comformant with WCAG 1.0. This information is provided in the WCAG 1.0 Techniques Documents that are available through the W3C/WAI web site.

Target technologies of this document

The UWEM primarily covers methods to evaluate documents based on the following technologies:

Expertise for evaluating Accessibility

The W3C Evaluation suite 4 extensively describes the expertise necessary for the evaluation of the accessibility of Web content for people with disabilities. Evaluation activities require diverse kinds of expertise and perspectives. Individuals evaluating the accessibility of web content require training and experience across a broad range of disciplines. A collaborative approach can bring better results for individuals evaluating web content.

The W3C/WAI evaluation suite writes: 5

Effective evaluation of Web accessibility requires more than simply running an evaluation tool on a Web site. Comprehensive and effective evaluations require evaluators with an understanding of Web technologies, evaluation tools, barriers that people with disabilities experience, assistive technologies and approaches that people with disabilities use, and accessibility guidelines and techniques.

This document describes the expert and automatic evaluation methodology. automatic evaluation can significantly reduce the time and effort needed for an evaluation but please note that many accessibility checks require human judgement and must be evaluated manually. This is described in more detail in Tests for conformance evaluation.

More information on using tools can be found in the W3C/WAI evaluation suite section “selecting tools” 6.

Acknowledgements

The following organizations worked on this UWEM document:

We thank the Web Accessibility Initiative's Team from the World Wide Web Consortium for all the useful comments to the draft versions of this document.

More information about the WAB Cluster

The projects participating in the WAB cluster are funded by the European Union in the second FP6 IST call (2003) of the eInclusion Strategic Objective. The WAB cluster Web site is available at http://www.wabcluster.org/. More information about the projects can be found on the project web sites:

  1. Evaluation of Web resources via user testing will be covered by future UWEM releases.
  2. http://www.w3.org/Consortium/Legal/IPR-FAQ-20000620
  3. http://www.w3.org/Consortium/Legal/IPR-FAQ-20000620#annotate
  4. http://www.w3.org/WAI/eval/
  5. http://www.w3.org/WAI/eval/reviewteams.html
  6. http://www.w3.org/WAI/eval/selectingtools.html
www.wabcluster.org