Web Accessibility Benchmarking Cluster

WAB Cluster

Sixth Framework Programme

Information Society Technologies Priority

Unified Web Evaluation Methodology (UWEM 0.5)

Contractual Date of Delivery to the EC: 28 February 2005 + 45 days
Actual Date of Delivery to the EC: 12 October 2005
Editors:
  • Eric Velleman (Accessibility Foundation),
  • Carlos A Velasco (Fraunhofer Institute for Applied Information Technology FIT),
  • Mikael Snaprud (Agder University College),
  • Dominique Burger (BrailleNet)
Contributors: See Appendix E for contributors list
Workpackage: WAB1a
Security: Public
Nature: Report
Version: P
Total number of pages:

Keywords:

Web Accessibility, WAB Cluster, World Wide Web Consortium, W3C, Web Accessibility Initiative, WAI, Web Content Accessibility Guidelines, WCAG, Unified Web site Evaluation Methodology, UWEM, Evaluation and Report Language, EARL.

DOCUMENT HISTORY
Version Version date Responsible Description
A 05-05-22 FIT Initial version with TOC
B 05-03-06 FIT Version containing Section 3, and integrating contributions to Sections 4, 5 and 7, plus Appendices A, C, D and E
C 05-06-05 Agder University College Added section 2.6 and 2.8
D 05-06-05 DCU Revised section 4; added section 12.2.
E 10-06-05 Accessibility Foundation Results of Conclave work: 6, 7 and 8 June 2005
F 05-06-20 City University Updated version of section 7; appendices for section 7.
G 05-06-21 Eric Velleman Updated sections with comments from W3C.
H 05-06-25 Agder University College Converted to master document, for easier maintenance and removed material from the evaluation suite in section 3.
I 05-06-25 Agder University College Added latest version of section 4 that Barry had uploaded,and the glossary.
J 05-06-25 Agder University College Added information on Xiaoming Zeng's WABScore method, and elaborated on aggregating page fractions, instead of whole pages for key use scenarios and a connection to  sampling/ random walk algorithms to section 6.
K 05-08-09 Agder University College Section 6 split into core part and complementary section.
L 05-08-20 Agder University College Section 6 restructured for better readability, and updated with comments from Andreas Prinz. Minor typos fixed in section 4 and 8.
M 05-08-22 Agder University College Fixed link to references and glossary.
N 05-09-30 Accessibility Foundation, FIT Global copy-editing; replacement of missing figures; full modification of Section 5 according to BenToWeb’s proposal; missing references and cross-references fixed.
O 05-10-12 FIT Incorporation of W3C copyright notice and final remarks from WAI and Cluster projects.
From Agder University College: Incorporation of review comments from the EIAO partners (05-10-11)
P 05-10-12 Accessibility Foundation Change to section 2 following remark from WAI

Table of Contents

Skip table of contents to content

1 Executive summary

This document is the result of a joint effort by 24 European organisations in three European projects combined in a cluster to develop a Unified Web Evaluation Methodology (UWEM). This methodology is based on the W3C Web Content Accessibility Guidelines 1.0 [WCAG10] and will be synchronised with the foreseen migration from WCAG 1.0 to WCAG 2.0 [WCAG20] in the near future. The UWEM offers an interpretation of the guidelines agreed among stakeholders within the aforementioned projects. This document is a draft and shall not be considered as an stable version. This document will serve as the basis for a evaluation of the draft methodology by all intended users.

The projects involved in making this methodology aim to ensure that evaluation tools and methods developed for global monitoring or for local evaluation, are compatible and coherent among themselves and with WAI. The projects will provide feedback and contribution to WAI for future guidelines or versions of guidelines. Their aim is to increase the value of evaluations by basing them on a shared interpretation of WCAG 1.0.

This document presents the 0.5 version of the Unified Web Evaluation Methodology, which is based on the W3C Web Content Accessibility Guidelines 1.0. UWEM 0.5 will be extensively evaluated in the next phase of the Cluster including user-, expert and (semi-) automated testing of the assessment methodology. This phase will also include a phase for public comments, evaluation and discussion.

UWEM 0.5 provides an evaluation procedure consisting of a system of principles and practices for manual and automatic evaluation of Web accessibility for humans and machine interfaces. The  methodology aims to be fully conformant with WCAG 1.0 guidelines. Currently the UWEM is limited to priority 1 guidelines, but in the coming phase of the Cluster, priority 2 guidelines will be added.

The methodology covers evaluations of one Web page, an entire site (irrespective of size), or multiple sites, a method for sampling, clarifications of the checkpoints, user testing protocols and information necessary for interpretation and integration/aggregation of results.

The UWEM 0.5 will be used by the projects included in the Cluster for an observatory (EIAO project), tools for benchmarking (BenToWeb project) and a certification scheme (SEAM project). More information about the Cluster, the UWEM and the projects involved can be found at: http://www.wabcluster.org/. Refer to Appendix A for the document license.

The document is organised in the following way. Section 2 outlines the requirements for the document and the basic properties of UWEM. Section 3 describes the UWEM conformance related to a sample to be evaluated, types of tests to conduct, conformance claims and confidence levels for the test results. Section 4 describes the approaches for scoping and sampling of a Web site for evaluation. The sampled resource set should cover critical tasks and secure a representative evaluation result. Section 5 describes how to carry out the tests related to the WCAG 1.0 checkpoints, including clarifications and user testing information. Section 6 describes a model for aggregating test results from accessibility barriers on a Web resource to regional aggregations of Web resources. The model also refers to a possible aggregation over different user groups who may expereince different barriers. Section 7 describes user testing protocols according to UWEM. The section also describes how to select a set of tasks to accomplish and how to compose a group of testers. Section 8 gives a description on how to present the results of the evaluation, specially taylored for policy makers using a score card approach. There are as well a set of relevant appendices.

2 Introduction

The Unified Web Evaluation Methodology should support that evaluation tools and methods developed for large scale monitoring or for local evaluation, are compatible and coherent among themselves and with W3C/WAI. This document is the result of a joint effort of three European projects with 24 organisations collaborating in the WAB Cluster to develop UWEM.

The purpose of the 0.5 version of UWEM is to provide a basis for evaluating the methodology from all the intended types of testing, including: user-, expert- and (semi-) automated testing of Web resources. The evaluation of UWEM is also planned to provide feedback and contribute to W3C/WAI for future guidelines or versions of guidelines. W3C/WAI staff have reviewed and provided input into previous drafts of this document in order to minimize potential fragmentation of technical content. This does not imply W3C or WAI endorsement of any part of this draft. W3C/WAI Working Groups have not yet reviewed the draft, and will have the opportunity to do so along with the public.

Part of the materials presented in this document are annotations of W3C documents (those included in section 5). In particular, we are targeting the following two documents:

According to the Intellectual Rights FAQ from W3C, section 5 of UWEM falls under an annotation “... that does not require the copying and modification of the document being annotated”. Therefore, all references to guidelines and checkpoints are duly quoted, and the URL to the original document is included. W3C is not responsible for any content not found at the original URL, and our annotations are non-normative.

2.1 Methodology definition

The UWEM is a Web evaluation methodology that provides an evaluation procedure consisting of a system of principles and practices for manual and automatic evaluation of Web accessibility for humans and machine interfaces. Version 0.5 of the methodology is designed to be conformant with WCAG 1.0 priority 1 checkpoints with regard to technical criteria. Future versions of this methodology are intended to be conformant with WCAG 2.0 from the same standpoint.

UWEM 0.5 offers an interpretation of the guidelines to be agreed among European stakeholders. It aims to increase the value of evaluations by basing them on a shared interpretation of WCAG 1.0 and a set of tests that are sufficiently robust to give stakeholders confidence in results. Web content producers may also wish to evaluate their own content and UWEM aims to also be suitable to these users.

The methodology is designed to meet the following requirements:

In the methodology we have included information about:

2.2 Target audience of the document

The target audience for this document include for example:

The European Commission, national governments and other organisations who wish to carry out benchmarking projects on Web accessibility will be able to use the UWEM to carry out the evaluations and compare their results in a meaningful way.

UWEM is an evaluation methodology and is not intended to provide information for Web content producers wishing to produce content compliant with WCAG 1.0. This information is provided in the WCAG 1.0 Techniques Documents that are available through the W3C/WAI website [WCAG10-TECHS].

2.3 Target technologies of this document

The 0.5 version of UWEM covers methods to evaluate documents based on the following technologies:

2.4 Acknowledgements

The following organisations worked on this UWEM document:

Accessibility Foundation (The Netherlands, Cluster coordinator); Agder University College (Norway, EIAO coordinator); Fraunhofer Institute for Applied Information Technology FIT (Germany, BenToWeb coordinator); Association Braillenet (France, SupportEAM coordinator), Vista Utredning AS (Norway); Forschungsintitut Technologie-Behindertenhilfe der Evangelischen Stiftung Volmarstein (Germany); The Manchester Metropolitan University (UK); Nettkroken as (Norway); University of Tromsø (Norway); FBL s.r.l. (Italy); Warsaw University of Technology, Faculty of Production Engineering, Institute of Production Systems Organisation (Poland); Aalborg University (Denmark); Intermedium as (Norway); Fundosa Teleservicios (Spain); Dublin City University (Ireland); Universität Linz, Institut integriert studieren (Austria); Katholieke Universiteit Leuven, Research & Development (Belgium); Accessinmind Limited (UK); Multimedia Campus Kiel (Germany); Department of Product and Systems Design, University of the Aegean (Greece); City University London (UK); ISdAC International Association (Belgium); FernUniversität in Hagen (Germany).

We thank the Web Accessibility Initiative's Team from the World Wide Web Consortium for all the useful criticisms to the different versions of this document.

2.5 More information about the WAB Cluster

The projects participating in the WAB cluster are funded by the European Union in the second FP6 IST call (2003) of the eInclusion Strategic Objective. The WAB cluster Web site is available at http://www.wabcluster.org/. More information about the projects can be found on the project websites:

3 Evaluation procedures and conformance

The evaluation procedures of UWEM 0.5 have the following objectives:

3.1 Large scale screening of Web sites

Large scale screening of Web sites, is an UWEM specific method that may help to quickly identify the scope of problems, or monitoring progress for a large number of Web sites or individual Web sites for some disability groups. However, automatic screening will not catch all of the problems on a site and should not be used to determine conformance.

3.2 Types of evaluation

The different types of evaluation methods have a number of strengths and weaknesses, as well as different levels of confidence associated with them. describes the levels of confidence of four different evaluation methods in their ability to benchmark accessibility.

Figure 1: UWEM Confidence Levels and Evaluation Types.

The figure shows for example, that automatic evaluation (Tool1 or Tool2) can only test for conformance to a subset of the  checkpoints (such as  the set provided in section 5), which further means that a subset of all possible accessibility barriers can be identified reliably by using automatic testing. This means that confidence in automatic evaluation as an overall indicator of accessibility is low, however it can identify some barriers reliably. Tool 1 and Tool 2 are here two fully automatic assessment tools that focus on checking slightly different accessibility issues, with some overlap of functionality.

Some tools can also act as decision support systems in a semi-automatic evaluation process where the system aids the testing process and points out  where the testers should focus and possibly give hints about earlier decisions in addition to  performing some tasks automatically. Expert testing, usually done with a semi-automatic testing/decision support system, will typically be more precise than semi-automatic testing done by a non-expert.

User testing is able to identify some barriers that are not caught by other testing means, and is able to identify barriers and also estimate the accessibility for the tested scenarios with a high level of confidence. However, user testing is also quite specialised, so that it is not generally suitable for conformance testing, since it is not able to test all aspects of the tests of section 5. The best approach to ensure both accessibility and UWEM conformance is to use a combined approach with both expert testing and user testing of the Web site.

The main advantages of automatic testing are:

The highest level of confidence is achieved by a combined approach of user testing and expert testing involving all possible tasks supported by a Web site. This method also provides feedback to the developers for the improvement of the accessibility and usability of the site. However, it is an expensive and time-consuming method, and for the purposes of UWEM 0.5, we propose user testing on a subset of tasks (see sections 4 and 7). See section 8 for reporting of results as scorecards.

3.3 Conformance requirements

UWEM 0.5 conformance requires that all the tests to the selected level in section 5 pass for every test on every page of the Web site according to the scope defined in section 4.

3.4 Conformance claims

The claims of conformance of accessibility according to the UWEM  methodology must use the following form:

  1. The UWEM version and its URI identifier, e.g., http://www.wabcluster.org/refs/UWEM-0.5/

  2. The URI to a document detailing the evaluation set (scope pattern list) to which the claim refers. A resource set conforms at a given confidence level only if all content provided by that resource so conforms.

  3. The level of confidence being claimed.

3.5 Levels of confidence

Confidence in the UWEM evaluation results is expressed not only by the choice of a method, but also by providing information about the sampling of resources from the Web site (see section 4) and the set of criteria that have been applied (see section 5). This confidence is at a macro-level and is not the same term as used for the tests of section 5.

Section 5 states the technologies to which UWEM can be applied (W3C technologies (X)HTML and CSS in different versions). There are sites therefore to which UWEM cannot technically be applied, like, e.g., a site making exclusive use of Macromedia Flash. For these sites, user testing might be necessary, although it cannot provide a complete view on the accessibility of the site.

The following levels of confidence in the different individual methods for assessing accessibility are proposed (see ). Note that every level includes the precedent:

Minimal Confidence (“Automatic evaluation”):
this method implies that all tests that can be automatically checked, as described in section 5 have been used for a set of resources, as defined in section 4. The resources tested must pass all tests. The tool integrates the results as described in section 8. This confidence level indicates that a tool (or a person) can check all automatised tests of section 5.
Medium Confidence (“Semi-Automatic evaluation”):
this method implies a human evaluator takes the results of all tests (see section 5) that can be automatically assessed with different tools, and integrates these results as described in section 8, applied on resources as defined in section 4. This level is a minimal extension of the previous level where two or more tool results are used, and the results are aggregated and compared by a human.
High Confidence (“Evaluation by experts”):
this method implies experts apply all tests (see section 5) on resources as defined in section 4, and integrate the evaluation results as described in section 8. The resources tested must pass all tests.
Highest Confidence for a task (“Evaluation by users”):
this method implies users evaluate the resources as defined in section 4, using the methodology as defined in section 7, and integrate the evaluation results as described in section 8. However, to have the highest confidence in user testing, users would need to undertake all the key tasks on a Web site, which is undoubtedly a very time-consuming procedure. The highest confidence level must include “Evaluation by experts” as well.

In deciding which method or combination of methods to choose for assessing accessibility, a Web site owner should bear in mind the above levels of confidence and also use the individual Web site accessibility scorecard presented in section 8.

4 Scope of a Web site and methods for sampling

4.1 Procedure to express the scope of a Web site: the Scope Pattern List

For the purposes of the UWEM a Web site is defined as an arbitrary collection of hyperlinked Web resources, each identified by one or more URIs [RFC2396] (each URI optionally complemented with a set of additional parameters, as defined by the XML Schema described in Appendix C). The scope of a Web site – the specific set of resources considered as belonging to it — can therefore be identified or expressed by giving a clear, repeatable and unambiguous procedure for deciding whether any arbitrary URI does or does not belong to the site.

According to the needs of different applications of UWEM, scope may be specified by a variety of different participants in the evaluation process – such as a site owner, a site operator, an inspection organisation, etc. This document does not offer advice on deciding upon the appropriate scope for any particular UWEM application: it only explains how such a scope should be unambiguously expressed.

The scope of a Web site should be expressed in the form of an ordered list of resource patterns, termed a Scope Pattern List. Each pattern is designated as either an include pattern or an exclude pattern. Each pattern is expressed using an XML Schema regular expression [XMLSCHEMA2]. The complete Scope Pattern List should be expressed using the XML Schema described in Appendix C. The scope status of any arbitrary URI is then determined by testing it against each pattern in turn until a match is found. If the match is to an include pattern, then the identified resource is inside the scope of the site. If the match is to an exclude pattern, or if no match is found, then the identified resource is outside the scope of the site.

4.2 Procedure to catalogue the Complete Resource Set of a Web site

Section 4.1 explains how the scope of a Web site should be identified, i.e., how to express a rule for deciding whether a given resource is inside or outside the scope of a site. However, this does not, in itself, identify any particular resources which indeed lie inside the scope of the Web site. Such an explicit set of particular resources is a pre-requisite for any evaluation of a Web site. The complete or exhaustive set of resources belonging to a site is called the “Complete Resource Set”.

The Complete Resource Set is normally made explicit, or catalogued, by identifying a set of one or more “seed” resources (the “Seed Resource Set”) and recursively “crawling” from there, accepting all linked resources which qualify as in scope per section 4.1. That is, starting with the Seed Resource Set, each resource is retrieved and analysed to identify any links (URIs) to other resources. Each of these URIs is assessed to see whether it is in scope. If so, the corresponding resource is also retrieved and the process is repeated. In principle this procedure should be followed exhaustively until no additional in-scope URIs can be identified.

Automated crawling is, in general, subject to limitations, for example with regard to resources accessed via form submission (typical of sites offering interactive services). This may be addressed, in certain cases, by providing additional parameters to support crawling, such as form input data, HTTP accept headers, etc., as described in Appendix C. However, in all cases, it will be important to ensure that any resources which might otherwise not be located by automated crawling, should be included as explicit elements of the Seed Resource Set. That is, it is a requirement on the choice of the Seed Resource Set that it be sufficiently extensive that recursive crawling from this set will generate the intended Complete Resource Set.

In general, the outcome of cataloguing the Complete Resource Set will depend not only on the Scope Pattern List and the Seed Resource Set, but also upon the specific, dynamic, state and configuration of the Web site when the crawling is carried out. Accordingly, any catalogue of the Complete Resource Set should be explicitly linked to the relevant Scope Pattern List and Seed Resource Set, and also be timestamped to show when it was created.

4.3 Procedure to generate Evaluation Samples

In general it will not be practical to test all site resources (all elements of the Complete Resource Set) against all evaluation criteria. Accordingly, we identify here certain subsets or “samples”, of the Complete Resource Set.

4.3.1 The Core Resource Set

The Core Resource Set is a set of generic resources, or resource types, which are likely to be present in most Web sites, and which are core to the use and accessibility evaluation of a site. The Core Resource Set therefore represents a minimal set of resources which should be included in any accessibility evaluation of the site. The Core Resource Set cannot, in general, be automatically identified, but requires human judgement to select. The Core Resource Set should consist of as many of the following resources as are applicable and which lie within the defined scope of the site, per section 4.1:

Note that, of course, there may be some overlap in the resources identified under the different points above: any given resource should appear only once in the Core Resource Set.

4.3.2 Sampled Resource Set(s)

A Sampled Resource Set is a resource set which is generated in a similar manner to the Complete Resource Set – by automated recursive crawling from a set of “seed” resources (either the standard Seed Resource Set, or some subset of it) – but where the crawling has been subject to certain pre-determined limits or constraints. A Sampled Resource Set would typically be used in the context of evaluations carried out over large numbers of sites (against automatic criteria only), where it is not feasible or necessary to evaluate the Complete Resource Set for each site. The basis for delimiting a Sampled Resource Set will be application specific, and should be explicitly disclosed in any evaluation report, but may, in general, include the following possible mechanisms, individually or in combinations:

4.3.3 Relationship between Resource Sets

The following figure illustrates a typical relationship between each of the Resource Sets discussed above. It shows that a Web site (the Complete Resource Set) is some distinguished subset of the entire Web (all existing Web resources). Sampled Resource Sets are more or less arbitrary subsets of the Complete Resource Set; so they are arbitrary subsets of the resources constituting a site. The Seed Resource Set is then a smaller subset, used to seed the crawling or cataloguing of the Complete Resource Set (and also, at least in part, any Sampled Resource Sets). Finally, the Core Resource Set is the smallest of all, being a subset of the Seed Resource Set, containing just that minimal set of resources which should be included in all evaluations.

Figure 2: Relationship between Resource Sets. See text for details.

5 Evaluation guidelines and checklists

5.1 Introduction

This section covers the UWEM 0.5 testing of Priority 1 checkpoints of WCAG 1.0 [WCAG10]. Further versions of UWEM might cover additional checkpoints. This interpretation refers in particular to (X)HTML and CSS tests that MUST be carried out to be compliant with this version of UWEM.

UWEM does not impose a way to carry out the tests, and some of the following tests could be made with the help of automatic or semi-automatic tools. This section is intended to be replaced in the future by the success criteria of WCAG 2.0 [WCAG20] and the accompanying techniques documents once it becomes a W3C Recommendation.

This section contains quotes of W3C recommendations and notes (in particular, [WCAG10] and [WCAG10-TECHS]). The quoted text is in each case immediately followed by a URI linking to the quoted text in its original context in the W3C/WAI documents. These materials are copyright of W3C and its use is subject to the conditions of the W3C Document License (see Appendix F or http://www.w3.org/Consortium/Legal/2002/copyright-documents-20021231 for further information of the use of these documents).

The structure of the tests is the following:

  1. Guideline 

    Quotation of the corresponding WCAG 1.0 guideline. Pointers to additional clarifications might be added.

    • Checkpoint 

      Quotation of the corresponding WCAG 1.0 checkpoint. Pointers to additional clarifications might be added.

      1. Summary overview 

        Tabular overview of the tests corresponding to the checkpoint.

      2. (X)HTML-specific tests 

        Set of tests to be made for conformance claims for (X)HTML resources. Each test consists of:

        • Title and ID: short descriptive title (informative) and unique identifier (normative).

        • Applicability criteria: elements, attributes and combinations thereof used to determine the applicability of the test. Whenever possible, the criteria will be presented as XPath expressions, otherwise a prose description will be given.

        • Test procedure: description in a tool-independent manner of the test procedure. The procedure shall not be written as to exclude possible machine-testing.

        • Confidence level: micro-confidence level for the individual test. Do not confuse with the confidence level of the conformance claim.

        • (Optional) User testing procedures: set of recommendations that can complement the above procedures when performing user testing.

      3. CSS-specific tests 

        Set of tests to be made for conformance claims for CSS resources. Each test consists of:

        • Title and ID: short descriptive title (informative) and unique identifier (normative).

        • Applicability criteria: CSS selectors, properties and combinations thereof used to determine the applicability of the test.

        • Test procedure: description in a tool-independent manner of the test procedure. The procedure shall not be written as to exclude possible machine-testing.

        • Confidence level: micro-confidence level for the individual test. Do not confuse with the confidence level of the conformance claim.

        • (Optional) User testing procedures: set of recommendations that can complement the above procedures when performing user testing.

  2. (Optional) Additional clarification issues, such as definition pointers.

This section will not repeat information available in W3C documents. It will provide pointers to the relevant places and will extend only information when necessary for the defined tests.

5.2 Guideline 1

“Provide equivalent alternatives to auditory and visual content.”

(See http://www.w3.org/TR/WCAG10/#gl-provide-equivalents)

This guideline provides information on how to support with complementary text alternatives auditory and visual content.

5.2.1 Checkpoint 1.1

Provide a text equivalent for every non-text element (e.g., via "alt", "longdesc", or in element content). This includes: images, graphical representations of text (including symbols), image map regions, animations (e.g., animated GIFs), applets and programmatic objects, art, frames, scripts, images used as list bullets, spacers, graphical buttons, sounds (played with or without user interaction), stand-alone audio files, audio tracks of video, and video. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-text-equivalent and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-text-equivalent)

5.2.1.1 Summary

Table 1: UWEM 0.5 tests for checkpoint 1.1.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 1.1_HTML_01 img/@alt, area/@alt,
input[@type='image']/@alt,
applet/@alt, object/*

Select non-text-elements without text alternative.

High
1.1_HTML_02 img/@alt, area/@alt,
input[@type='image']/@alt,
applet/@alt, object/*

Select non-text-elements with empty text alternative. Decide whether non-text-element is purely decorative.

High
1.1_HTML_03 img/@alt, area/@alt,
input[@type='image']/@alt,
applet/@alt, object/*

Select non-text-elements with text alternative containing only whitespace. Decide whether non-text-element represents whitespace.

High
1.1_HTML_04 img/@alt, area/@alt,
input[@type='image']/@alt,
applet/@alt, object/*

Select non-text-elements with non-empty, non-whitespace-only text alternative. Decide whether text alternative represents the non-text-element's function within the context.

High
1.1_HTML_05 img/@longdesc,
object//a/@href

Select long description document referenced by the non-text-element. Decide whether the non-text-element is described by text in the document.

High
CSS N/A

5.2.1.2 (X)HTML tests

5.2.1.2.1 Test 1.1_HTML_01

This test is targeted to find media elements without a text alternative.

5.2.1.2.2 Test 1.1_HTML_02

This test is targeted to analyse non-text elements with an empty text alternative.

5.2.1.2.3 Test 1.1_HTML_03

This test is targeted to analyse non-text elements with only white space as a text alternative.

5.2.1.2.4 Test 1.1_HTML_04

This test is targeted to analyse non-text elements with non-whitespace-only text alternative.

5.2.1.2.5 Test 1.1_HTML_05

This test is targeted to analyse long descriptions of media elements.

5.2.1.3 CSS tests

For this checkpoint there are not applicable tests.

5.2.2 Checkpoint 1.2

Provide redundant text links for each active region of a server-side image map. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-redundant-server-links and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-redundant-server-links)

5.2.2.1 Summary

Table 2: UWEM 0.5 tests for checkpoint 1.2.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 1.2_HTML_01 img/@ismap,
object[@type='image']/@ismap

Select active region without redundant text link.

High
CSS N/A

5.2.2.2 (X)HTML tests

5.2.2.2.1 Test 1.2_HTML_01

This test is targeted to find active regions of a server-side image map without redundant text link.

5.2.2.3 CSS tests

For this checkpoint there are no applicable tests.

5.2.3 Checkpoint 1.3

Until user agents can automatically read aloud the text equivalent of a visual track, provide an auditory description of the important information of the visual track of a multimedia presentation. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-auditory-descriptions and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-auditory-descriptions)

5.2.3.1 Summary

Table 3: UWEM 0.5 tests for checkpoint 1.3.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 1.3_HTML_01 object, applet, a

Select multimedia presentations without auditory description of the important information of the visual track.

High
CSS N/A

5.2.3.2 (X)HTML tests

5.2.3.2.1 Test 1.3_HTML_01

This test is targeted to find multimedia presentations without an auditory description of the important information of their visual track.

5.2.3.3 CSS tests

For this checkpoint there are not applicable tests.

5.3 Guideline 2

“Don't rely on color alone.”

(See http://www.w3.org/TR/WCAG10/#gl-color)

This guideline provides information on how to use color appropriately.

5.3.1 Checkpoint 2.1

Ensure that all information conveyed with color is also available without color, for example from context or markup. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-color-convey and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-color-convey)

5.3.1.1 Summary

Table 4: UWEM 0.5 tests for checkpoint 2.1.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 2.1_HTML_01 body

Decide whether references in text via color are redundant.

High
2.1_HTML_02 img, area,
input[@type='image'], applet, object

Select non-text-elements. Decide whether color information is redundant.

High
2.1_HTML_03 */@color, */@bgcolor,
*/@link, */@vlink, */@alink,
*/@text

Select elements with attributes defining colors. Decide whether color information is redundant.

CSS 2.1_CSS_01 color, background-color,
background, border-color,
border, outline-color, outline

Select elements for which colors are defined. Decide whether color information is redundant.

5.3.1.2 (X)HTML tests

5.3.1.2.1 Test 2.1_HTML_01

This test is targeted to find phrases in text that refer to parts of a document only by mentioning their color.

5.3.1.2.2 Test 2.1_HTML_02

This test is targeted to find phrases in non-text resources that refer to parts of a document only by mentioning their color.

5.3.1.2.3 Test 2.1_HTML_03

This test is targeted to find colored elements without redundant methods of conveying the information.

5.3.1.3 CSS tests

5.3.1.3.1 Test 2.1_CSS_01

This test is targeted to find colored elements without redundant methods of conveying the information.

5.3.2 Checkpoint 2.2

Ensure that foreground and background color combinations provide sufficient contrast when viewed by someone having color deficits or when viewed on a black and white screen. [Priority 2 for images, Priority 3 for text].

(See http://www.w3.org/TR/WCAG10/#tech-color-contrast and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-color-contrast)

5.3.2.1 Summary

Table 5: UWEM 0.5 tests for checkpoint 2.2.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 2.2_HTML_01 img, area,
input[@type='image'], applet,
object

Select non-text-elements. Decide whether contrast in non-text-element is high enough to convey the information

Low
2.2_HTML_02 */@color, */@bgcolor

Select elements with attributes defining colors. Check contrast.

Low
2.2_HTML_03 */@link, */@vlink, */@alink,
*/@text

Select elements with attributes defining colors. Check contrast.

Low
CSS 2.2_CSS_01 color, background-color

Select elements for which color OR background-color are defined. Check whether both color and background-color are defined.

High
2.2_CSS_02 color, background-color

Select elements for which color and/or background-color are defined. Check color contrast.

Low
2.2_CSS_03 :link color, :link background-color, :link background,
:visited color, :visited background-color, :visited background,
:hover color, :hover background-color, :hover background,
:active color, :active background-color, :active background

Select pseudo classes for links where color and/or background color are defined. Check color contrast.

Low

5.3.2.2 (X)HTML tests

5.3.2.2.1 Test 2.2_HTML_01

This test is targeted to find media elements without enough color contrast.

5.3.2.2.2 Test 2.2_HTML_02

This test is targeted to find text without enough color contrast.

5.3.2.2.3 Test 2.2_HTML_03

This test is targeted to find text without enough color contrast.

5.3.2.3 CSS tests

5.3.2.3.1 Test 2.2_CSS_01

This test is targeted to find text without enough color contrast.

5.3.2.3.2 Test 2.2_CSS_02

This test is targeted to find text without enough color contrast.

5.3.2.3.3 Test 2.2_CSS_03

This test is targeted to find text without enough color contrast.

5.4 Guideline 4

“Clarify natural language usage.”

(See http://www.w3.org/TR/WCAG10/#gl-abbreviated-and-foreign)

This guideline provides information on how to facilitate pronunciation or interpretation of abbreviated or foreign text.

5.4.1 Checkpoint 4.1

Clearly identify changes in the natural language of a document's text and any  text equivalents (e.g., captions). [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-identify-changes and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-identify-changes)

5.4.1.1 Summary

Table 6: UWEM 0.5 tests for checkpoint 4.1.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 4.1_HTML_01 text(), img/@alt, applet/@alt,
area/@alt, input/@alt,
meta/@content,
option/@label,
optgroup/@label,
object/@standby,
table/@summary, */@title
(all elements except base, basefont, head, html, meta, param, script and title), input[@type='text']/@value, input[@type='submit']/@value, frame/@name, iframe/@name

For each word, attribute value, text node and element, check if language corresponds with:

a) the specified language of its nearest ancestor,  

b) the predominant language if no ancestor specifies language.

High
4.1_HTML_02 text(), img/@alt, applet/@alt,
area/@alt, input/@alt,
meta/@content,
option/@label,
optgroup/@label,
object/@standby,
table/@summary, */@title
(all elements except base, basefont, head, html, meta, param, script and title), input[@type='text']/@value, input[@type='submit']/@value, frame/@name, iframe/@name

For each word, attribute value, text node and element, check if text direction corresponds with:

a) the specified text direction of its nearest ancestor,  

b) the predominant text direction if no ancestor specifies text direction.

High
CSS 4.1_CSS_01 *:after {content: “...”;},
*:before {content: “...”;},
* { content: “...”},
* { cue: url(“...”);},
* { cue-before: url(“...”);},
* { cue-after: url(“...”);},
* { list-style-type: ...;},
* { list-style: ...;}

Select any elements which have associated CSS rules that generate content. Check if the generated content is in a different language than the context.

5.4.1.2 (X)HTML tests

5.4.1.2.1 Test 4.1_HTML_01

This test is targeted to find changes in natural language that are not marked up. Marking up natural language changes can make a document more accessible to multilingual users.

5.4.1.2.2 Test 4.1_HTML_02

This test is targeted to find changes in text direction (for natural languages) that are not marked up.

5.4.1.3 CSS tests

5.4.1.3.1 Test 4.1_CSS_01

This test is targeted to analyse long descriptions of media elements.

5.5 Guideline 5

“Create tables that transform gracefully.”

(See http://www.w3.org/TR/WCAG10/#gl-table-markup)

This guideline provides information on how to identify properly marked up tables,

5.5.1 Checkpoint 5.1

For data tables, identify row and column headers. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-table-headers and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-table-headers)

Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 5.1_HTML_01 th/@scope,
th/@id,
td/@headers

Verify that the scope attribute for the table heading is set. Test passes if it is set. Otherwise, continue with verifying that id attribute in table header exists, that the headers attribute in table definition exists, and that the table definition parameters matches the table header IDs.

High
5.1_HTML_02 th/@axis,
td/@axis

If the axis attribute is used in tables, check that it is used consistently.

Medium
5.1_HTML_03 table/colgroup,
table/thead,
table/tfoot,
table/tbody

Verify that colgroup, if present, comes first, tfoot comes before tbody. Test 5.1 fails if colgroup, thead, tfoot and tbody are used inconsistently.

High
5.1_HTML_04 pre

Inspect the preformatted text to see if it could have been better presented as a proper table.

Medium
CSS N/A

Table 7: UWEM 0.5 tests for checkpoint 5.1.

5.5.1.1 (X)HTML tests

5.5.1.1.1 Test 5.1_HTML_01

This test is targeted to find table cell elements with scope attribute or id and headers attributes that can aid assistive technology in presenting the  information in the table properly.

5.5.1.1.2 Test 5.1_HTML_02

This test is targeted to find table element with axis attribute that can aid assistive technology in presenting varying type specific information in the table columns properly.

5.5.1.1.3 Test 5.1_HTML_03

This test is targeted to aid assisting technology in identifying information about repeated table headers, footers and table row and column grouping.

5.5.1.1.4 Test 5.1_HTML_04

This test is targeted to identify preformatted text used instead of proper tables.

5.5.2 Checkpoint 5.2

For data tables that have two or more logical levels of row or column headers, use markup to associate data cells and header cells. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-table-structure and the techniques in http://www.w3.org/TR/WCAG10-HTML-TECHS/#identifying-table-rows-columns

Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 5.2_HTML_01 table

If the table contains more than two logical levels of row or column headers, verify that the table marks up the rows and columns properly by using test 5.1. If this test fails, or returns CannotTell/ NotApplicable, then test 5.2 fails.

High
CSS N/A
5.5.2.0.1 Test 5.2_HTML_01

This test is targeted to identify tables with more than two logical levels of rows or columns that are not marked up properly by using table markup that accosiates rows and columns and can aid assistive technologies in presenting the tables properly.

5.6 Guideline 6

“Ensure that pages featuring new technologies transform gracefully.”

(See http://www.w3.org/TR/WCAG10/#gl-new-technologies)

This guideline provides information on ensuring that pages are accessible even when newer technologies are not supported or are turned off.

5.6.1 Checkpoint 6.1

Organize documents so they may be read without style sheets. For example, when an HTML document is rendered without associated style sheets, it must still be possible to read the document. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-order-style-sheets and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-order-style-sheets)

5.6.1.1 Summary

Table 8: UWEM 0.5 tests for checkpoint 6.1.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 6.1_HTML_01 link[@rel='stylesheet'],
style, */@style

Deactivate all CSS applied to document. Ensure that:

  1. Content can be read.

  2. Content does not become invisible.

  3. Content is not obscured by other content.

  4. The intended reading order is maintained.

High
6.1_HTML_02 script, */@onfocus, */@onblur, */@onkeypress, */@onkeydown, */@onkeyup, */@onsubmit, */@onreset, */@onselect, */@onchange
(*/@onload, */@onunload, */@onclick, */@ondblclick, */@onmousedown, */@onmouseup, */@onmouseover, */@onmousemove, */@onmouseout)

Deactivate all CSS applied to document by script. Ensure that:

  1. Content can be read.

  2. Content does not become invisible.

  3. Content is not obscured by other content.

  4. The intended reading order is maintained.

High
CSS N/A

5.6.1.2 (X)HTML tests

5.6.1.2.1 Test 6.1_HTML_01

This test analyses the effect on the readability of the document of CSS applied in standalone stylesheets, embedded stylesheets and style attributes of document elements.

5.6.1.2.2 Test 6.1_HTML_02

This test analyses the effect on the readability of the document of CSS applied programmatically.

5.6.1.3 CSS tests

For this checkpoint there are no applicable tests.

5.6.2 Checkpoint 6.2

Ensure that equivalents for dynamic content are updated when the dynamic content changes. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-dynamic-source and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-dynamic-source)

5.6.2.1 Summary

Table 9: UWEM 0.5 tests for checkpoint 6.2.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 6.2_HTML_01 frame/@src

Check that the content pointed to by the src attribute complies with checkpoint 1.1.

High
6.2_HTML_02 frame, a/@href,
script, */@onfocus, */@onblur, */@onkeypress, */@onkeydown, */@onkeyup, */@onsubmit, */@onreset, */@onselect, */@onchange
(*/@onload, */@onunload, */@onclick, */@ondblclick, */@onmousedown, */@onmouseup, */@onmouseover, */@onmousemove, */@onmouseout)

Search for links, script or other elements in the delivery unit that cause other content to be loaded into the frame. Ensure that content loaded into the frame complies with checkpoint 1.1.

High
CSS N/A

5.6.2.2 (X)HTML tests

5.6.2.2.1 Test 6.2_HTML_01

This test analyses the text equivalent of any non-text content loaded into the frame by the browser as a result of the value the src attribute.

5.6.2.2.2 Test 6.2_HTML_02

This test analyses the text equivalent of any non-text content loaded into the frame by the browser as a result of link activation or script execution.

5.6.2.3 CSS tests

For this checkpoint there are no applicable tests.

5.6.3 Checkpoint 6.3

Ensure that pages are usable when scripts, applets, or other programmatic objects are turned off or not supported. If this is not possible, provide equivalent information on an alternative accessible page. [Priority 1]

(See http://www.w3.org/TR/WCAG10/#tech-scripts and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-scripts)

5.6.3.1 Summary

Table 10: UWEM 0.5 tests for checkpoint 6.3.
Format Test ID Elements/
Attributes/
Selectors
Inspection Procedure Conf. Level
HTML 6.3_HTML_01 object, applet, embed

Ensure that all information and functionality provided by the embedded objects is available when these are not loaded or do not function as intended.

High
6.3_HTML_02 script, a[starts-with(@href, 'javascript:')], */@onfocus, */@onblur, */@onkeypress, */@onkeydown, */@onkeyup, */@onsubmit, */@onreset, */@onselect, */@onchange (*/@onload, */@onunload, */@onclick, */@ondblclick, */@onmousedown, */@onmouseup, */@onmouseover, */@onmousemove, */@onmouseout)

Ensure that all information and functionality provided by the script is available when this is not executed.

High
CSS N/A

5.6.3.2 (X)HTML tests

5.6.3.2.1 Test 6.3_HTML_01

This test determines whether information and functionality provided by embedded content is also available without said content.