Web Accessibility Benchmarking Cluster

Sixth Framework Programme
Information Society Technologies Priority
| Contractual Date of Delivery to the EC: | 28 February 2005 + 45 days |
|---|---|
| Actual Date of Delivery to the EC: | 12 October 2005 |
| Editors: |
|
| Contributors: | See Appendix E for contributors list |
| Workpackage: | WAB1a |
| Security: | Public |
| Nature: | Report |
| Version: | P |
| Total number of pages: |
Web Accessibility, WAB Cluster, World Wide Web Consortium, W3C, Web Accessibility Initiative, WAI, Web Content Accessibility Guidelines, WCAG, Unified Web site Evaluation Methodology, UWEM, Evaluation and Report Language, EARL.
| Version | Version date | Responsible | Description |
|---|---|---|---|
| A | 05-05-22 | FIT | Initial version with TOC |
| B | 05-03-06 | FIT | Version containing Section 3, and integrating contributions to Sections 4, 5 and 7, plus Appendices A, C, D and E |
| C | 05-06-05 | Agder University College | Added section 2.6 and 2.8 |
| D | 05-06-05 | DCU | Revised section 4; added section 12.2. |
| E | 10-06-05 | Accessibility Foundation | Results of Conclave work: 6, 7 and 8 June 2005 |
| F | 05-06-20 | City University | Updated version of section 7; appendices for section 7. |
| G | 05-06-21 | Eric Velleman | Updated sections with comments from W3C. |
| H | 05-06-25 | Agder University College | Converted to master document, for easier maintenance and removed material from the evaluation suite in section 3. |
| I | 05-06-25 | Agder University College | Added latest version of section 4 that Barry had uploaded,and the glossary. |
| J | 05-06-25 | Agder University College | Added information on Xiaoming Zeng's WABScore method, and elaborated on aggregating page fractions, instead of whole pages for key use scenarios and a connection to sampling/ random walk algorithms to section 6. |
| K | 05-08-09 | Agder University College | Section 6 split into core part and complementary section. |
| L | 05-08-20 | Agder University College | Section 6 restructured for better readability, and updated with comments from Andreas Prinz. Minor typos fixed in section 4 and 8. |
| M | 05-08-22 | Agder University College | Fixed link to references and glossary. |
| N | 05-09-30 | Accessibility Foundation, FIT | Global copy-editing; replacement of missing figures; full modification of Section 5 according to BenToWeb’s proposal; missing references and cross-references fixed. |
| O | 05-10-12 | FIT | Incorporation of W3C copyright notice and final remarks from WAI and Cluster projects. From Agder University College: Incorporation of review comments from the EIAO partners (05-10-11) |
| P | 05-10-12 | Accessibility Foundation | Change to section 2 following remark from WAI |
Skip table of contents to content
This document is the result of a joint effort by 24 European organisations in three European projects combined in a cluster to develop a Unified Web Evaluation Methodology (UWEM). This methodology is based on the W3C Web Content Accessibility Guidelines 1.0 [WCAG10] and will be synchronised with the foreseen migration from WCAG 1.0 to WCAG 2.0 [WCAG20] in the near future. The UWEM offers an interpretation of the guidelines agreed among stakeholders within the aforementioned projects. This document is a draft and shall not be considered as an stable version. This document will serve as the basis for a evaluation of the draft methodology by all intended users.
The projects involved in making this methodology aim to ensure that evaluation tools and methods developed for global monitoring or for local evaluation, are compatible and coherent among themselves and with WAI. The projects will provide feedback and contribution to WAI for future guidelines or versions of guidelines. Their aim is to increase the value of evaluations by basing them on a shared interpretation of WCAG 1.0.
This document presents the 0.5 version of the Unified Web Evaluation Methodology, which is based on the W3C Web Content Accessibility Guidelines 1.0. UWEM 0.5 will be extensively evaluated in the next phase of the Cluster including user-, expert and (semi-) automated testing of the assessment methodology. This phase will also include a phase for public comments, evaluation and discussion.
UWEM 0.5 provides an evaluation procedure consisting of a system of principles and practices for manual and automatic evaluation of Web accessibility for humans and machine interfaces. The methodology aims to be fully conformant with WCAG 1.0 guidelines. Currently the UWEM is limited to priority 1 guidelines, but in the coming phase of the Cluster, priority 2 guidelines will be added.
The methodology covers evaluations of one Web page, an entire site (irrespective of size), or multiple sites, a method for sampling, clarifications of the checkpoints, user testing protocols and information necessary for interpretation and integration/aggregation of results.
The UWEM 0.5 will be used by the projects included in the Cluster for an observatory (EIAO project), tools for benchmarking (BenToWeb project) and a certification scheme (SEAM project). More information about the Cluster, the UWEM and the projects involved can be found at: http://www.wabcluster.org/. Refer to Appendix A for the document license.
The document is organised in the following way. Section 2 outlines the requirements for the document and the basic properties of UWEM. Section 3 describes the UWEM conformance related to a sample to be evaluated, types of tests to conduct, conformance claims and confidence levels for the test results. Section 4 describes the approaches for scoping and sampling of a Web site for evaluation. The sampled resource set should cover critical tasks and secure a representative evaluation result. Section 5 describes how to carry out the tests related to the WCAG 1.0 checkpoints, including clarifications and user testing information. Section 6 describes a model for aggregating test results from accessibility barriers on a Web resource to regional aggregations of Web resources. The model also refers to a possible aggregation over different user groups who may expereince different barriers. Section 7 describes user testing protocols according to UWEM. The section also describes how to select a set of tasks to accomplish and how to compose a group of testers. Section 8 gives a description on how to present the results of the evaluation, specially taylored for policy makers using a score card approach. There are as well a set of relevant appendices.
The Unified Web Evaluation Methodology should support that evaluation tools and methods developed for large scale monitoring or for local evaluation, are compatible and coherent among themselves and with W3C/WAI. This document is the result of a joint effort of three European projects with 24 organisations collaborating in the WAB Cluster to develop UWEM.
The purpose of the 0.5 version of UWEM is to provide a basis for evaluating the methodology from all the intended types of testing, including: user-, expert- and (semi-) automated testing of Web resources. The evaluation of UWEM is also planned to provide feedback and contribute to W3C/WAI for future guidelines or versions of guidelines. W3C/WAI staff have reviewed and provided input into previous drafts of this document in order to minimize potential fragmentation of technical content. This does not imply W3C or WAI endorsement of any part of this draft. W3C/WAI Working Groups have not yet reviewed the draft, and will have the opportunity to do so along with the public.
Part of the materials presented in this document are annotations of W3C documents (those included in section 5). In particular, we are targeting the following two documents:
According to the Intellectual Rights FAQ from W3C, section 5 of UWEM falls under an annotation “... that does not require the copying and modification of the document being annotated”. Therefore, all references to guidelines and checkpoints are duly quoted, and the URL to the original document is included. W3C is not responsible for any content not found at the original URL, and our annotations are non-normative.
The UWEM is a Web evaluation methodology that provides an evaluation procedure consisting of a system of principles and practices for manual and automatic evaluation of Web accessibility for humans and machine interfaces. Version 0.5 of the methodology is designed to be conformant with WCAG 1.0 priority 1 checkpoints with regard to technical criteria. Future versions of this methodology are intended to be conformant with WCAG 2.0 from the same standpoint.
UWEM 0.5 offers an interpretation of the guidelines to be agreed among European stakeholders. It aims to increase the value of evaluations by basing them on a shared interpretation of WCAG 1.0 and a set of tests that are sufficiently robust to give stakeholders confidence in results. Web content producers may also wish to evaluate their own content and UWEM aims to also be suitable to these users.
The methodology is designed to meet the following requirements:
In the methodology we have included information about:
The target audience for this document include for example:
The European Commission, national governments and other organisations who wish to carry out benchmarking projects on Web accessibility will be able to use the UWEM to carry out the evaluations and compare their results in a meaningful way.
UWEM is an evaluation methodology and is not intended to provide information for Web content producers wishing to produce content compliant with WCAG 1.0. This information is provided in the WCAG 1.0 Techniques Documents that are available through the W3C/WAI website [WCAG10-TECHS].
The 0.5 version of UWEM covers methods to evaluate documents based on the following technologies:
The following organisations worked on this UWEM document:
Accessibility Foundation (The Netherlands, Cluster coordinator); Agder University College (Norway, EIAO coordinator); Fraunhofer Institute for Applied Information Technology FIT (Germany, BenToWeb coordinator); Association Braillenet (France, SupportEAM coordinator), Vista Utredning AS (Norway); Forschungsintitut Technologie-Behindertenhilfe der Evangelischen Stiftung Volmarstein (Germany); The Manchester Metropolitan University (UK); Nettkroken as (Norway); University of Tromsø (Norway); FBL s.r.l. (Italy); Warsaw University of Technology, Faculty of Production Engineering, Institute of Production Systems Organisation (Poland); Aalborg University (Denmark); Intermedium as (Norway); Fundosa Teleservicios (Spain); Dublin City University (Ireland); Universität Linz, Institut integriert studieren (Austria); Katholieke Universiteit Leuven, Research & Development (Belgium); Accessinmind Limited (UK); Multimedia Campus Kiel (Germany); Department of Product and Systems Design, University of the Aegean (Greece); City University London (UK); ISdAC International Association (Belgium); FernUniversität in Hagen (Germany).
We thank the Web Accessibility Initiative's Team from the World Wide Web Consortium for all the useful criticisms to the different versions of this document.
The projects participating in the WAB cluster are funded by the European Union in the second FP6 IST call (2003) of the eInclusion Strategic Objective. The WAB cluster Web site is available at http://www.wabcluster.org/. More information about the projects can be found on the project websites:
The evaluation procedures of UWEM 0.5 have the following objectives:
To improve replicability of its evaluation results, for which the scoping and sampling scheme as described in section 4 is proposed.
To clarify interpretation, facilitate automatic evaluation and improve replicability of its test results (see section 5).
To support aggregation of the evaluation results for individual Web pages, Web sites and even sets of Web sites. UWEM 0.5 will also support aggregations focusing on different user groups, as indicated in section 6.
To support large scale screening of Web sites, to identify problem areas in Web sites.
To evaluate conformance to the checks of section 5 to the selected level. It includes procedures for evaluation down to specific test level during development of Web sites.
Large scale screening of Web sites, is an UWEM specific method that may help to quickly identify the scope of problems, or monitoring progress for a large number of Web sites or individual Web sites for some disability groups. However, automatic screening will not catch all of the problems on a site and should not be used to determine conformance.
The different types of evaluation methods have a number of strengths and weaknesses, as well as different levels of confidence associated with them. describes the levels of confidence of four different evaluation methods in their ability to benchmark accessibility.

Figure 1: UWEM Confidence Levels and Evaluation Types.
The figure shows for example, that automatic evaluation (Tool1 or Tool2) can only test for conformance to a subset of the checkpoints (such as the set provided in section 5), which further means that a subset of all possible accessibility barriers can be identified reliably by using automatic testing. This means that confidence in automatic evaluation as an overall indicator of accessibility is low, however it can identify some barriers reliably. Tool 1 and Tool 2 are here two fully automatic assessment tools that focus on checking slightly different accessibility issues, with some overlap of functionality.
Some tools can also act as decision support systems in a semi-automatic evaluation process where the system aids the testing process and points out where the testers should focus and possibly give hints about earlier decisions in addition to performing some tasks automatically. Expert testing, usually done with a semi-automatic testing/decision support system, will typically be more precise than semi-automatic testing done by a non-expert.
User testing is able to identify some barriers that are not caught by other testing means, and is able to identify barriers and also estimate the accessibility for the tested scenarios with a high level of confidence. However, user testing is also quite specialised, so that it is not generally suitable for conformance testing, since it is not able to test all aspects of the tests of section 5. The best approach to ensure both accessibility and UWEM conformance is to use a combined approach with both expert testing and user testing of the Web site.
The main advantages of automatic testing are:
The method is replicable (although the dynamic nature of some Web technologies mean complete replication is impossible [LEVENE99]).
It may be applied to very large number of resources on a Web site and to multiple Web sites.
The highest level of confidence is achieved by a combined approach of user testing and expert testing involving all possible tasks supported by a Web site. This method also provides feedback to the developers for the improvement of the accessibility and usability of the site. However, it is an expensive and time-consuming method, and for the purposes of UWEM 0.5, we propose user testing on a subset of tasks (see sections 4 and 7). See section 8 for reporting of results as scorecards.
UWEM 0.5 conformance requires that all the tests to the selected level in section 5 pass for every test on every page of the Web site according to the scope defined in section 4.
The claims of conformance of accessibility according to the UWEM methodology must use the following form:
The UWEM version and its URI identifier, e.g., http://www.wabcluster.org/refs/UWEM-0.5/
The URI to a document detailing the evaluation set (scope pattern list) to which the claim refers. A resource set conforms at a given confidence level only if all content provided by that resource so conforms.
The level of confidence being claimed.
Confidence in the UWEM evaluation results is expressed not only by the choice of a method, but also by providing information about the sampling of resources from the Web site (see section 4) and the set of criteria that have been applied (see section 5). This confidence is at a macro-level and is not the same term as used for the tests of section 5.
Section 5 states the technologies to which UWEM can be applied (W3C technologies (X)HTML and CSS in different versions). There are sites therefore to which UWEM cannot technically be applied, like, e.g., a site making exclusive use of Macromedia Flash. For these sites, user testing might be necessary, although it cannot provide a complete view on the accessibility of the site.
The following levels of confidence in the different individual methods for assessing accessibility are proposed (see ). Note that every level includes the precedent:
In deciding which method or combination of methods to choose for assessing accessibility, a Web site owner should bear in mind the above levels of confidence and also use the individual Web site accessibility scorecard presented in section 8.
For the purposes of the UWEM a Web site is defined as an arbitrary collection of hyperlinked Web resources, each identified by one or more URIs [RFC2396] (each URI optionally complemented with a set of additional parameters, as defined by the XML Schema described in Appendix C). The scope of a Web site – the specific set of resources considered as belonging to it — can therefore be identified or expressed by giving a clear, repeatable and unambiguous procedure for deciding whether any arbitrary URI does or does not belong to the site.
According to the needs of different applications of UWEM, scope may be specified by a variety of different participants in the evaluation process – such as a site owner, a site operator, an inspection organisation, etc. This document does not offer advice on deciding upon the appropriate scope for any particular UWEM application: it only explains how such a scope should be unambiguously expressed.
The scope of a Web site should be expressed in the form of an ordered list of resource patterns, termed a Scope Pattern List. Each pattern is designated as either an include pattern or an exclude pattern. Each pattern is expressed using an XML Schema regular expression [XMLSCHEMA2]. The complete Scope Pattern List should be expressed using the XML Schema described in Appendix C. The scope status of any arbitrary URI is then determined by testing it against each pattern in turn until a match is found. If the match is to an include pattern, then the identified resource is inside the scope of the site. If the match is to an exclude pattern, or if no match is found, then the identified resource is outside the scope of the site.
Section 4.1 explains how the scope of a Web site should be identified, i.e., how to express a rule for deciding whether a given resource is inside or outside the scope of a site. However, this does not, in itself, identify any particular resources which indeed lie inside the scope of the Web site. Such an explicit set of particular resources is a pre-requisite for any evaluation of a Web site. The complete or exhaustive set of resources belonging to a site is called the “Complete Resource Set”.
The Complete Resource Set is normally made explicit, or catalogued, by identifying a set of one or more “seed” resources (the “Seed Resource Set”) and recursively “crawling” from there, accepting all linked resources which qualify as in scope per section 4.1. That is, starting with the Seed Resource Set, each resource is retrieved and analysed to identify any links (URIs) to other resources. Each of these URIs is assessed to see whether it is in scope. If so, the corresponding resource is also retrieved and the process is repeated. In principle this procedure should be followed exhaustively until no additional in-scope URIs can be identified.
Automated crawling is, in general, subject to limitations, for example with regard to resources accessed via form submission (typical of sites offering interactive services). This may be addressed, in certain cases, by providing additional parameters to support crawling, such as form input data, HTTP accept headers, etc., as described in Appendix C. However, in all cases, it will be important to ensure that any resources which might otherwise not be located by automated crawling, should be included as explicit elements of the Seed Resource Set. That is, it is a requirement on the choice of the Seed Resource Set that it be sufficiently extensive that recursive crawling from this set will generate the intended Complete Resource Set.
In general, the outcome of cataloguing the Complete Resource Set will depend not only on the Scope Pattern List and the Seed Resource Set, but also upon the specific, dynamic, state and configuration of the Web site when the crawling is carried out. Accordingly, any catalogue of the Complete Resource Set should be explicitly linked to the relevant Scope Pattern List and Seed Resource Set, and also be timestamped to show when it was created.
In general it will not be practical to test all site resources (all elements of the Complete Resource Set) against all evaluation criteria. Accordingly, we identify here certain subsets or “samples”, of the Complete Resource Set.
The Core Resource Set is a set of generic resources, or resource types, which are likely to be present in most Web sites, and which are core to the use and accessibility evaluation of a site. The Core Resource Set therefore represents a minimal set of resources which should be included in any accessibility evaluation of the site. The Core Resource Set cannot, in general, be automatically identified, but requires human judgement to select. The Core Resource Set should consist of as many of the following resources as are applicable and which lie within the defined scope of the site, per section 4.1:
Note that, of course, there may be some overlap in the resources identified under the different points above: any given resource should appear only once in the Core Resource Set.
A Sampled Resource Set is a resource set which is generated in a similar manner to the Complete Resource Set – by automated recursive crawling from a set of “seed” resources (either the standard Seed Resource Set, or some subset of it) – but where the crawling has been subject to certain pre-determined limits or constraints. A Sampled Resource Set would typically be used in the context of evaluations carried out over large numbers of sites (against automatic criteria only), where it is not feasible or necessary to evaluate the Complete Resource Set for each site. The basis for delimiting a Sampled Resource Set will be application specific, and should be explicitly disclosed in any evaluation report, but may, in general, include the following possible mechanisms, individually or in combinations:
The following figure illustrates a typical relationship between each of the Resource Sets discussed above. It shows that a Web site (the Complete Resource Set) is some distinguished subset of the entire Web (all existing Web resources). Sampled Resource Sets are more or less arbitrary subsets of the Complete Resource Set; so they are arbitrary subsets of the resources constituting a site. The Seed Resource Set is then a smaller subset, used to seed the crawling or cataloguing of the Complete Resource Set (and also, at least in part, any Sampled Resource Sets). Finally, the Core Resource Set is the smallest of all, being a subset of the Seed Resource Set, containing just that minimal set of resources which should be included in all evaluations.

Figure 2: Relationship between Resource Sets. See text for details.
This section covers the UWEM 0.5 testing of Priority 1 checkpoints of WCAG 1.0 [WCAG10]. Further versions of UWEM might cover additional checkpoints. This interpretation refers in particular to (X)HTML and CSS tests that MUST be carried out to be compliant with this version of UWEM.
UWEM does not impose a way to carry out the tests, and some of the following tests could be made with the help of automatic or semi-automatic tools. This section is intended to be replaced in the future by the success criteria of WCAG 2.0 [WCAG20] and the accompanying techniques documents once it becomes a W3C Recommendation.
This section contains quotes of W3C recommendations and notes (in particular, [WCAG10] and [WCAG10-TECHS]). The quoted text is in each case immediately followed by a URI linking to the quoted text in its original context in the W3C/WAI documents. These materials are copyright of W3C and its use is subject to the conditions of the W3C Document License (see Appendix F or http://www.w3.org/Consortium/Legal/2002/copyright-documents-20021231 for further information of the use of these documents).
The structure of the tests is the following:
Guideline
Quotation of the corresponding WCAG 1.0 guideline. Pointers to additional clarifications might be added.
Checkpoint
Quotation of the corresponding WCAG 1.0 checkpoint. Pointers to additional clarifications might be added.
Summary overview
Tabular overview of the tests corresponding to the checkpoint.
(X)HTML-specific tests
Set of tests to be made for conformance claims for (X)HTML resources. Each test consists of:
Title and ID: short descriptive title (informative) and unique identifier (normative).
Applicability criteria: elements, attributes and combinations thereof used to determine the applicability of the test. Whenever possible, the criteria will be presented as XPath expressions, otherwise a prose description will be given.
Test procedure: description in a tool-independent manner of the test procedure. The procedure shall not be written as to exclude possible machine-testing.
Confidence level: micro-confidence level for the individual test. Do not confuse with the confidence level of the conformance claim.
(Optional) User testing procedures: set of recommendations that can complement the above procedures when performing user testing.
CSS-specific tests
Set of tests to be made for conformance claims for CSS resources. Each test consists of:
Title and ID: short descriptive title (informative) and unique identifier (normative).
Applicability criteria: CSS selectors, properties and combinations thereof used to determine the applicability of the test.
Test procedure: description in a tool-independent manner of the test procedure. The procedure shall not be written as to exclude possible machine-testing.
Confidence level: micro-confidence level for the individual test. Do not confuse with the confidence level of the conformance claim.
(Optional) User testing procedures: set of recommendations that can complement the above procedures when performing user testing.
(Optional) Additional clarification issues, such as definition pointers.
This section will not repeat information available in W3C documents. It will provide pointers to the relevant places and will extend only information when necessary for the defined tests.
“Provide equivalent alternatives to auditory and visual content.”
(See http://www.w3.org/TR/WCAG10/#gl-provide-equivalents)
This guideline provides information on how to support with complementary text alternatives auditory and visual content.
Provide a text equivalent for every non-text element (e.g., via "alt", "longdesc", or in element content). This includes: images, graphical representations of text (including symbols), image map regions, animations (e.g., animated GIFs), applets and programmatic objects, art, frames, scripts, images used as list bullets, spacers, graphical buttons, sounds (played with or without user interaction), stand-alone audio files, audio tracks of video, and video. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-text-equivalent and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-text-equivalent)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 1.1_HTML_01 | img/@alt, area/@alt, input[@type='image']/@alt, applet/@alt, object/* |
Select non-text-elements without text alternative. |
High |
| 1.1_HTML_02 | img/@alt, area/@alt, input[@type='image']/@alt, applet/@alt, object/* |
Select non-text-elements with empty text alternative. Decide whether non-text-element is purely decorative. |
High | |
| 1.1_HTML_03 | img/@alt, area/@alt, input[@type='image']/@alt, applet/@alt, object/* |
Select non-text-elements with text alternative containing only whitespace. Decide whether non-text-element represents whitespace. |
High | |
| 1.1_HTML_04 | img/@alt, area/@alt, input[@type='image']/@alt, applet/@alt, object/* |
Select non-text-elements with non-empty, non-whitespace-only text alternative. Decide whether text alternative represents the non-text-element's function within the context. |
High | |
| 1.1_HTML_05 | img/@longdesc, object//a/@href |
Select long description document referenced by the non-text-element. Decide whether the non-text-element is described by text in the document. |
High | |
| CSS | N/A |
This test is targeted to find media elements without a text alternative.
Applicability criteria: Select the following combination of elements/attributes:
img/@alt
area/@alt
input[@type='image']/@alt
applet/@alt
object/*
Test procedure: Select non-text-elements without an alt attribute. The non-availability of any text alternative leads to an error.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to analyse non-text elements with an empty text alternative.
Applicability criteria: Select the following combination of elements/attributes:
img/@alt
area/@alt
input[@type='image']/@alt
applet/@alt
object/*
Test procedure: Select non-text-elements without an empty text alternative. The presence of an empty alt attribute indicates the possibility of a pure decorative image. If that is not the case, an error must be flagged.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to analyse non-text elements with only white space as a text alternative.
Applicability criteria: Select the following combination of elements/attributes:
img/@alt
area/@alt
input[@type='image']/@alt
applet/@alt
object/*
Test procedure: Select non-text-elements with only white space as a text alternative. The presence of a white space as an alt attribute indicates the possibility of a pure decorative image. However, the convention is to use an empty alt attribute, and therefore an error must be flagged unless the media element truly represents a white space.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to analyse non-text elements with non-whitespace-only text alternative.
Applicability criteria: Select the following combination of elements/attributes:
img/@alt
area/@alt
input[@type='image']/@alt
applet/@alt
object/*
Test procedure: Select non-text-elements with non-whitespace-only text alternative. Decide whether the text alternative appropriately represents the non-text-element's function within the context.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to analyse long descriptions of media elements.
Applicability criteria: Select the following combination of elements/attributes:
img/@longdesc
object//a/@href
Test procedure: Examine the long descriptions of the media elements (linked from the above attributes), and decide whether they describe appropriately the element.
Confidence level: High.
User testing procedures: Not Available.
For this checkpoint there are not applicable tests.
Provide redundant text links for each active region of a server-side image map. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-redundant-server-links and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-redundant-server-links)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 1.2_HTML_01 | img/@ismap, object[@type='image']/@ismap |
Select active region without redundant text link. |
High |
| CSS | N/A |
This test is targeted to find active regions of a server-side image map without redundant text link.
Applicability criteria: Select the following combination of elements/attributes:
img/@ismap
object[type='image']/@ismap
Test procedure: Identify the active regions of the image map. Select the regions that don't have a redundant text link on the page. The non-availability of a text link leads to an error.
Confidence level: High.
User testing procedures: Not Available.
For this checkpoint there are no applicable tests.
Until user agents can automatically read aloud the text equivalent of a visual track, provide an auditory description of the important information of the visual track of a multimedia presentation. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-auditory-descriptions and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-auditory-descriptions)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 1.3_HTML_01 | object, applet, a | Select multimedia presentations without auditory description of the important information of the visual track. |
High |
| CSS | N/A |
This test is targeted to find multimedia presentations without an auditory description of the important information of their visual track.
Applicability criteria: Select the following combination of elements/attributes:
object
applet
a
Test procedure: Select multimedia presentations with an auditory description of the important information of their visual track. The non-availability of an auditory description leads to an error.
Confidence level: High.
User testing procedures: Not Available.
For this checkpoint there are not applicable tests.
“Don't rely on color alone.”
(See http://www.w3.org/TR/WCAG10/#gl-color)
This guideline provides information on how to use color appropriately.
Ensure that all information conveyed with color is also available without color, for example from context or markup. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-color-convey and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-color-convey)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 2.1_HTML_01 | body | Decide whether references in text via color are redundant. |
High |
| 2.1_HTML_02 | img, area, input[@type='image'], applet, object |
Select non-text-elements. Decide whether color information is redundant. |
High | |
| 2.1_HTML_03 | */@color, */@bgcolor, */@link, */@vlink, */@alink, */@text |
Select elements with attributes defining colors. Decide whether color information is redundant. |
||
| CSS | 2.1_CSS_01 | color, background-color, background, border-color, border, outline-color, outline |
Select elements for which colors are defined. Decide whether color information is redundant. |
This test is targeted to find phrases in text that refer to parts of a document only by mentioning their color.
Applicability criteria: Select the following combination of elements/attributes:
body
Test procedure: Decide whether references in text via color are redundant.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to find phrases in non-text resources that refer to parts of a document only by mentioning their color.
Applicability criteria: Select the following combination of elements/attributes:
img
area
input[@type='image']
applet
object
Test procedure: Decide whether color information is redundant.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to find colored elements without redundant methods of conveying the information.
Applicability criteria: Select the following combination of elements/attributes:
*/@color
*/@bgcolor
*/@link
*/@vlink
*/@alink
*/@text
Test procedure: Decide whether color information is redundant.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to find colored elements without redundant methods of conveying the information.
Applicability criteria: Select the following combination of elements/attributes:
color
background-color
background
border-color
border
outline-color
outline
Test procedure: Decide whether color information is redundant.
Confidence level: High.
User testing procedures: Not Available.
Ensure that foreground and background color combinations provide sufficient contrast when viewed by someone having color deficits or when viewed on a black and white screen. [Priority 2 for images, Priority 3 for text].
(See http://www.w3.org/TR/WCAG10/#tech-color-contrast and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-color-contrast)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 2.2_HTML_01 | img, area, input[@type='image'], applet, object |
Select non-text-elements. Decide whether contrast in non-text-element is high enough to convey the information |
Low |
| 2.2_HTML_02 | */@color, */@bgcolor | Select elements with attributes defining colors. Check contrast. |
Low | |
| 2.2_HTML_03 | */@link, */@vlink, */@alink, */@text |
Select elements with attributes defining colors. Check contrast. |
Low | |
| CSS | 2.2_CSS_01 | color, background-color | Select elements for which color OR background-color are defined. Check whether both color and background-color are defined. |
High |
| 2.2_CSS_02 | color, background-color | Select elements for which color and/or background-color are defined. Check color contrast. |
Low | |
| 2.2_CSS_03 | :link color, :link background-color, :link background, :visited color, :visited background-color, :visited background, :hover color, :hover background-color, :hover background, :active color, :active background-color, :active background |
Select pseudo classes for links where color and/or background color are defined. Check color contrast. |
Low |
This test is targeted to find media elements without enough color contrast.
Applicability criteria: Select the following combination of elements/attributes:
img
area
input[@type='image']
applet
object
Test procedure: Decide whether contrast in non-text-element is high enough to convey the information
Confidence level: Low.
User testing procedures: Not Available.
This test is targeted to find text without enough color contrast.
Applicability criteria: Select the following combination of elements/attributes:
*/@color
*/@bgcolor
Test procedure: Check contrast.
Confidence level: Low.
User testing procedures: Not Available.
This test is targeted to find text without enough color contrast.
Applicability criteria: Select the following combination of elements/attributes:
*/@link
*/@vlink
*/@alink
*/@text
Test procedure: Check contrast.
Confidence level: Low.
User testing procedures: Not Available.
This test is targeted to find text without enough color contrast.
Applicability criteria: Select the following combination of elements/attributes:
color
background-color
Test procedure: Check whether both color and background-color are defined.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to find text without enough color contrast.
Applicability criteria: Select the following combination of elements/attributes:
color
background-color
Test procedure: Check color contrast.
Confidence level: Low.
User testing procedures: Not Available.
This test is targeted to find text without enough color contrast.
Applicability criteria: Select the following combination of elements/attributes:
:link color
:link background-color
:link background
:visited color
:visited background-color
:visited background
:hover color
:hover background-color
:hover background
:active color
:active background-color
:active background
Test procedure: Check color contrast.
Confidence level: Low.
User testing procedures: Not Available.
“Clarify natural language usage.”
(See http://www.w3.org/TR/WCAG10/#gl-abbreviated-and-foreign)
This guideline provides information on how to facilitate pronunciation or interpretation of abbreviated or foreign text.
Clearly identify changes in the natural language of a document's text and any text equivalents (e.g., captions). [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-identify-changes and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-identify-changes)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 4.1_HTML_01 | text(), img/@alt, applet/@alt, area/@alt, input/@alt, meta/@content, option/@label, optgroup/@label, object/@standby, table/@summary, */@title (all elements except base, basefont, head, html, meta, param, script and title), input[@type='text']/@value, input[@type='submit']/@value, frame/@name, iframe/@name |
For each word, attribute value, text node and element, check if language corresponds with: a) the specified language of its nearest ancestor, b) the predominant language if no ancestor specifies language. |
High |
| 4.1_HTML_02 | text(), img/@alt, applet/@alt, area/@alt, input/@alt, meta/@content, option/@label, optgroup/@label, object/@standby, table/@summary, */@title (all elements except base, basefont, head, html, meta, param, script and title), input[@type='text']/@value, input[@type='submit']/@value, frame/@name, iframe/@name |
For each word, attribute value, text node and element, check if text direction corresponds with: a) the specified text direction of its nearest ancestor, b) the predominant text direction if no ancestor specifies text direction. |
High | |
| CSS | 4.1_CSS_01 | *:after {content: “...”;}, *:before {content: “...”;}, * { content: “...”}, * { cue: url(“...”);}, * { cue-before: url(“...”);}, * { cue-after: url(“...”);}, * { list-style-type: ...;}, * { list-style: ...;} |
Select any elements which have associated CSS rules that generate content. Check if the generated content is in a different language than the context. |
This test is targeted to find changes in natural language that are not marked up. Marking up natural language changes can make a document more accessible to multilingual users.
Applicability criteria: Select the following combination of elements/attributes:
text()
img/@alt
applet/@alt
area/@alt
input/@alt
meta/@content
option/@label
optgroup/@label
object/@standby
table/@summary
*/@title (all elements except base, basefont, head, html, meta, param, script and title)
input[@type='text']/@value
input[@type='submit']/@value
frame/@name
iframe/@name
Test procedure: For each word, attribute value, text node and element, check if language corresponds with:
If this procedure finds a change in natural language, this change should be marked up with a “lang” attribute with the corresponding language code. After this test, go to test 4.1_HTML_01.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to find changes in text direction (for natural languages) that are not marked up.
Applicability criteria: Select the following combination of elements/attributes:
text()
img/@alt
applet/@alt
area/@alt
input/@alt
meta/@content
option/@label
optgroup/@label
object/@standby
table/@summary
*/@title (all elements except base, basefont, head, html, meta, param, script and title)
input[@type='text']/@value
input[@type='submit']/@value
frame/@name
iframe/@name
Test procedure: For each word, attribute value, text node and element, check if text direction corresponds with:
If this procedure finds a change in text direction, this change should be marked up with a “dir” attribute or the “bdo” element (which requires the “dir” attribute) with the corresponding text direction code.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to analyse long descriptions of media elements.
Applicability criteria: Select the following combination of CSS rules/selectors:
*:after {content: “...”;},
*:before {content: “...”;},
* { content: “...”},
* { cue: url(“...”);},
* { cue-before: url(“...”);},
* { cue-after: url(“...”);},
* { list-style-type: ...;},
* { list-style: ...;}
Test procedure: Select any elements which have associated CSS rules that generate content. Check if the generated content is in a different language than the context.
Confidence level: High.
User testing procedures: Not Available.
“Create tables that transform gracefully.”
(See http://www.w3.org/TR/WCAG10/#gl-table-markup)
This guideline provides information on how to identify properly marked up tables,
For data tables, identify row and column headers. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-table-headers and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-table-headers)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 5.1_HTML_01 | th/@scope, th/@id, td/@headers |
Verify that the scope attribute for the table heading is set. Test passes if it is set. Otherwise, continue with verifying that id attribute in table header exists, that the headers attribute in table definition exists, and that the table definition parameters matches the table header IDs. |
High |
| 5.1_HTML_02 | th/@axis, td/@axis |
If the axis attribute is used in tables, check that it is used consistently. |
Medium | |
| 5.1_HTML_03 | table/colgroup, table/thead, table/tfoot, table/tbody |
Verify that colgroup, if present, comes first, tfoot comes before tbody. Test 5.1 fails if colgroup, thead, tfoot and tbody are used inconsistently. |
High | |
| 5.1_HTML_04 | pre | Inspect the preformatted text to see if it could have been better presented as a proper table. |
Medium | |
| CSS | N/A |
Table 7: UWEM 0.5 tests for checkpoint 5.1.
This test is targeted to find table cell elements with scope attribute or id and headers attributes that can aid assistive technology in presenting the information in the table properly.
Applicability criteria: Select the following combination of elements/attributes:
th/@scope
th/@id
td/@headers
Test procedure: Verify that the scope attribute for the table header cell is set. Test passes if it is set. Otherwise, continue with verifying that id attribute in table header cell exists, that the headers attribute in table data cell exists, and that the table cell parameters matches the table header ids.
Confidence level: High.
User testing procedures: Not Available.
This test is targeted to find table element with axis attribute that can aid assistive technology in presenting varying type specific information in the table columns properly.
Applicability criteria: Select the following combination of elements/attributes:
th/@axis
td/@axis
Test procedure: If the axis attribute is used in tables, check that it is used consistently.
Confidence level: Medium
User testing procedures: Not Available.
This test is targeted to aid assisting technology in identifying information about repeated table headers, footers and table row and column grouping.
Applicability criteria: Select the following combination of elements/attributes:
table/colgroup
table/thead
table/tfoot
table/tbody
Test procedure: Verify that colgroup, if present, comes first, tfoot comes before tbody. Test 5.1 fails if colgroup, thead, tfoot and tbody are used inconsistently.
Confidence level: High
User testing procedures: Not Available.
This test is targeted to identify preformatted text used instead of proper tables.
Applicability criteria: Select the following combination of elements/attributes:
pre
Test procedure: Inspect the preformatted text to see if it could have been better presented as a proper table.
Confidence level: Medium
User testing procedures: Not Available.
For data tables that have two or more logical levels of row or column headers, use markup to associate data cells and header cells. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-table-structure and the techniques in http://www.w3.org/TR/WCAG10-HTML-TECHS/#identifying-table-rows-columns)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 5.2_HTML_01 | table | If the table contains more than two logical levels of row or column headers, verify that the table marks up the rows and columns properly by using test 5.1. If this test fails, or returns CannotTell/ NotApplicable, then test 5.2 fails. |
High |
| CSS | N/A |
This test is targeted to identify tables with more than two logical levels of rows or columns that are not marked up properly by using table markup that accosiates rows and columns and can aid assistive technologies in presenting the tables properly.
Applicability criteria: Select the following combination of elements/attributes:
table
Test procedure: If the table contains more than two logical levels of row or column headers, then verify that the table marks up the rows and columns properly by using test 5.1. If this test fails, or returns CannotTell/ NotApplicable, then test 5.2 fails.
Confidence level: Medium
User testing procedures: Not Available.
“Ensure that pages featuring new technologies transform gracefully.”
(See http://www.w3.org/TR/WCAG10/#gl-new-technologies)
This guideline provides information on ensuring that pages are accessible even when newer technologies are not supported or are turned off.
Organize documents so they may be read without style sheets. For example, when an HTML document is rendered without associated style sheets, it must still be possible to read the document. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-order-style-sheets and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-order-style-sheets)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 6.1_HTML_01 | link[@rel='stylesheet'], style, */@style |
Deactivate all CSS applied to document. Ensure that:
|
High |
| 6.1_HTML_02 | script, */@onfocus, */@onblur, */@onkeypress, */@onkeydown, */@onkeyup, */@onsubmit, */@onreset, */@onselect, */@onchange (*/@onload, */@onunload, */@onclick, */@ondblclick, */@onmousedown, */@onmouseup, */@onmouseover, */@onmousemove, */@onmouseout) |
Deactivate all CSS applied to document by script. Ensure that:
|
High | |
| CSS | N/A |
This test analyses the effect on the readability of the document of CSS applied in standalone stylesheets, embedded stylesheets and style attributes of document elements.
Applicability criteria: Select the following combination of elements/attributes:
link[@rel='stylesheet']
style
*/@style
Test procedure: Deactivate all CSS applied to document declaratively. Ensure that content can be read; content does not become invisible; content is not obscured by other content; the intended reading order is maintained.
Confidence level: High.
User testing procedures: Not Available.
This test analyses the effect on the readability of the document of CSS applied programmatically.
Applicability criteria: Select the following combination of elements/attributes:
script,
*/@onfocus
*/@onblur
*/@onkeypress
*/@onkeydown
*/@onkeyup
*/@onsubmit
*/@onreset
*/@onselect
*/@onchange
(*/@onload
*/@onunload
*/@onclick
*/@ondblclick
*/@onmousedown
*/@onmouseup
*/@onmouseover
*/@onmousemove
*/@onmouseout)
Test procedure: Deactivate all script in standalone script files applied to the content, script in script blocks and in event handler attributes. Ensure that content can be read; content does not become invisible; content is not obscured by other content; the intended reading order is maintained. Check that on the ocurrence of actions that would otherwise trigger events, content is still readable when listeners are turned off.
Confidence level: High.
User testing procedures: Not Available.
For this checkpoint there are no applicable tests.
Ensure that equivalents for dynamic content are updated when the dynamic content changes. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-dynamic-source and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-dynamic-source)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 6.2_HTML_01 | frame/@src | Check that the content pointed to by the src attribute complies with checkpoint 1.1. |
High |
| 6.2_HTML_02 | frame, a/@href, script, */@onfocus, */@onblur, */@onkeypress, */@onkeydown, */@onkeyup, */@onsubmit, */@onreset, */@onselect, */@onchange (*/@onload, */@onunload, */@onclick, */@ondblclick, */@onmousedown, */@onmouseup, */@onmouseover, */@onmousemove, */@onmouseout) |
Search for links, script or other elements in the delivery unit that cause other content to be loaded into the frame. Ensure that content loaded into the frame complies with checkpoint 1.1. |
High | |
| CSS | N/A |
This test analyses the text equivalent of any non-text content loaded into the frame by the browser as a result of the value the src attribute.
Applicability criteria: Select the following combination of elements/attributes:
frame/@src
Test procedure: Check that the content pointed to by the src attribute complies with checkpoint 1.1.
Confidence level: High.
User testing procedures: Not Available.
This test analyses the text equivalent of any non-text content loaded into the frame by the browser as a result of link activation or script execution.
Applicability criteria: Select the following combination of elements/attributes:
script
a/@href,
*/@onfocus
*/@onblur
*/@onkeypress
*/@onkeydown
*/@onkeyup
*/@onsubmit
*/@onreset
*/@onselect
*/@onchange
(*/@onload
*/@onunload
*/@onclick
*/@ondblclick
*/@onmousedown
*/@onmouseup
*/@onmouseover
*/@onmousemove
*/@onmouseout)
Test procedure: Search for links, script or other elements in the delivery unit that cause other content to be loaded into the frame. Ensure that content loaded into the frame complies with checkpoint 1.1.
Confidence level: High.
User testing procedures: Not Available.
For this checkpoint there are no applicable tests.
Ensure that pages are usable when scripts, applets, or other programmatic objects are turned off or not supported. If this is not possible, provide equivalent information on an alternative accessible page. [Priority 1]
(See http://www.w3.org/TR/WCAG10/#tech-scripts and the techniques in http://www.w3.org/TR/WAI-WEBCONTENT-TECHS/#tech-scripts)
| Format | Test ID | Elements/ Attributes/ Selectors |
Inspection Procedure | Conf. Level |
|---|---|---|---|---|
| HTML | 6.3_HTML_01 | object, applet, embed | Ensure that all information and functionality provided by the embedded objects is available when these are not loaded or do not function as intended. |
High |
| 6.3_HTML_02 | script, a[starts-with(@href, 'javascript:')], */@onfocus, */@onblur, */@onkeypress, */@onkeydown, */@onkeyup, */@onsubmit, */@onreset, */@onselect, */@onchange (*/@onload, */@onunload, */@onclick, */@ondblclick, */@onmousedown, */@onmouseup, */@onmouseover, */@onmousemove, */@onmouseout) | Ensure that all information and functionality provided by the script is available when this is not executed. |
High | |
| CSS | N/A |
This test determines whether information and functionality provided by embedded content is also available without said content.
Applicability criteria: Select the following combination