Quality Assurance in the context of EUROPA encompasses a set of activities whose aim is to ensure that EUROPA web sites are compliant with the external and internal (EUROPA-specific and project-specific) standards in force. In order to be effective, Quality assurance activities must be integrated into the entire site development process, starting from the conception stage right up to the site’s final transition into production (going “on-line”); Quality cannot be added to a project “after the fact” or just at the final stages.
There are several closely related concepts classed within the subject of Quality in the context of EUROPA projects. First, some definitions of these closely related terms:
Quality Assurance is process (or set of processes) of enforcing quality control standards by applying the planned, systematic quality activities and working to improve the processes that are used in producing the web sites and its components, infrastructure and content.
Quality assurance examines the processes of site implementation from inputs to output.
Quality control involves the formal and systematic use of testing to measure the achievements of a specified standards and recommendations; the measurement and enforcement of defined level of standards.
Testing is the process of verifying correct functioning of an Information System or a web site. This verification involves designing, possibly developing, executing and evaluating the results of tests. During testing you compare the site to the standard, recording deviations and shortcomings.
Validation and Compliance
“Quality” is measured by the degree of compliance with the aforementioned standards and other recommendations. Full compliance is the general aim. Validation refers to the process of verifying the compliance to all these standards and other recommendations, and is a mandatory activity. These various verifications take place at each of the appropriate phases of the project.
The mandatory standards (which are applicable in all cases) include:
In addition, a number of other standards and/or guidelines may be applicable, depending on the site or other factors, such as:
Other specific “functional” validations may also be necessary, such as:
Some QA controls are conducted in the form of tests. There are a variety of tests to be applied as a part of the QA of a web site. These are described in more detail: (subsection)
Quality control is a continual process that is applied in the appropriate way at each stage of any given project. Most importantly, checklists have to be used before a site goes into production.
Each project phase has specific Quality Assurance activities (such as the application of standard procedure or controls) which are carried out during that phase. The set of project phases can therfore be considered as the top level "steps" of the activity. See the section on workflow for a more detailed description of these phases.
1) Plan phase
3) Design phase
4) Build phase
A detailed description of the quality-related aspects of these top-level steps (for example, “check against presentation standards” for the “Design” phase, etc.) can be found in the respective activity descriptions. A separate activity sheet is defined for each of these quality-related steps which will be referenced by the appropriate related project activity, in sequence.
This section provides a walkthrough of the process of site implementation with a stress of different quality checkpoints that should be considered to assure that the site is compliant to standards.
Details concerning the quality-related aspects of these top-level steps can be found in the respective activity descriptions. A separate activity sheet is defined for each of these quality-related steps which will be referenced by the appropriate related project activity.
For each project phase, the principal focus of the QA activities is described.
1) Definition phase
During this phase the quality objectives are defined in terms of the specific sites characteristics; standard QA activities which are applicable to all projects are applied here; the purpose is to ensure the proper setup of the project and proper identification of the specific quality objectives for the site. These objectives can guide later QA, QC and testing activities
2) Content Phase
3) Design Phase
4) Development Phase
5) Publication Phase
6) Maintenance Phase
a. PLAN : establish the objectives and processes necessary
b. DO – implement the processes
c. CHECK – monitor and evaluate
d. ACT – apply actions for necessary improvements
The chances of success for any site development project are greatly enhanced if it starts on a good basis. Establishment of a clear Project definition and good-quality planning are an important part of this.
The QA activities of this phase are focused on verifications of the use of proper Project Management procedures and actions, and definition and verifications of the outputs, and as such are rather project specific.
Proper project management of any site development project will include mandatory execution of the following steps, and verification of the results of each step:
The objectives of this phase concern the quality of the content of the site.
The QA activities in this phase are concerned with content definition, presentation and structure, including the architecture of your information (and its retrieval) for your audience; this includes the site’s overall structure, navigation mans, search functions, site index, table of contents, URL’s, etc.)
A formal definition of the requirements for and specifications of the content should be produced.
Writing for the web is different from writing for printed publications. Make sure the site content complies with the Writing for the web guidelines, especially:
The editing quality of a site is not confined to its original language. It is important to check the quality in other language versions, in particular to verify that the integrity of texts received has not been altered in the course of HTML formatting etc.
The Quality assurance activities in this phase are focused on:
Look for the following things:
In judging the content ask yourself:
The objective of QA activities in this phase focus on the presentation aspects of the site in question. In general, online publications must meet the same quality criteria as documents published on paper. Disseminating information on the Internet has several advantages, such as:
Quality control should therefore be even more stringent and regular, both at technical and editing levels.
One of the important QA activities with respect to design involves the use of the checklist for the presentation/graphic design aspects of the site in question, as follows.
In addition to the general principle of ensuring that the designer or contractor has access to the latest version of IPG, it must also be ensured that these following rules are respected:
The use of the template covers these aspects:
If you deem it necessary to depart from the IPG rules in some way, discuss this with the Europa team before developing your new site.
During this project phase the various program elements of the site are created and tested. It is a particularly important activity for dynamic sites, which can entail significant programming. Quality control activities during this phase are more technical than in other phases, and include internal functional testing of dynamic site functionality (unit testing).
Testing is a type of formal validation in which specific results are sought and the actual results obtained are recorded. The purpose is to identify shortcomings, errors or other non-compliance, and hence any necessary corrective actions.
For quality control to be effective, you must test the same things the same way every time you test. This is best achieved via production of a test plan. The test plan is a description of the specific areas (functionality elements, etc.) to be tested, the specific tests to be run (and the results expected in each case), how frequently you will test them, and where in the development or publication process you will test them.
Following is a list of some of the different types of tests that may need to be performed:
The objective of the QA activities during the maintenance phase is to ensure that a continuous process of evaluation and improvement is being followed.
The decisions made as to the content, form and frequency of feedback to be gathered from users of the site, and how this information is to be used to modify or update the site are implemented during the maintenance phase. For some sites, there will be a natural schedule for updating, improvement or extension of the site, such as sites which are concerned with specific external events, or for which the content is particularly short lifetime of validity.
The main activities cover the systematic monitoring and evaluation of defined quality indicators (such as the results of different means of feedback to measure site effectiveness, etc.). Special emphasis is placed on the cyclic and periodic, regular nature of the process of improvements.
There are two types of QA processes involved in the maintenance phase:
This self-assessment checklist may help you to measure how well your website meets IPG requirements and best practices. The checklist is based on the standards in force and recommendations of this website.
You may find useful this quick checklist for successful websites to help identify whether a site is focused or not on its users.
|Accessibility of the information||The site is well referenced in search engines and indexes.|
|Various entry points to the information are proposed (thematic, alphabetical, chronological pages, etc.).|
|The information is presented in a coherent manner on all pages of the site.|
|The information is not duplicated and preference is given to putting links to the source sites (i.e, any other site of EUROPA being the leader for the concerned matter). Links to sites dealing with matters of the same or similar themes are present.|
|Various information entries are presented (i.e. contact page, about this site, site map, site index, etc).|
|Accessibility to the information and editorial quality||The page is easy to read and use the appropriate style of writing to match the content and the audience|
|The rules for writing for the web are applied.|
|Accessibility to the information for disabled or differently equipped people - WAI initiative||The technical specifications needed to reach the conformity AA of the WCAG 2.0 are implemented on the site and each page of it.|
|The page is WCAG 2.0, level AA compliant.|
|Accessibility statement||A page explaining the accessibility policy of the site and, in case of justified technical or practical reasons for not complying with WCAG 2.0, level AA guidelines, stating the exceptions is well presented on the site.|
|Architecture||The URL is within the europa.eu domain and has been approved by the webmaster and the EUROPA Team.|
|The folders and pages are named in lower cases, without space or special characters. All the names are in English.|
|The site is organised by theme and not by language, the linguistical character of the pages being put in their filenames (ending with _xx, xx being the appropriate ISO 639 code).|
|A clear and coherent organisation of the site and of its tree is ensured in order to make it manageable by more than one person.|
|Apart from the projects in development, the site's content is the same on both staging and production server, with no cross-over between the servers. A particular attention is thus given to delete obsolete and orphan files on both servers. The folders having received an automatic redirection instruction are also empty on both servers.|
|The default multilingual homepage is well present in the site (index.htm or index.html).|
|Archiving||The sites having a real historical, legal or informative value are archived. They display the standard archive stamp and host under the /archive folder. See also "Archiving".|
|Banner||See under "Template - Banner".|
|Colours for text||The colours fixed in the central CSS are respected.|
|Colours in the HTML code||The colours are defined in hexadecimal ("#nnnnnn") instead of names.|
|Colours palette||If possible, only the colours of a browser safe palette are used.|
|Compressed files - File format||Use ZIP format. These files contain only authorised formats.|
|Contact - Compulsory link||See under "Template - Service tools".|
|Cookies||If cookies are stored longer than current session, explicit permission is asked for using the cookie consent kit.|
|Cookies - Compulsory link||See under "Template - Service tools".|
|Copyright and disclaimer||
All third party material published has the necessary copyrights.
|Creation of new site or substantial modifications||The EUROPA team has been informed.|
|The information is present in the central What's New page of EUROPA.|
|The readers are informed (various media used).|
|CSS||The central CSS are used and these instructions are not over-ridden by specific ones.|
|The correctness of the CSS file is well validated (online or downloadable validator).|
|Deletion or change of URL of sites or substantial parts of site||The EUROPA team is informed.|
|The readers are informed (various media used).|
|A redirection system is present.|
|DOCTYPE||See under "HTML validation".|
|Download - File formats||Pure text files (.txt) files, where information is not available in a richer format are allowed. Other formats can also be used but the standard format (pdf or html) must also be provided. In that case, both the most popular format (MS Office) and the Open Document Format (ODF) must be offered.|
|The format and size are announced next to the link.|
|Editorial quality||See chapter Writing for the web. See also "Accessibility to the information" and "Up-to-date information".|
|E-Mail address||The functional email address is regularly checked (good state of working).|
|The emails are permanently read (no "auto-reply") and receive a quick and appropriate answer.|
|Entertainment files - File format||Flash and Shockwave formats are allowed.|
|Entertainment files - Bypass||A by-pass ("skip") for entertainment files is always proposed in the page.|
|Entertainment files - Plug-in||Provide a link to download the plug-in.|
|Entertainment files - Content alternative||A link to a HTML page describing the content of the file is present.|
|File format||For specific types of files, see under:
- Compressed files
- Entertainment files
- Proprietary formats
|Flash||See "Entertainment files".|
|Frames||If they are well authorised by the EUROPA team, the frames have a NOFRAMES alternative.|
|Graphical design||The page respects the presentation as proposed in the templates.|
|The page is displayed without horizontal scrolling on a 1024x764 pixels screen.|
|HTML - Declaration||The inserted HTML declaration (!DOCTYPE) is correct and appropriate.|
|HTML - Validation||The correctness of the HTML code is well validated according to the HTML 4.01 transitional standard.|
|Images as background||There is no use of large images for backgrounds.|
|Images - Content alternative||Complex images such as graphics and charts provide a link to a page describing the content of the image.
The link can be proposed via the LONGDESC attribute.
|Images - File format||
Generally, .GIF or .JPG. PNG is also allowed but only when it is really needed (special transparency).
|Images - HTML tag attributes||All required attributes are present (width, height, alt).|
|Images - HTML tag attributes - ALT attribute||The ALT attribute gives the appropriate information in the language of the page.
- If the image is decorative, use an empty ALT
- If the image is functional or a link (like a print button), give a text alternative of its function (i.e. print this page),
See also under "Images - Content alternative".
|Images - Links||See under "Links via images".|
|Images - Size||The images' size is as small as possible. It is an acceptable size for an acceptable download time.|
|Images - Source||The site contains no duplication of central images.|
|Images - Transparency||If needed, the PNG transparency is ensured in order to avoid a degradation of the graphical display of the page.|
|Image maps - Links||See under "Links via image maps".|
This notice is present at the top of each HTML page.
|Linguistic navigation||Present on all pages, even if only 1 language available.|
|The same level of navigation is ensured, i.e. there is an equivalence in the content of the various linguistical versions proposed.|
|Links - Confusion||As it is forbidden, any underlined text without proposing any link is absent.|
|Links - Quality||The validity of the links is checked.|
|The links always point to the source site (i.e, any other site of EUROPA being the leader for the concerned matter) as the duplication of the source information is forbidden.|
|In general links should only point to pages without access restriction. A page may only contain links to restricted sites when it is aimed at a limited audience. In that case the restriction should be clearly announced next to the link|
|The following rule is applied:
- absolute URLs are used only for links to other web sites than EUROPA,
- semi-absolute URLs to root are used for links to other sites of EUROPA.
- relative URLs are used for navigation inside the same site,
|See also "Up-to-date information"|
|Links via images||The image has no border and proposes a description of the target in its ALT attribute. If different, the language of the target is also indicated in the ALT attribute.|
|Links via image maps||The image has no border and proposes a general description in its ALT attribute. A textual alternative of its content is also present.
Each area has it own ALT attribute proposing a description of the target and, if necessary, indicates the language of the target.
|Links via text||The general rule recommending the underlining of the text proposing a link is applied.
In case of non-respect of the general recommendation, it means that one of the 3 following situation is present:
- authorised exception for well recognisable lists or navigation bars,
- authorized exception if the link is clearly announced in an alternative way,
- compulsory exception for links pointing to another language: the text must then not be a link, but must be followed by the appropriate linguistic navigation images with correct ALT attribute.
|Mail forms||The destination mailbox is regularly checked (good state of working).
A reply is quickly sent to the reader.
Presence of a warning in case of collection of personal data
|Metadata - Coding of text - HTML pages||The coding of text is clearly defined in the meta tag which contains the appropriate UTF-8 code of the character used.|
|Metadata "Reference" - HTML pages||Its content is in capital letters, each section separated by a "/".|
|Metadata "Creator" - HTML & PDF pages||Its content is in capital letters, each section being separated by a "/".|
|Metadata "Language" - HTML & PDF pages||Its content is composed of 2 lower-case letters as determined in the ISO 639 codes.|
|Metadata "Classification" - HTML pages||Its content is the appropriate code of 2 numbers, as proposed in the list.|
|Metadata "Keywords" - HTML & PDF pages||Its content is in the language of the page, upper case & accented characters can be used, each word being separated by a comma (without comma, the words are considered as an expression).
The 3 compulsory keywords are present. The other keywords are significant words describing the document.
|Metadata "Description" - HTML & PDF pages||Its content is 1 or 2 sentences describing the context and the content of the page (of the site for the homepages). It is written in the language of the page and it is not longer than 150 characters - ". See also the recommendations in Metadata - Instructions|
|Metadata "Date" - HTML pages||Date of creation of the page. The creation date together with the date of last update on the template footer is used by the Europa search engine to determine the freshness of the page.|
|Multilingualism||General, official or stable information is available in the 24 languages.|
|The page may not be a mix of various languages. Each page is thus unilingual.
A multilingual "news" section in a page is nevertheless accepted.
|The quality of all the linguistic versions of the page is checked.|
|Multilingualism - Language policy||Explain the language policy of the website in the "About this site" page or in a dedicated page.|
|Navigability||The navigation scheme and labels are consistent across all the website.|
|It is easy to navigate from the page to related ones, to other sections of the site and to other sites of EUROPA.|
|Navigation path ("Page path")||See under "Template - Navigation path".|
|PDF files||Use only PDF for long documents that are intended for printing, legally restricted documents, claim and application forms for downloading and printing and documents designed specifically for print.|
|Create an HTML summary/abstract whenever possible.|
|Use a table of contents for long PDF documents.|
|Add appropriate metadata / document properties including the language.|
|Add structure (heading, lists, etc) and alt text in the source document creation application. Test and fix tags using Acrobat Professional.|
|Optimise PDF files to get a smaller size and to deliver the document page by page.|
|Try to keep the size of the PDF files under 2 MB.|
|Describe in the link that the document is a PDF and state its size in KB.|
|Plug-ins||No non-standard plug-ins are used.|
|An alternative solution to access content is provided for those internet users not having the plug-in.|
|Proprietary formats||Proprietary formats are admitted if agreed by the EUROPA team and if alternative format is proposed next to the link.|
|See also under "Download - File formats" and "Compressed files - File format"|
|Search - Compulsory link||See under "Template - Service tools".|
|Sound - Content alternative||A link to a page describing the content of the sound is present.|
|Sound - File format||MP3 is the prefered format. Other formats are allowed: WAV, MIDI|
|Tables||Tables are not used for layout purposes.|
|Tables are expressed in percentages or are not larger than 760 pixels (standard 1024x768 screen with horizontal scrollbar).|
|The configuration tags contain the necessary attributes giving the appropriate information and make thus the tables WAI compliant.|
|Template - Banner||The logo of the European Commission, its colours and shape is respected|
|A banner, in the language of the page, is present on each page.|
|There is no link on the banner and the DG name, if mentioned, is no abbreviation.|
|The standard size is respected.|
|Template - Title||See under Title|
|Template - Service tools - Optional links||These tools are corresponding to what the templates recommend and, if necessary, the links are adapted.|
|Template - Service tools- Compulsory links||
Contact link is included in each page and points to a functional (and not personal) e-mail address or to the site's contact page.
|Search link is included in all pages. The link points to the EUROPA search page (in the same language) or the specific search facility of the site.|
|Cookies link is included in all pages. The link points to a local cookie notice page informing the user about the cookies used on the specific site.|
|Template - Legal notice||
|Template - Linguistic navigation||Present on all pages, even if only 1 language available.|
|The same level of navigation is ensured, i.e. there is equivalence in the content of the various linguistic versions proposed.|
|Template - Navigation path||This navigation bar is included in each page and the inserted links point to pages in the same language.|
|Template - Date of the latest update||This information is well present on all pages|
|Template - Top link||This feature is present at the bottom of all pages. When pages are longer than two vertical screens a top link is introduced between paragraphs.|
|Text - Coding declaration||See under "Metadata - Coding of text - HTML pages".|
|Text - Encoding||The correct character as specified in the UTF-8 code fixed in the page is used and the use of its HTML equivalent is avoided.|
|Text - File format||HTML or optimised PDF
The .TXT format is used when it's not available in rich format.
|Text - Underlined||This is refused if the text does not propose any link (see "Links confusion").|
|Text font||Verdana is used for all typographical elements. The "sans-serif" font is proposed as the minimum alternative in the CSS.|
|Text font - Content||The fonts are not embedded.|
|Text pages - Size||The page has an appropriate size ensuring an acceptable download time.|
|Title (<title>)||The title is present and adapted on each pages. It's in the language of the page.|
|The title is coherent, useful, useable and informative.|
|The recommended structure (EUROPA - Page title for europa.eu domain or Page title - European Commission for ec.europa.eu) and length (maximum 70 characters) are applied.|
|Up-to-date information||The information furnished is regularly checked.|
|The consistency of information in the various linguistic versions of a page is ensured.|
|Obsolete or orphan pages are deleted from both staging and production servers.
If necessary, a redirection system is installed.
|Up-to-date information - Archiving||The sites having a real historical, legal or informative value are archived better than deleted from the site.|
Measure user satisfaction using an online survey preferably once per year.
Names of directories and files are in conformity with the IPG rules (in English, typography, linguistic attributes).
|Video - Content Alternative||A link to a page describing the content of the video is present.|
|Video - File format||MP4 is the recommended format. Other formats can be used: FLV, AVI, MPEG, MOV, RM, WMV.|
The efficient execution of quality control activities can be greatly enhanced by application or use of the appropriate tools.
The webmasters toolbox provides several tools to check the quality of your pages.
The following are recommended:
HTML validation - to check the correctness of the document against declared DOCTYPE
CSS validation – checking the validity of external style sheets called by your document.
Validating Accessibility - measuring how easy the site is to navigate and understand by users with different interactivity impairments, such as blind users (using screen readers) or mobility impaired users (using only keyboard commands)
Cross-browsers compatibility tools – measuring how well the site supports different browser applications and different operating systems.
Links validating – verification of links integrity
Website performance - check how long it takes to download a page
Multiple validators - checking HTML, CSS, accessibility, spell checker, broken links, screenshots, SEO, social services, etc.