Navigation path

Tools

Table of content


1. Our tools

This chapter presents information about the technical aspects of the EUROPA web sites.

Discover many tools that are available to make (ec.)europa.eu web sites:

Use quality control tools and services and read about the use of third-party tools and services.

There are some other tools and services available for use on EUROPA:

Top

1.1. Corporate Web Content Management

WCM Login

For information on product management, see CWCM Portal.

The Corporate Web Content Management (CWCM) helps to manage and automate the creation of Europa site. It proposes

  • A decentralised approach allowing web content management for everyone :
    • Access rights to contents and user's roles are defined at site level by site owners: anyone registered in LDAP and having access to NET1 can play a role in any site. The communication between the so-defined actors can be managed by workflows.
    • No technical skill needed to be a EUROPA information provider: by clearly separating presentation (XSL) and content (XML), the CWCM allows the information providers to produce web pages without having to care about technical formatting details. The system is able to dynamically and automatically assemble basic pieces of content into well-formatted web pages according to predefined models.
    • No specific hardware/software needed: access to repository is done through a web based application. A browser is enough.
  • A mixed approach to web content presentation:
    • The presentation of a site is defined (using XSL) and managed by site owner.
    • Europa corporate presentation ensured by availibility of common presentation elements.
    • Contents are stored inside a central repository and are queryable by everyone allowing easy cross-referencing between sites.
  • A centralised approach to web content publication: the flows between the various technical elements used for the publication of the web contents are managed inside the tool.

Version in use on EUROPA

WebPublisher has been customized (CWCM project).

Efficient use of Documentum requires its customization to the standards of the Commission. The CWCM creates this customized environment by defining a basic set of content definitions common to all Commission websites, and standard templates to present that content.

See also CWCM releases.

Guidelines and references

You can find guidelines about the Corporate Web Content Management System(CWCMS) in the CWCMS Portal.

Trainings

Please consult Syslog, the training information system, which gives you access to the training catalogue, the training map and allows you to introduce your application for a training course.

Top

1.2. Application servers

An application server is a server-based software system which hosts applications providing functionality to client machines. It provides a hosting environment for the execution of server-based applications. It also accelerates the development of applications by providing them with a standardised framework of application building blocks and support services. An application server can be used to host complex Web-based applications such as Content Management Systems (CMS), online payment systems and gaming applications.

References:

 

Top

1.2.1. ColdFusion

Coldfusion is a commercial product developed by Adobe Systems.

Description

The name Coldfusion covers actually two things: Coldfusion is a technical markup language, CFML (Coldfusion Markup Language), similar to HTML, aimed to describe dynamic content and Coldfusion is also an Application Server, a software engine, able to interpret and execute the Coldfusion markup language.

Main features

  • Simplified database access
  • Client and server cache management
  • Client-side code generation, especially for form widgets and validation
  • Conversion from HTML to PDF and FlashPaper
  • Data retrieval from common enterprise systems such as Active Directory, LDAP, SMTP, POP, HTTP, FTP, Microsoft Exchange Server and common data formats such as RSS and Atom
  • File indexing and searching service based on Verity K2
  • GUI administration
  • Server, application, client, session, and request scopes
  • XML parsing, querying (XPath), validation and transformation (XSLT)
  • Server clustering
  • Task scheduling
  • Graphing and reporting
  • Simplified file manipulation including raster graphics (and CAPTCHA) and zip archives (introduction of video manipulation is planned in a future release)
  • Simplified web service implementation

Versions in use on EUROPA

  • ColdFusion 8 and 9

 

Guidelines and references

As (most of)  Europa is hosted by the Data Centre (DIGIT), guidelines for hosting Coldfusion applications at the Data Centre apply for Europa Coldfusion driven sub-sites.

 

Training

Please consult Syslog , the training information system, which gives you access to the training catalogue, the training map and allows you to introduce your application for a training course.

How to request a ColdFusion hosting ?

You have to fill up a form in Mirella as detailed on our page "Request a dynamic application".

Top

1.2.2. JAVA platform

Following the success of the Java platform, the term java application server sometimes refers to a J2EE or Java EE application server.

Description

Some of the better-known Java Enterprise Edition application servers include:

  • WebLogic Server (Oracle)
  • Apache Tomcat (Apache Software Foundation)
  • WebSphere Application Server and WebSphere Application Server Community Edition (IBM)
  • JBoss (Red Hat)
  • SAP Netweaver AS (ABAP/Java) (SAP)
  • etc.

Versions in use on EUROPA

  • WebLogic 11g SP1

 

Guidelines and references

Training

Please consult Syslog , the training information system, which gives you access to the training catalogue, the training map and allows you to introduce your application for a training course.

How to request a JAVA platform hosting ?

You have to fill up a form in Mirella as detailed on our page  "Request a dynamic application".

Top

1.3. Databases

A Database is a collection of data organised in a structured way. Databases are used to store and retrieve important amounts of data. They are managed with Database Management Systems (DBMS).

Three main database types exist: hierarchical, relational (RDBMS) and object oriented databases (ODBMS). There difference resides in the way they establish relations among data. The most common database type for web applications is relational database.

 

Top

1.3.1. Oracle Database

An Oracle Database is a relational database management system (RDBMS) produced and marketed by Oracle Corporation.

Description

Oracle stores data logically in the form of tablespaces (containing tables, indexes, etc.) and physically in the form of data files. Stored procedures and functions can be stored and executed within the database. These can be developed in SQL (link to SQL template), PL/SQL (link template PL/SQL), or in the object-oriented language Java.

Versions in use on EUROPA

  • Oracle 10.2 and 9.2

 

Guidelines and references

Most databases are hosted at the DC and, therefore, the webmaster or IRM need not worry about their creation or administration. On the other hand, he/she must enter a creation request using the Mirella application and it may take several weeks for the database to be created and made available for use.
For more information consult the following guidelines:

 

Training

Please consult Syslog , the training information system, which gives you access to the training catalogue, the training map and allows you to introduce your application for a training course.

How to request an Oracle Database hosting ?

You have to fill up a form in Mirella.

Top

1.4. Transfer tools

In order to publish your files on the EUROPA server you need a transfer tool. There are 3 main types:

Those such as FileZilla, WS FTP Pro, etc... adapted to transfer :

  • Many local files/folders,
  • A variety of files/folders,
  • No immediate production.

Information to set up your FTP tool:

  • Host name/address : doteu.staging.cc.cec.eu.int
  • Host type : UNIX
  • UserID : CUD-login
  • Password : central pass words - internet
  • Path to the subsite/s

The EUROPA  team at COMM A5 is in charge of granting access rights to the static EUROPA server. The central management programme Staging Manager accessible from the IE navigator, is adapted for:

  • A limited number of files/folders or a compressed file (format .zip),
  • The automatic creation of .request files...,
  • Start immediate production.

This tool is accessible at http://doteu.staging.cc.cec.eu.int:8888/

Every procedure that uses command systems or IT programmes that support the FTP transfer instructions is adapted for intensive or regular internal production (e.g. news).

This method needs to be able to perform the following operations:

  • The selection and FTP transfer of files,
  • The creation and transfer of the .request update file,
  • [The http: command: URL, login, password for immediate production ]

An old procedure that had been installed by the Data Centre is still being used for a limited number of sites. This is due to disappear in favour of the latest solution.

Top

1.4.1. Staging Manager

The Staging Manager tool http://doteu.staging.cc.cec.eu.int:8888/ offers DGs all the functions they need to manage static sites on EUROPA:

  • FTP: transfer/delete their sites' files/directories
  • checking
  • preparation
  • release of updates
  • monitoring of update transfer to the production environment
  • synchronisation of staging and production sites. 

It is intended for all EUROPA site managers, including external contractors (contractors should see External access.) 

Description

Built by Datacentre, the programme uses only the HTTP protocol to connect to the server and works only with the standard Commission browser (MS Internet Explorer 7 and higher) - it does not work with other browsers.

The Staging Manager does not enable you to load more than one local file to the server staging without individual selection: this tool is therefore best used for small numbers of files. To transfer larger numbers of files, Staging Manager enables you to use ZIP files, with automatic unzipping.

Traditional FTP tools are still the suggested alternative for large-scale transfers of local files to the staging environment.

The Staging Manager also compares files and indexes between the staging and production environments, enabling each manager to synchronise and tidy up their contents.

Read and write authorisations are identical to the ftp access allocated to webmasters in their individual staging environment: depending on the specific rights they have, they can write to the file request.update or request.delete and launch a production run (mise en production).

Please follow the procedure Access for transfers to static sites for new access authorisations or changes to existing access authorisations.

Production runs are processed in parallel (up to 20 at a time), to speed up the process and limit queuing. 

Sites are put online in 3 steps:

  • new files are copied from the local or development site to the staging server;
  • new files are verified on the staging server;
  • once verified, the files are copied to the production server.

The transfer to the production server applies to the files/directories on the staging server that are indicated in the .request.update or .request.delete folders placed at the root of the site in order to be copied or deleted on the production server.

It is possible in an emergency to immediately copy files/directories to a production server by using the Staging Manager's save and execute function.

Procedure description

Four special files are being used by the staging to production transfer procedure (also know as MEP). For each 'subsite' these files have to be located in the root directory of that 'subsite' on the staging server.
All file names listed inside these special files are relative to the root directory of the subsite they apply to (metacharacters "* or ." are not allowed). This should be clarified with an example below.
The special files used by the MEP (Mise En Production) update procedure and at any time are likely to exist within each subsite document root directory of the staging server are the following:

  • .request.delete lists all files and/or directories that are to be deleted from the production server.
  • .request.update lists files and/or directories that have to be added or updated from the staging server to the production server.
    • Files are copied only if a newer version exists on the staging server (based on the last modification date).
    • Directories include recursively all subdirectories. All files found in the production directories that do not have a counterpart on the staging site any longer will be deleted from the production site!
    • "<ALL>" can be specified as the only entry in order to copy the whole hierarchy. Even if this is the easiest way to proceed, this must be avoided as much as possible, because it implies a much longer processing time (<ALL> really does mean ALL).
    • The files are first copied to a temporary environment where checks are made. Missing files or files containing errors (absolute links, etc.) might stop the procedure and none of the files will be copied to the production server at all.
  • .request.lock is created at the start of the automatic procedure meaning that no one should modify any of the files from the staging directory until this file has been removed by the update procedure.
  • .request.output contains the results of the procedure. All the files which have been deleted and copied are listed. Error and warning messages can be found.

Important remarks:

  • The '.request.update' and '.request.delete' files should be created only after all modifications on the staging site have been completed, because the presence of these files means that the listed data are ready to be copied.
  • The presence of '.request.lock' file means that the procedure has started and has not completed yet. No one should modify any of the transferred data on the staging server until this file has been removed by the procedure.
  • '.request.delete', '.request.update' and '.request.lock' are always deleted at the end of the procedure.
  • '.request.output' contains only the result of the latest execution of the procedure. This file is overwritten at each execution, so the preceding results will be lost. If you want to keep a history of your modifications, then you will need to save this file under another name. The contents of the .request.output can be sent to automatically to the relevant webmasters by email.

Scheduled or immediate production

Files and directories are automatically copied to the production server everyday between 13.00 and 22.00.
In certain circumstances, the webmaster can ask for this timeframe to be changed by emailing COMM EUROPA MANAGEMENT.

The '.request', etc. files must be ready and correct prior to the scheduled move to the production server!

It is possible in an emergency to immediately copy files/directories to a production server by using the Staging Manager's save and execute function.

Transfers to production servers use significant amounts of resources and reduce server performance. 

N.B. Because of the reverse proxy's caching mechanism, it may take up to 15 minutes before newly copied html pages are displayed, slowing down the operation! If necessary, you can force a refresh of the proxy with Firefox pressing ctrl + shift - reload.

As a reminder, the reverse proxy makes possible faster access to EUROPA by memorising all the static pages (with the exception of PDFs). The refresh algorithm works like this:

  • < 15 min. for html pages
  • = 8 hours for static files (images, css, js, etc.)

Responsibility for transfers to production servers

 Managers authorised to carry out transfers to a production server must agree to:

  • Check the pages on the staging server before they are copied to the production server. The IPG checklist details what needs to be checked.
  • Check that the files have been as soon as they see the production report and, if necessary, correct any errors so that the transfer can take place correctly.
  • Limit immediate transfers to emergency situations and for a limited number of files. If these limits are not respected, the Data Centre will inform the EUROPA team.
  • Properly synchronize the content of the staging and production sites using the Staging Manager tool.

File names, extensions and invalid characters

Some file names and extensions cannot be copied to the production server. In addition, certain characters may not be used, because they are reserved for the operating systems of the Data Center servers. The use of these characters would prevent the server management and maintenance programs from functioning correctly. See the list of directory/file names excluded from the transfer mechanism.

Every time a transfer is carried out, a '.request.output' file is created and placed at the root of the site. This report is emailed to recipients chosen by the DG's webmaster.

It is important to read each report to be sure that the operation was successfully completed. The last lines of the report indicate the status of the transfer:

  • "Errors": the transfer was a complete failure
  • "Warnings": check the details and make any necessary corrections
  • nnn bytes: size of the files transferred.

Warnings are typically displayed when certain files are not copied. It is important to verify that the problem files do not impact the overall coherency of the site.

Here are 4 examples:

  1. Correct transfer: no errors, no warnings and nnn octets copied.
    Total bytes copied: 541955
    no error
    no warning
  2. Correct transfer: a file was not copied; check whether it is in fact necessary on the production server.
    Warning File not copied: north_korea/nav/_vti_cnf/north_korea_nav.htm

    [...]
    Total bytes copied: 242601
    no error
    1 warning(s) during update
  3. Transfer failed: If even 1 error occurs, no file will be copied or deleted.
    Error: can not read news_corner/speech12_en.htm, file not found
    Too many errors (1). No file copied or deleted.
    1 error(s) during update
    no warning
  4. Transfer rejected because the date of the files on the staging server is older than those on the production server. No error detected but 0 octet transfered.
    Total bytes copied: 0
    no error
    no warning

Assistance

The Data Centre's EUROPA Webdesk team is available to provide assistance during transfers. Please submit a detailed description of the problem via the local helpdesk.

Top

1.5. Webmaster toolbox

The Webmaster toolbox includes:

  • a list of tools and applications that EUROPA webmasters can use in their daily work
    • HTML editors
    • XML/XLS editors
    • graphic design tools
    • audio editing tools
    • video tools
    • flash animation editing tools
    • quality control tools
    • auxiliary tools
    • web accessibility tools
  • a Browsers test environment.
Top