Page tree
Skip to end of metadata
Go to start of metadata

Purpose

Built by Data Centre, the Staging Manager programme uses only the HTTP protocol to connect to the server and works only with the standard Commission browser (MS Internet Explorer 7 and higher) - it does not work with other browsers.

The Staging Manager does not enable you to load more than one local file to the server staging without individual selection: this tool is therefore best used for small numbers of files. To transfer larger numbers of files, Staging Manager enables you to use ZIP files, with automatic unzipping.

Traditional FTP tools are still the suggested alternative for large-scale transfers of local files to the staging environment.

The Staging Manager also compares files and indexes between the staging and production environments, enabling each manager to synchronise and tidy up their content.

Read and write authorisations are identical to the ftp access allocated to webmasters in their individual staging environment: depending on the specific rights they have, they can write to the file ‘request.update’ or ‘request.delete’ and launch a production run (‘mise en production’).

Production runs are processed in parallel (up to 20 at a time) to speed up the process and limit queuing. 

Sites are put online in 3 steps

  • new files are copied from the local or development site to the staging server
  • new files are verified on the staging server
  • once verified, the files are copied to the production server

The transfer to the production server applies to the files/directories on the staging server that are indicated in the .request.update or .request.delete folders placed at the root of the site in order to be copied or deleted on the production server.

In an emergency you can immediately copy files/directories to a production server by using the Staging Manager's save and execute function.

Rules

The Europa team at DG Communication B3 is responsible for granting access rights to the static Europa server. The central management programme Staging Manager, accessible from the IE navigator, is adapted for

  • a limited number of files/folders or a compressed file (format .zip),
  • the automatic creation of request files
  • start immediate production

The Staging Manager tool is accessible at http://doteu.staging.cc.cec.eu.int:8888/

Procedures

Procedure description

Four special files are used by the staging to production transfer procedure (also known as MEP, or Mise en production). For each 'subsite' these files have to be located in the root directory of that 'subsite' on the staging server.


All file names listed inside these special files are relative to the root directory of the subsite they apply to (metacharacters "* or ." are not allowed).


The special files used by the MEP update procedure are likely to exist within each subsite document root directory of the staging server as follows

  • .request.delete lists all files and/or directories that are to be deleted from the production server
  • .request.update lists files and/or directories that have to be added or updated from the staging server to the production server
    • files are copied only if a newer version exists on the staging server (based on the last modification date)
    • directories include recursively all subdirectories. All files found in the production directories that do not have a counterpart on the staging site any longer will be deleted from the production site
    • "<ALL>" can be specified as the only entry in order to copy the whole hierarchy. Even if this is the easiest way to proceed, this must be avoided as much as possible, because it implies a much longer processing time (<ALL> really does mean ALL)
    • the files are first copied to a temporary environment where checks are made. Missing files or files containing errors (absolute links, etc.) might stop the procedure and none of the files will be copied to the production server at all
  • .request.lock is created at the start of the automatic procedure meaning that no one should modify any of the files from the staging directory until this file has been removed by the update procedure
  • .request.output contains the results of the procedure. All the files which have been deleted and copied are listed. Error and warning messages can be found

Important remarks

  • the '.request.update' and '.request.delete' files should be created only after all modifications on the staging site have been completed, because the presence of these files means that the listed data are ready to be copied
  • the presence of '.request.lock' file means that the procedure has started and has not completed yet. You should not modify any of the transferred data on the staging server until this file has been removed by the procedure
  • '.request.delete', '.request.update' and '.request.lock' are always deleted at the end of the procedure
  • '.request.output' contains only the result of the latest execution of the procedure. This file is overwritten at each execution, so the preceding results will be lost. If you want to keep a history of your modifications, then you will need to save this file under another name. The contents of the .request.output can be sent automatically to the relevant webmasters by email

Scheduled or immediate production

Files and directories are automatically copied to the production server everyday between 13.00 and 22.00.

In certain circumstances, the webmaster can ask for this timeframe to be changed by emailing Comm Europa Management.

The '.request', etc. files must be ready and correct prior to the scheduled move to the production server.

In an emergency you can immediately copy files/directories to a production server by using the Staging Manager's save and execute function.

Transfers to production servers use significant amounts of resources and reduce server performance. 

N.B. Because of the reverse proxy's caching mechanism, it may take up to 15 minutes before newly copied html pages are displayed, slowing down the operation. If necessary, you can force a refresh of the proxy with Firefox, by pressing ctrl + shift - reload.

The reverse proxy makes access to Europa faster by memorising all the static pages (with the exception of PDFs). The refresh algorithm works like this

  • < 15 min. for html pages
  • = 8 hours for static files (images, css, js, etc.)

Guidelines

File names, extensions and invalid characters

Some file names and extensions cannot be copied to the production server. In addition, certain characters may not be used, because they are reserved for the operating systems of the Data Centre servers. The use of these characters would prevent the server management and maintenance programs from functioning correctly.

See:

DownloadPDF - 261.3 KB

Every time a transfer is carried out, a '.request.output' file is created and placed at the root of the site. This report is emailed to recipients chosen by the DG's webmaster.

It is important to read each report to be sure that the operation was successfully completed. The last lines of the report indicate the status of the transfer

  • "Errors": the transfer was a complete failure
  • "Warnings": check the details and make any necessary corrections
  • nnn bytes: size of the files transferred

Warnings are typically displayed when certain files are not copied. It is important to verify that the problem files do not impact the overall coherency of the site.

Here are 4 examples

  1. Correct transfer: no errors, no warnings and nnn octets copied.
    Total bytes copied: 541955
    no error
    no warning
  2. Correct transfer: a file was not copied; check whether it is in fact necessary on the production server.
    Warning File not copied: north_korea/nav/_vti_cnf/north_korea_nav.htm 
    [...]
    Total bytes copied: 242601
    no error
    1 warning(s) during update
  3. Transfer failed: If even 1 error occurs, no file will be copied or deleted.
    Error: can not read news_corner/speech12_en.htm, file not found
    Too many errors (1). No file copied or deleted.
    1 error(s) during update
    no warning
  4. Transfer rejected because the date of the files on the staging server is older than those on the production server. No error detected but 0 octet transferred.
    Total bytes copied: 0
    no error
    no warning

Background

Transfer tools

In order to publish your files on the Europa server you need a transfer tool. There are 3 main types

Classic FTP tool

Those such as FileZilla, WS FTP Pro, etc. adapted to transfer

  • many local files/folders
  • a variety of files/folders
  • no immediate production

Information to set up your FTP tool

  • Host name/address : doteu.staging.cc.cec.eu.int
  • Host type : UNIX
  • UserID : CUD-login
  • Password : central pass words - internet
  • Path to the subsite/s

Staging Manager tool

See 'Rules' above. 

Procedure or programme

Every procedure that uses command systems or IT programmes that support the FTP transfer instructions is adapted for intensive or regular internal production (e.g. news).

This method needs to be able to perform the following operations

  • the selection and FTP transfer of files
  • the creation and transfer of the .request update file
  • [The http: command: URL, login, password for immediate production ]

An old procedure that was installed by the Data Centre is still being used for a limited number of sites. This is due to disappear in favour of the latest solution.

Policy

Responsibility for transfers to production servers

Managers authorised to carry out transfers to a production server must agree to

  • check the pages on the staging server before they are copied to the production server.
  • check that the files have been as soon as they see the production report and, if necessary, correct any errors so that the transfer can take place correctly
  • limit immediate transfers to emergency situations and for a limited number of files. If these limits are not respected, the Data Centre will inform the Europa team
  • properly synchronize the content of the staging and production sites using the Staging Manager tool




Contact and support

Type of access rights

FTP-access

Makes it possible to upload files/folders from your own development-environment (local disc, shared disc or development-server at your DG) to the staging-server of Europa.

Permission to make scheduled transfers

Makes it possible to place .request.update and/or .request.delete-files commands, for lists of files to be transferred from the staging to the production server, or to be deleted from the production-server. Updates take place automatically at 13.00 and at 22.00 every day.

Permission to make immediate updates

Makes it possible to place the .request.update and/or .request.delete-files commands and to execute the transfer immediately (via the "save and execute" function of the staging-manager).

Please note that in order to obtain permission to make scheduled and urgent updates to the production server, the request has to be made by the relevant DG webmaster and the person requesting access.

How to obtain access rights

In order to obtain access rights, the webmaster of your DG should send an application to the Europa team, giving the following information (please copy + paste the following text to your mail and fill in)

  • URL(s) for which access is asked for:
    http://ec.europa.eu....
     
  • CUD-login(s) of the person(s) who should get FTP-access to the staging-server of Europa
      
  • CUD-login(s) of the person(s) who should get the possibility of making updates on the production-server
     
  • CUD-login(s) of the person(s) who should get the possibility of making immediate updates on the production-server (those logins have to figure also under point 3)
      
  • please indicate the email addresses of the persons who should get the automatically generated transfer-reports:
    ...@ec.europa.eu
  • No labels

The Europa Web Guide is the official rulebook for the European Commission's web presence, covering editorial, legal, technical, visual and contractual aspects.
All European Commission web sites must observe the rules and guidelines it contains.
Web practitioners are invited to observe its contents and keep abreast of updates. Read more.