Resources    About stylesheets    Help

Information on time-series data exchange resources

This document is written to accompany XML time-series files, which meet the Environment Agency's Data Standard. You may have linked to here from a data file, which has been marked up by the "EA Standard XHTML" stylesheet. If you have, you may find helpful information in the help section.

This document was updated on 12th March 2008. Please note that if you wish to download any of the resources listed below you will need access to the internet. The latest version of the Information Document is currently maintained at http://websites.chrisbeales.net/tsx.

Top    About stylesheets    Help   

Resources

The EA XML Time-Series Data eXchange Standard (TSX), was designed as a file format for computer systems to exchange data, for example to send river flow data from the Environment Agency's hydrometric archive into a water company's archive.

For more information about this Data Standard, please refer to the Plain English Document, which is located at http://www.environment-agency.gov.uk/xmlschemas/. This document contains important information about how the XML data standard works, what you can expect to find in a TSX document, and what the information actually means.

The following other resources may also be of use...

Resource name Resource description
XML Analyser (TSX) v4.5


[Screen-shot of Analyser] 		This is a Microsoft® Excel spreadsheet, which contains a number of useful utilities, for example:
  • XML document validation (against the Environment Agency Data Standard)
  • Apply stylesheets to mark-up XML documents into new files
  • Add (or remove) stylesheet references within the XML document
  • Concatenate XML documents together
  • Convert small TSX documents into spreadsheets
  • Write those spreadsheets back to TSX documents
Note that the first 3 functions (green buttons on the tool) can automatically work through a whole directory of XML documents. To do this, you need to tick the 'Whole directory?' box, which is next to the XML data file name.
  • The download file is compressed to about 280kB.
  • You will need to have the "Microsoft® XML Core Services (MSXML) 4.0" SP2 library installed on your computer (click here to get this).
  • Note that the XML Analyser is an operational tool so there are a number of extra functions, specifically designed for data exchange in Environment Agency Thames Region.
"EA Standard xHTML" stylesheet This is the standard stylesheet for viewing a TSX document in a web browser. You should also be able to print and copy-and-paste your data from here as well.
  • Data is formatted into tables
  • A table of contents is created, which is helpful for navigating between sets-of-values
  • Validated data, of different qualities, are coloured to help them stand out
  • Data comments are shown next to the values in the data table
  • Links to reference material are also available
The data is marked-up into XHTML v1.0 (strict).
"EA Standard XHTML (simple)" stylesheet This is the same stylesheet as above but the mapping of comments, in the data table, has been disabled in this simple version of the stylesheet. This may be important if you have large TSX files with lots of comments, which will take a long time to load.
"EA Standard CSV text" stylesheet This is the standard stylesheet for converting TSX data into Comma Separated Values (CSV).
  • This can be useful for users of older systems, spreadsheets, etc, which cannot read XML documents directly.
  • As with the standard XHTML stylesheet, data comments are mapped to values and links to reference material are added to the output.
"EA Standard CSV text (simple)" stylesheet This is the same stylesheet as above but the mapping of comments, in the data 'table', has been disabled in this simple version of the stylesheet. This may be important if you have large TSX files with lots of comments, which will take a long time to load.
"Summary Mean Flag Fixer" stylesheet This stylesheet has been designed, specifically to deal with summary data, sourced from the Environment Agency's Hydrometric Archive (WISKI). These data, for example daily mean river flow data, are often quality flagged in a rather complex manner. The stylesheet interprets these complex quality flags, and outputs a simple 'overall' flag.
  • Note that this is an XML-to-XML stylesheet
  • The XML produced is identical to the original document, apart from 'flag1', which will have been altered to be good, suspect, unchecked or missing
"LTA Probability Bands" stylesheet and other related utilities


[Screen-shot Probability Bands Graph]


[Instructions for using these tools] 		The {tsx-Trans_LTA Probability Bands.xsl} stylesheet generates long-term probability rankings for the data in your XML document. These bands are generated using the method recommended by the Environment Agency - Hydrology Policy Team. They are extremely useful for comparing your current data against long-term trends at a site: for example, they indicate whether your current data is within the Normal range for the site, whether it is above or below this, and, if so, is that Notably or Extremely significant. It also gives you Maximum and Minimum data bands.
  • Note that this is an XML-to-XML stylesheet
  • You may wish to apply the Summary Mean Flag Fixer before using this stylesheet, assuming that you are using daily/monthly summary data from the EA Hydrometric Archive
  • Please bear in mind that the presence of missing data (i.e. 'NaN' or 'Not a Number' values) will cause problems. You should ensure that your data sets are complete before using this stylesheet.
Also packaged with the stylesheet are
  • The {tsx-Trans_Table of Probability Bands (CSV).xsl} stylesheet, which marks-up the results of the {tsx-Trans_LTA Probability Bands.xsl} stylesheet into a CSV table so that they can be easily read into a spreadsheet
  • The {tsx-Trans_EA Standard XHTML (not sorted).xsl} stylesheet, which is identical to the EA Standard except that the sorting of data has been disabled (you will want to use this to look at your results!)
  • The {tsx_Monthly Data comparison v1'0.xls} spreadsheet, which is a template that you can use to generate a table of your data (as generated by {tsx-Trans_EA Standard CSV text.xsl}) against the probability bands that you have marked up using {tsx-Trans_Table of Probability Bands (CSV).xsl}
These stylesheets are designed to run using a Microsoft ® platform. We will have to wait for later versions of the eXtensible Stylesheet Language (XSL) to be taken up before you can run them with other web browers/opperating systems.
Other utilities A set of other stylesheets that you might find useful, e.g. to alter values/units in your XML document.

Top    Resources    Help   

About XML files and XSL stylesheets

An XML file is a data file. You can open it up in a text editor (like Windows ® Notepad) and it is possible to read all of the data inside. However, the way it is written is not particularly friendly. For this reason, stylesheets have been designed, to present the data in nice tables and to arrange the other information in a way that is easy to read.

Stylesheets can do a lot more than simply making XML data presentable. They can be used to convert XML data into other formats, like fixed-width or Comma Separated Values (CSV) text files. They can also be used to change the XML data, for example to: sort, summarise, filter or mathematically alter the data, which may involve complex calculations or just a simple converter (e.g. millimetres into metres - by multiplying by 1000).

The use of XML and XSL stylesheets offer other key advantages. Most notably, these standards are independent of any particular computer system or supplier. You do not need special software to use XML/XSL (NB: all I have ever used is a text editor!) but there are packages available if you want to use them. There are also a lot of free resources out there to help, and tools that carry out the stylesheet mark-up and validation for you.

Schema documents add another dimension to this. They contain sets of rules for your XML documents (e.g. '...dates have to be in this format...', '...every Station must have a stationReference...'), which act like the rules behind database tables. This, in effect, turns your XML data documents into a kind of text-based database. The Environment Agency's Time-Series Data Exchange Format (TSDXS), is our Schema for these time-series XML documents.

Stylesheet mark-up

Using tools, like the 'XML Analyser' (which can download from the resources section) you can mark-up your XML document with a stylesheet. The result of this transformation will then be written to a new file, which - depending on what the stylesheet does - may be a webpage (i.e. HTML - like this 'Information' webpage), a text file (e.g. CSV), or a new XML document.

Stylesheet references

'Stylesheet references' are especially useful for applying presentation stylesheets (i.e. ones that are designed to make the XML data attractive and easy-to-read). Most web browsers (e.g. Microsoft ® Internet Explorer) can take your XML document and apply a stylesheet to it whenever you open the document. This means that you can see your data in a nice webpage but you do not lose any of the advantages that it has in XML format, e.g.

A stylesheet reference is a simple line to the top of the XML document, like the following:

<?xml-stylesheet type="text/xsl" href="tsx-Trans_EA Standard XHTML.xsl"?>

The 'href' part of this directs your web browser to the directory and file name of the stylesheet. For this reason it is important that your XML documents, and XSL stylesheets, are located in the correct directories, relative to one-another.

Note that the XML Analyser tool can also be used to add (or change) stylesheet references, which will save you having to manually edit your XML document in a text editor.

Top    Resources    About stylesheets   

Help for "EA Standard XHTML" stylesheet

If you cannot see Environment Agency logo at the top-right of the file...

The picture file, called 'ealogo.jpg' must be in the same directory as the 'tsx-Trans_EA Standard XHTML.xsl' stylesheet. Note that also in that directory you should have the the 'tsx.css' file, and the schema documents: 'EATimeSeriesDataExchangeFormat.xsd' and 'EAMetadata.xsd'. You can download all of these from the Resources table (follow this link)

If it is taking a long time to load...

If you see the phrase "This stylesheet will map comments with values in the data tables" at the top of the document, you may find that it takes some time for your web browser to process the file. This can be an issue if you have large XML documents, containing a lot of comments. In this case your computer will have to map the dates/times of the values with the start/end of each comment, which can take a lot of work.

If you are not interested in viewing the comments in the data table, you can change the for the simple version described in the table of resources. Note that you can use the 'XML Analyser (TSDXS)' as a tool to easily change your stylesheet references.

However, if you are not able to get to the internet to download the simple version of the stylesheet, it is quite straightforward to adjust yourself. There is a 'switch' inside of the "EA Standard XHTML" stylesheet so that the mapping of comments, in the data tables, is disabled. You will need to open the stylesheet ('tsx-Trans_EA Standard XHTML.xsl') in a text editor, and then change line 16, from:

<xsl:variable name="matchComments" select="'True'" />

to

<xsl:variable name="matchComments" select="'False'" />

Note that this facility is also present in the 'tsx-Trans_EA Standard CSV text.xsl' stylesheet.

If you do not want your data to be sorted...

Both the XHTML and CSV stylesheets sort the data, in each table, by date and time. Most of the time this is the way that you will want to see your data. However, if you want your data to be ordered in the same way as it has been written to the XML document, you can disable this sorting. Similar to the above switch, you will need to change line 17 to:

<xsl:variable name="sortSetofValues" select="'False'" />