Using Magics
Introduction There are four interfaces to Magics. A FORTRAN interface (see ) provides the usage familiar to users of MAGICS 6; C and Python interfaces (see and ) provide identical functionality for C and Python programmers; and an XML-based interface, MagML (see ), is an interpreted plot description that can be parsed and run without the need for a compile/link cycle. A Magics program consists of calls to action routines, each of which is configured by changing zero or more parameters. Parameters in Magics are the attributes to be assigned to the various items that make up plotted output, such as line style, colour and size of plot. They can also be used to define such items as map projection, contouring minimum/maximum levels. Sensible default values for most parameters exist, so it may be the case that very few parameters need to be set in order to obtain meaningful output. An example of a parameter that has no default value is GRIB_INPUT_FILE_NAME - if loading a GRIB file, this parameter must first be set. Exactly how parameters are set and action routines are called depends on which interface is used. Most examples in this manual will use the FORTRAN interface, but enough information will be given to enable the user to perform all tasks using MagML, C and Python. This chapter also describes various tools Magics++ provides to work with the Magics library. These tools are the MagML_interpretor, the magics-config script and the magicsCompatibilityChecker.
FORTRAN Interface A short example program is the best introduction to Magics programming. The following code contains enough comments to describe what the program is doing, and is followed by the plot that it generates.
Program: Using FORTRAN
Resultant Plot
Magics Initialisation Routines in FORTRAN The first Magics routine to be called must be POPEN and the last to be called must be PCLOSE. These two routines perform mandatory initialisation and termination functions. The formats of POPEN and PCLOSE, which have no arguments, are: CALL POPEN CALL PCLOSE
Magics Action Routines in FORTRAN In order to plot something, an action routine must be called. In the example program, three plotting commands are called: PCOAST, PCONT and PTEXT. The following table shows all of the available action routines. Magics Action Routines Syntax Description CALL PCOAST Plots coastlines and grids CALL PCONT Plots contour fields CALL PWIND Plots wind fields CALL POBS Plots observations CALL PIMAGE Plots images, for example satellite CALL PTEXT Plots text CALL PGRAPH Plots graphs CALL PAXIS Plots axes CALL PSYMB Plots marker symbols CALL PLINE Plots lines and shapes over geographic areas CALL PBOXPLOT Plots boxplots
Each action routine has its own set of parameters which have a unique prefix to identify them. For instance, the parameter CONTOUR_LINE_STYLE affects the operation of PCONT. These parameters can be set by the user and, if not set, the default values will be used. Even though parameters can be set anywhere in the user program, action routines will always use the last value assigned to a parameter before the action routine is called.
Magics Parameter Setting in FORTRAN In Magics, plots generated using action routines will be drawn using information from the relevant Magics parameters. These parameter values can be changed dynamically by calling parameter-setting routines. There are only a few parameter-setting routines in Magics, one for each data type. The names of the parameters are passed as parameters using FORTRAN character strings. This method allows easy-to-remember, English language keywords and the list of keywords can be easily extended. The set of routines for parameter setting is divided into four subsets: single parameter setting, single parameter resetting, array parameter setting and multiple parameter setting. Values of Magics parameters can also be retrieved by using the function ENQR and passing it the parameter name as an input parameter.
Single Parameter Setting and Resetting There are three subroutines to assign a single value to a parameter, each postfixed with a C , I or R , corresponding to a different argument type: Magics Single Parameter Setting Routines Function Call Syntax Description CALL PSETC ( MAGICS_PARAMETER, CVALUE ) For setting character arguments CALL PSETI ( MAGICS_PARAMETER, IVALUE ) For setting integer arguments CALL PSETR ( MAGICS_PARAMETER, RVALUE ) For setting real arguments
The first argument should be a character variable or constant specifying the Magics parameter to be set.The VALUE argument should be of the corresponding type for the chosen subroutine. A parameter may be reset to its default value using the function PRESET: There are three subroutines to assign a single value to a parameter, each postfixed with a C, I or R , corresponding to a different argument type: Magics Single Parameter Resetting Routine Function Call Syntax Description CALL PRESET( MAGICS_PARAMETER ) For resetting a parameter of any data type
Array Parameter Setting There are seven subroutines for passing information to Magics in the form of arrays: Magics Array Parameter Setting Routines Syntax Description CALL PSET1C ( MAGICS_PARAMETER, CARRAY, N1 ) For 1-dimensional character array arguments CALL PSET1I ( MAGICS_PARAMETER, IARRAY, N1 ) For 1-dimensional integer array arguments CALL PSET2I ( MAGICS_PARAMETER, IARRAY, N1, N2 ) For 2-dimensional integer array arguments CALL PSET3I ( MAGICS_PARAMETER, IARRAY, N1, N2, N3 ) For 3-dimensional integer array arguments CALL PSET1R ( MAGICS_PARAMETER, RARRAY, N1 ) For 1-dimensional real array arguments CALL PSET2R ( MAGICS_PARAMETER, RARRAY, N1, N2 ) For 2-dimensional real array arguments CALL PSET3R ( MAGICS_PARAMETER, RARRAY, N1, N2, N3 ) For 3-dimensional real array arguments
The fifth character of the subroutine name indicates that an array is being passed and also indicates its dimensionality. The sixth character indicates the FORTRAN type of the array, i.e. Character, Integer or Real. The first argument should be a character variable or constant specifying the Magics parameter to be set.The VALUE argument should be of the corresponding type for the chosen subroutine. The second argument should be an array of the appropriate data type containing the data to be assigned to the Magics parameter. Arguments N1, N2, N3 are integers which inform Magics of the size of the array being passed, i.e. number of elements in each dimension of the array. This type of subroutine can be used, for example, to pass lists of contour levels to Magics, e.g. CALL PSET1R('CONTOUR_LEVEL_LIST', RLIST, 25) where the real array RLIST contains a list of 25 contour levels.
Magics Pseudo-Action Routines in FORTRAN As distinct from action routines, the pseudo action routines perform a type of action that does not result in plotted output but signifies a change of state in Magics, such as moving to a new page. Pseudo action routine PNEW will change from one plot area to another. The format of PNEW is: Magics Pseudo-Action Routines Function Call Description CALL PNEW ( 'SUPER_PAGE' ) To start a new super-page CALL PNEW ( 'PAGE' ) To start a new page CALL PNEW ( 'SUBPAGE' ) To start a new subpage
The change of plotting area does not take place when PNEW is called but when the next action routine is called. Pseudo action routine PNEW is described in .
Magics Data Loading Routines in FORTRAN The following action routines are used to read data: Magics Data Reading Routines Function Call Description CALL PGRIB Reads contouring and wind fields presented to Magics in the form of WMO standard GRIB code CALL PNETCDF Reads data from a NetCDF file CALL PBUFR Reads observation data from a BUFR file. Note that this replaces POBS from MAGICS 6.
The magicsCompatibilityChecker With its Fortran interface, Magics++ tries to be as backwards compatible as possible to MAGICS 6. Unfortunately this was not always possible and changes in the API have occurred. To help users to access if their Magics Fortran programs need changing a script is provided. This script is called magicsCompatibilityChecker and is called from the command line followed by the name of code file to be checked. An output will be printed to alert users of changes in the Fortran API. magicsCompatibilityChecker mycode.f
C Interface Magics provides a C programming interface. This interface is identical to the FORTRAN interface in terms of functionality, but with different action routine names in order to avoid conflicts with some C standard library names.
Converting From FORTRAN to C If you wish to copy an existing FORTRAN Magics program and change it to C, then here are the things that you will need to change within the body of the program: In order to derive the C form of an action routine, take the FORTRAN routine name, change it to lower case and replace the initial P with mag_. For example, POPEN becomes mag_open. Strings in C use double quotes, not single quotes (" not '). Remember to place a semicolon at the end of each line. Remember that in C, function names are case-sensitive, but the Magics parameter names are not. In FORTRAN, if a 2-dimensional array is declared as DIMENSION FIELD (NLON,NLAT), the equivalent in C would be double Field[NumLat][NumLon], ie the array indices are 'the other way around'.
C Interface Example The following example shows how to use Magics in a C program.
Resultant plot from example C program
Python Interfaces
Procedural Python interface From version 2.7 onwards Magics provides a procedural Python programming interface. This interface is identical to the FORTRAN and C interfaces in terms of functionality, with action routine names similar to the Fortran interface.
Converting From FORTRAN to Python If you wish to copy an existing FORTRAN Magics program and change it to Python, then here are the things that you will need to change within the body of the program: The Magics interface needs to be imported at the start of your script using import Magics and all Magics commands need to be prefix with the namespace Magics.. This is not strictly necessary but keeps a clear separation of Magics and other code. In order to derive the Python form of an action routine, take the FORTRAN routine name, change it to lower case and replace the initial P with the namespace Magics. There are few exception such as, POPEN becomes Magics.init() and PCLOSE becomes Magics.finalize(). Strings in Python use double quotes, not single quotes (" not '). In Python, function names are case-sensitive, but the Magics parameter names are not. In FORTRAN, if a 2-dimensional array is declared as DIMENSION FIELD (NLON,NLAT), the equivalent in Python and follows the same convention as C. Please be aware that for arrays Magics uses NumPy.
Python Interface Example The following example shows how to use Magics in a Python program.
Page 1 of resultant plot from example Python program
Page 2 of resultant plot from example Python program
Higher-level Python Interfaces Additional to the procedural Python interface, Magics also offers since version 2.9 a higher level Python interface called MagMacro. This interface offers a more higher level and object-oriented interface, based on Metview Macro.
Python Interface Example The following example shows how to use Magics in a Python program.
Resultant plot from above example Python program
User setup and compilation This section will discuss the setup needed by users to use the installed Magics library and its applications.
Magics++ environment variables The variable $MAGPLUS_HOME is the path where Magics will find all auxillary data at runtime. If Magics is installed where the configure prefix option was set to, this variable does NOT need to be set. Otherwise the variable needs to be set for all users of Magics programs. If Magics reports problems at run-time about missing coastlines or fonts this variable might not be set correctly. Magics++ provides feedback on the console. You can control the messages plotted by setting any of the following environment variables: $MAGPLUS_INFO - Information and hints to what Magics++ is doing $MAGPLUS_DEBUG - Debug information $MAGPLUS_QUIET - suppresses the Magics++ header, footer and info output – only errors and warnings will cause output
System environment variables Hint: If you choose a standard system folder, such as /usr/local/, for your installation you might not have to change any system environment variables. $LD_LIBRARY_PATH - To work with shared libraries the locations of these libraries need to be known at run time. This environment variable contains a list of paths where at run-time the system loader will look for libraries. To use the shared library version of Magics++ the location of the library needs to be set in $LD_LIBRARY_PATH. If the variable is not set correctly the loader might pick up a version of Magics++ which was not intended. If EmosLib is compiled with Pgf90, it binds in libpgc.so dynamically at runtime and so the path to this library needs to be in $LD_LIBRARY_PATH. $PATH - Magics++ provides not only libraries, but also some executables. To use these, the $PATH needs to include the location of these programs. Usually this will be $MAGPLUS_HOME/bin.
magics-config To simplify the user setup Magics++ installs a script called magics-config, which prints out the options to compile Magics++ programs. The options are based on the options used to compile Magics++. The program outputs the compilation and linkage options which can be used directly in the compilation instruction, as shown below. To compile a C program and link it to Magics++ you only need to type gcc shade.c -o shade `magics-config --clibs --cxxflags` To compile simple Magics Fortran programs you can use magics-config by typing magics-config --compile=test.f In case a different suffix is used for Fortran code the call can be extended to magics-config --compile=test.F90 --suffix=F90 There is a similar shortcut for C code called --compileC=test.c. To do so $MAGPLUS_HOME/bin needs to be added to the $PATH first or magics-config called with its full path. Please be aware that the option '--print-setup' will tell you which environment variables you need to set before you can compile and run Magics++ programs. You might want to add this setup in your shell or start-up scripts. Below you find the usage help for magics-config. Usage: magics-config [OPTION] ... Uses by default version 2.12.5 of Magics++. Generic options --version output Magics++ version information. --help display this help and exit. --print-setup print how the environment can be set up Compilation support options --cxxflags print pre-processor and compiler flags for C/C++ --libs print linking flags for C++ --clibs print linking flags for C --f90static print library linking information (static) --f90shared print library linking information (shared) --compile compile simple Magics++ Fortran programs (default single precision!) Example: magics-config --compile=wind.f --compileC compile simple Magics++ C programs Example: magics-config --compileC=wind.c --suffix set suffix of Fortran or C code files --double include compiler option for fortran double precision (default single - only for interface - internally only double is used) --64bit include fortran option for 64 bit C/C++ flags use shared libraries if possible, otherwise static libraries are used. '--double' has NO effect on C/C++ programs. Install directories Magics++ was configured to --prefix[=DIR]
Compiling with pkg-config pkg-config is a script that provides a unified interface for querying installed libraries for the purpose of compiling software from its source code. pkg-config was originally designed for Linux but is now also available for the various BSDs, Microsoft Windows, Mac OS X, and Solaris. It outputs various information about installed libraries. This information may include parameters for C or C++ compiler and parameters for linker. This script is very similar to what magics-config offers, but pkg-config is more general while magics-config has very Magics specific options. gcc -o test test.c $(pkg-config --libs --cflags magics) To find the information about Magics compilation the environment variable PKG_CONFIG_PATH needs to contain the directory $MAGPLUS_HOME/lib/pkgconfig. For more information visit pkg-config on Wikipedia.
Using Magics++ in your own configure scripts [advanced] To simplify the use of the Magics library in software also using the autotools package, a script called magics.m4 is provided. The file can be found in the Magics++ install directory under share/aclocal/. It is recommended to copy the file into the own project and include it in the configure.ac file by typing m4_include(m4/magics.m4) This will provide the configure script with the option --with-magics to query Magics on the system. Internally magics.m4 uses magics-config.
MagML Interface New in Magics++ is an interface, based on XML, for describing plots. This interface is different to other interfaces, which use programming languages to describe a plot. In contrast to programming languages, MagML cannot offer procedural programming. However, flexibility is provided by the use of templates - see . MagML offers users a way to describe their plots in a text file. The MagML interpreter program reads this text file and generates the plots; this process eliminates the need for compiling a FORTRAN or C program. Variable elements such as the path to a data file can be supplied on the command line. A plot is described through a hierarchy of objects. Data access is possible by defining an input data file or a database request.
The MagML Interpreter To generate Magics plots from MagML files, a program called magmlx is provided. The program will be available to the user after the Magics environment is set up. The command-line usage of the interpreter is as follows: magmlx file.magml The parameters shown as are described in . A short example program is the best introduction to Magics programming with MagML. The MagML source is followed by the plot that it generates. The important parts of the code will then be discussed.
Program: Using MagML
Resultant plot from example MagML program
MagML Discussion First we establish that we are describing a Magics plot by providing a magics tag, and that the version of MagML we are compatible with is 3.0. Future versions of the MagML specification may break or cause unexpected results in existing MagML plots, and so the version attribute can be used by the MagML interpreter to inform the user of any relevant changes. Page dimensions can also be specified here. The drivers object defines the output files. See for details of the output drivers. In this example, we produce two output files: one in PostScript format and one in GIF format. Filenames are given without extensions - these will be added automatically. Now we define a page object which will contain all the plots to be made on a single page of output. The map object sets up an actual plot area within the page. Geographic view parameters are set here with the cylindrical object which is set to use its default parameters - our plot will be in cylindrical projection. Next the coastlines are set up - the coast and grid are sub-objects of the coastlines object and have attributes which may be set. Next we define a plot. A plot contains two things: a description of the data we want to plot and a description of how we want to plot it. Here we define a grib object with its path specified. Then we define a contouring object (contour) which contains the information needed to plot it. This object contains a sub-object (isoline) which sets the properties of the isolines. Finally, two text objects set up an automatic and a user title.
MagML Templates The flexibility of MagML is greatly enhanced by the use of templates. A template is a MagML description file which contains zero or more variables, placeholders which are replaced by user-defined text supplied to the magml program on the command line. For instance, instead of giving a contour colour in the MagML file as: <isoline colour='red'> you could declare that this attribute will be specified on the command line: <isoline colour='$iso_colour'> This value can then be supplied on the command line, for example: magmlx file.magml A default value can be provided in order to deal with cases where no value is given on the command line: <isoline colour='${iso_colour=red}'> If passing a string which contains spaces, place quotes around it: magmlx file.magml Note that a 'standard' MagML description file is simply a special case of a template - one with zero variables to be replaced!
MagML Definitions It is possible to predefine some elements at the start of a MagML file and then reference them later in the same file. This technique allows you to re-use elements within a MagML description as demonstrated in the following example: The definitions element contains all the elements we may wish to re-use. Each one is assigned a unique id which is then referenced by setting it as the value of a use_id attribute as shown above. This idea can be expanded to re-use an element multiple times. Another use for definitions is to generate fixed variations of a plot. This is especially powerful when combined with the idea of templates (see ). In the following example, we set up two different map views - one displays a global area, the other is restricted to Europe. The id of the area to use is determined by the variable area. This is set to a default value of globe but can also be set on the command line. If this MagML description is stored in a file called plotmap.magml, then it could be invoked on the command line in any of the following three ways: magmlx plotmap.magml (will plot a global map) magmlx plotmap.magml (will plot a global map) magmlx plotmap.magml (will plot a European map)
MagML Parameter Tables Each chapter contains complete descriptions of one or more MagML elements relating to that chapter's topic. All possible attributes and sub-elements are listed in MagML format. This means that it is possible to copy and paste those elements into your own MagML code. All attributes are listed with their default values, meaning that you can delete all the elements and attributes that you do not wish to alter. The following listing gives the overall structure of a MagML description. If you copy this, you will need to edit it, as it contains more information than a simple skeleton of a MagML element. Elements in this table have comments showing whether they can be repeated or not in a single MagML description. A plot may contain only one element to describe the data source, but multiple elements to visualise the data. For instance a single data source may be plotted with contours, text and a legend. Some of the above elements are hyperlinked to their detailed descriptions in their respective chapters. In the PDF documents, this may take you to the start of the chapter; you will then need to click on the appropriate link in the table of contents to see the description.
MagML Coding Styles MagML version 3 has an important difference from previous versions. Whilst backward-compatibility has been mostly retained, a new flexibility has been introduced. Versions 1 and 2 used a strict system of hierarchical objects with short attribute names linked to their tag. Version 3 of MagML maintains this system, but also allows for a more 'flat' expression of objects. An example follows, and then some explanation. The first version shows the deepest possible hierarchy of objects. This was the only option available in MagML versions 1 and 2. The second version of the code shows a completely flat structure with just one object - the attribute names are fully expanded in order to uniquely identify them. Which format you use is largely a matter of personal preference, but there are some issues which may affect your decision. The hierarchical format has the advantage that definitions (see ) can be more precisely targeted at a sub-object. The flat format shows the attribute names in their fully expanded form and will be more familiar to users of the Fortran or C interfaces. It will also be more familiar to users of Metview, as it also uses these expanded names.
Using GRIB loops in Magml MagML has a feature which allows for the automatic production of a series of plots, overlays or an animation from the fields in a single GRIB file.
Producing one plot per field Animations are generated when the output driver is 'kml' or 'animated_gif'. The PostScript driver ('ps') produces a single file with each plot on a separate page. The single-page drivers such as 'png' and 'svg' produce one file for each plot. Note that in all cases except 'kml', this facility can be used to generate multiple plots per page by setting the map dimensions to be less than 50% of the page width or height. This provides a simple way to plot, for instance, thumbnails of all fields in a GRIB file. A GRIB loop is enabled by placing a gribloop object just below the page level. The following example shows how to set it up, and an explanation follows. The layer within the gribloop will be repeated over the fields of the GRIB file. The 'loop_step_span' parameter is used in KML output to specify the duration of the validity of the field; this allows fields with different base times to be overlaid in applications such as Google Earth.
Producing one plot in total When placed inside the map level, the gribloop will cause all fields to be overlaid on the one map. The following example shows how to do this.
See for the complete set of MagML objects and parameters that are available for this purpose.
MagML Page Layout Some elements of a plot are positionable and resizeable. These are the page, map, legend and text objects. See for the specific attributes but, for example, specifying <map width='50%'> will produce a map which occupies half of the width of the page. More specifically, this is 50% of the map object's parent page object, which may also have a non-default size and position. If the size does not include a percentage sign, then it is assumed to be given in centimetres. By default, an object's position is 'inline'. In this case, the ordering of the objects in the MagML code is important - they will be automatically arranged left-to-right and top-to-bottom: if there is sufficient space to the right of the previous object, then that is where the current one will be placed; otherwise, it will go below. The top and left attributes of the object will not be considered in this mode. An alternative is to position an object as 'block'. Here, the object will be placed below the previously specified one. This mode has the property that the layout of the objects on screen roughly follows their layout in the source MagML code (since an object's MagML code will be below that of the preceding object's code). The object's top attribute will not be considered in this mode, but its left attribute will. The final alternative is to specify an object's position as 'absolute' in which case the positional attributes for that object will be used - remember that the position is relative to the parent object. In this case, both the top and left attributes of the object will be considered.
Metgrams from Weather Parameter files Magics' Python interface allows users to visualise Weather Parameter files. These files are a product of ECMWF and are disseminated in BUFR format. A more detailed description of the format can be found at http://www.ecmwf.int/services/dissemination/3.1/Meteorological_Bulletin_M3_1_20.html To reproduce what is called a classic metgram as shown below, a small python script needs to be written. The user can control a few parameters such as: The PostScript filename for the output The station position and title. ( Note that the position should be in the BUFR input file) The list of parameters to be displayed Example code: from metgram import * #Definition of the output ps = ps({"output_fullname":"metgram.ps"}) #definition of the Station mystation = station({"input_filename":"GRA07041200071412001", "station_latitude":"40.62", "station_longitude":"35.82", "station_name":"My Test Station"}) metgram(ps, mystation, cloud, humidity, precip, msl, wind, temperature) The script can be excuted by typing python myexample.py In the first version (Magics 2.12), the user can create a PostScript using the function ps. In this function, the user can further specify the name of the PostScript file using the parameter key output_fullname: ps = ps({"output_fullname":"metgram.ps"}) The metgram application is expecting a BUFR file as input, and a location to extract from this file. The location is defined by its latitude and longitude, and its name will appear in the title: mystation = station({"input_filename":"GRA07041200071412001", "station_latitude":"40.62", "station_longitude":"35.82", "station_name":"My Test Station"}) The table below gives an overview of the parameters currently supported: Metgram parameters name BUFR descriptor scaling offset visualisation cloud 20010 1 0 blue bar humidity 13003 1 0 red curve precip 13011 1 0 blue bar msl 10051 0.01 0 red curve wind u:11003v:11004 1 0 red flags temperature T2m: 12004T850:12001 1 -273.15 T2m as red curveT850 as blue curve
For future releases it is planned to add more parameters and to allow users to specify their own tailored parameters and visualisations.
Setting colours Various parameters let the user set colours for different action routines. Magics offers various ways to specify a colour value: By name - A list of 56 predefined colour names, such as BLUE, RED, RED_ORANGE. By RGB(A) values - The user specifies the colours by their Red-Green-Blue(-Alpha) value. Examples: RGB(1.,0.,0.) for red, RGB(1.,0,1.) for purple. By RGB(A) Hex values - Similar as above the RGB(A) components are expressed in HTML hex notation. Examples: #FF0000 for red. By HSL(A) values - These are three numbers: the first is an angle from 0 to 360 representing Hue, then two numbers from 0 to 1 representing Saturation and Lightness. The following diagram shows the HSL colour wheel: However the colours are specified, when Magics generates a colour scale from one colour to another, it internally traverses the HSL wheel either in a clockwise or anti-clockwise direction (this is user-configurable). This is important in order to understand the series of colours produced. Magics++ now also supports transparency. Unfortunately not all output formats (PostScript) support this feature. Please be aware that colours might be converted to a different colour space, dependent on the selected (graphical) output format. The following is a chart showing all available named colours in Magics.