P-GRADE Grid
Portal is an open source workflow-oriented Grid portal that enables
the development, execution and monitoring of workflows composed of
sequential jobs, parallel jobs and application services. The P-GRADE
Portal can be used to develop and execute workflow applications on Grid
systems built with Globus, EGEE (LCG or gLite) and ARC middleware
technologies. The aim of this tutorial is to demonstrate the EGEE
capabilities of the P-GRADE Portal using the GILDA training
infrastructure.
The tutorial covers the following
P-GRADE Portal features:
- Managing user certificates
- Importing workflows
- Developing, executing and monitoring workflows
- Using Storage Elements
For a detailed overview of all the
capabilities of the P-GRADE Portal please visit its official homepage.
Login and configuration
- Open the URL: http://portal.p-grade.hu/multi-grid
- Login to the P-GRADE Portal using the account which was given to you by the tutor.
- Set notification information: P-GRADE Portal is able to
notify the user in email about state changes of the grid application.
To enable this feature please click on the “Workflow” portlet, then on
the “Notify” sub-portlet. Set
Email settings:
- Enabled: Yes
- Email address: Specify your email address
- Email subject: P-GRADE Portal workflow status change
Workflow Change Settings:
- Enabled: Yes
- Message: Specify a message content using the listed keys as macros to be automatically replaced by actual values
An example setting is shown on the screenshot:
Import a workflow
In this excercise you
will download and then import an example workflow into your GILDA P-GRADE
Portal account. The workflow was created by the Centre of Parallel
Computing at the University of Westminster. The workflow simulates the
development of traffic on a city road network. The result of the workflow
is a JPG file that graphically present the density of cars on the roads.
Further information on the workflow can be found here.
- Download the archive file of the traffic simulation workflow onto
your desktop computer. The archive file can be found here.
- Go back to the portal and click on the "Upload"
submenu of the "Workflow" tab
- Select the previously downloaded archive file using the
"Browse" button and click OK
- Click on the "Workflow manager" submenu of the
"Workflow" tab
- Click on the "Workflow editor" button and open the
previously imported workflow ("Open" item in the Workflow menu)
- Check the mapping of the jobs. All of them should be allocated to
"gilda_GLITE_BROKER" grid. ("Workflow properties" item
in the "Workflow" menu)
- Save the workflow ("Save" or "Save as" in the "Workflow" menu)
Download a short-term proxy credential into the
Portal
- Open on the "Certificates" tab
- Click on the "Download" button
- Submit the download form with the following values:
- Hostname: grid001.ct.infn.it or n40.hpcc.sztaki.hu
- Port: 7512
- Login: parisXX
- Password: parisXX
- Lifetime: 24
- Description: optional
- If download is successful then set your proxy for the
"gilda_GLITE_BROKER" Grid
Execute and monitor the Traffic simulation
workflow
- Go back to the Web interface of the portal and click on the
"Refresh" button on the "Workflow" tab
- Click on the "Submit" button of the workflow
- The
notification engine asks if you wish to get informed in email about
status changes. Select "ON ANY STATUS CHANGE" or "ON WORKFLOW END ONLY"
- Monitor the progress of the workflow from the Workflow Editor and
using the "Workflow portlet" (Click on "Details" then
on "Visualize")
Define a new workflow: Matrix multiplication
Note: The "Matrix operations" batch program
will be used as the jobs of the workflow. Please read the description of
this program here, and download the program with two sample input
matrixes onto your desktop computer from here. You can use any filename to save this binary on the local computer. File name extension does not matter either.
- Open a new workflow (Workflow menu in the Editor) and define a new
job with the following parameters:
- Name: Multiplication
- Job type: SEQ
- Job executable: local path of the previously
downloaded matrix_operations file
- Attributes: M V
- Grid: gilda_GLITE_BROKER
- Define a port for the job with the following parameters:
- Port Name: 0
- Type: In
- File type: Local
- File: local path of the previously downloaded
INPUT1 file
- Internal File Name (case sensitive): INPUT1
- Define a second port to the job with the following parameters
- Port Name: 1
- Type: In
- File type: Local
- File: local path of the previously downloaded
INPUT2 file
- Internal File Name (case sensitive): INPUT2
- Define a third port to the job with the following parameters:
- Port Name: 2
- Type: Out
- File type: Local
- Internal File Name (case sensitive): OUTPUT
- File storage type: Permanent
- Save your workflow as Multiply, go back to the browser, click
"Refresh" then "Submit" on the
"Workflow manager" panel.
- Monitor the execution from the workflow editor and from the browser
- After the workflow reached fininshed state download
and unzip its result file.
Save the multiplication into a Grid file
- Save the Multiply workflow as Multiply_remoteout ("Save as"
in the "Workflow" menu)
- Modify port 2 to a remote file:
- Double click on port 2
- Set the "File type" parameter to Remote
- Set the "File" field to lfn:/grid/gilda/tutorials/parisXX/multiply_output
(XX refers to your ID on the t-infrastructure)
- Save your workflow.
- Submit your workflow and monitor its execution from the workflow
editor and from the browser.
- Please note that after the execution there is no result file to
download!.
- Check the existence of the result Grid file from the File
Management portlet:
- Open the File Management portlet
- Set Select Grid listbox to gilda, then click on
View in the same line
- Set Select VO listbox to gilda, then click on
View in the same line
- Select lfc-gilda.ct.infn.it LFC host and click on
List
Be patient and then navigate into
/grid/gilda/tutorials/parisXX directory using the "File Browser"
and the "Change Directory" buttons. Download the matrix_output file
to your desktop and check its content. It should be the same as earlier
when local output file were used.
Define a matrix operations workflow
In
this exercise you should define a workflow which computes the
following expression: AB[*, 0]T * AB[*, 1] (A and B
represent the previously downloaded INPUT1 and INPUT2 matrixes)
Hint:
The "Matrix operations" program reads and produces files in the same
format. Add the matrix_operation program 4 times to the Multiply
workflow as it is shown in the figure below. Do not forget to save the
new workflow under a different name (e.g. Matrix_operations)
The
jobs should compute the following operations:
- Multip: A*B (command line parameter:
M)
- Column0: A*B[*, 0] (command line
parameters: C 0)
- Column1: A*B[*, 1] (command line
parameters: C 1)
- Transpose: A*B[*, 0]T
(command line parameter: T)
- Multip.2: A*B[*, 0]T * A*B[*, 1]
(command line parameter: M)

Figure 1: Matrix workflow to compute AB[*,
0]T * AB[*, 1] The complete workflow can be downloaded
from here. Do
not forget to reallocate the jobs to the "gilda_GLITE_broker" Grid after
importing the workflow!
|