Approaching Astrogrid

There are certain topics on which it is essential to have some background information if you are to make sense of doing astronomy in Astrogrid. The next five topics are the briefest introduction to the most important.

Workflow

Astrogrid uses workflow to do astronomy. That fact requires explanation.



In one sentence: Astrogrid provides a network of computer resources which can be choreograped to do astronomical investigations. Such investigations can be multi-stepped tasks, arranged across the network, taking place over a period of time (certainly minutes, probably hours, maybe days, and could be weeks depending on the complexity of the work) and can be designed to do numbers of tasks simultaneously. An investigation can indeed be a simple single-step task, though even here Astrogrid has something new to offer in the use of the registry, for which see below. The Astrogrid forte, however, is to use workflow to deal with complexity. It would be very difficult (but not impossible!) for an astronomer to do this solely using his/her own desktop machine. With Astrogrid the workflow paradigm is built in and you only require a browser in order to start.



(Incidentally, Astrogrid does not ignore desktop applications; the home page gives access to a new suite of desktop programs being developed as a part of the Astrogrid 2 project.)



The workflow paradigm enables us to do long-running investigations. An astronomer can logon, submit a workflow for execution, logoff and wait for an email to indicate finish of the job, then logon again to access the results. There are pro's and con's to this situation.



Against:

  • Another set of skills has to be learnt in order to design a workflow and also a query to execute within a workflow.
  • There is no immediate source of gratification. Not only does a workflow need to be designed, which itself takes time, but when executed it is run asynchronously - you fire and forget, and interrogate the results later when the job is finished.




For:
  • Workflows become a source of intellectual capital. They can be reused, refined over time and shared.
  • Because Astrogrid workflows execute across a distributed computer network, not only can complexity be tackled, but the registry can be used to search for suitable data sources and tools within that network.




You will encounter workflow in the following portal pages: Workflow Editor, Job Monitor and Home Page.



We've racked up the investment both in skills and in intellectual capital. The price and the reward of progress I'm afraid.

The Registry

Collections of astronomical data are diverse in size, content, location and often in data format. The amount of data available is forever growing, and at an ever increasing rate. Tools that are used to extract, analyse and process the data have been built up over the years, often in different computer languages, and often across diverse computer operating systems. You're interested in brown dwarfs in one area of the sky? OK. First of all find all the data collections that have relevant data. Then seek authority to search and download to your desktop the data that you need to analyse. Make sure you have a big enough machine both in cpu cycles to process the data and disk space to hold the data. Then organize your research to use the tools available. Perhaps you may need to write your own. You want to repeat a step? Make sure you recorded the details. You want to share your results or your method of work? Make sure the rest of us know the details of your setup. This situation is one that can easily defeat meaningful research. Unfortunately computer systems can hide interesting data almost as well as the universe hides its secrets. The registry is the IVOA answer.



The IVOA movement has spawned attempts to standardize the holding and processing of astronomical data, and specifically to record and publish details of the data (metadata, or data about data) in a shared registry. Astrogrid attempts to use the registry for recording, searching and helping to process astronomical data, no matter where the data is held or where the tools are located. It is an absolutely necessary adjunct to the distributed workflow paradigm.



You will encounter the Registry in the following portal pages: Resource Browser, Workflow Editor and Query Editor.

ADQL

ADQL stands for Astronomical Data Query Language. ADQL - as the name implies - is a variant of SQL, which is the de facto standard for query languages in relational databases. ADQL represents an initiative within astronomy to produce a specifically astronomical standard query language with a relational intent. Astrogrid uses ADQL for querying astronomical data collections. As a user some understanding of ADQL is required in order to format queries.



You will encounter ADQL in the following portal pages: Query Editor.

MySpace

MySpace is the virtual file system used by Astrogrid. A lot of the artifacts used by workflow are held within MySpace. A user can look upon their home folder within MySpace as a distributed file system, organizing subfolders, saving workflows and queries, writing results (perhaps Votables) to MySpace, uploading and downloading files from/to their desktop if they so wish. However, the critical background fact to remember is that MySpace is used as a conduit for data flows within our use of workflow.



You will encounter MySpace in the following portal pages: MySpace Explorer, Query Editor and Workflow Editor.

Go to Contents

Portal Pages

The Astrogrid portal consists of seven main pages, listed below. As soon as you logon successfully your home page is displayed. The home page and all subsequent main pages have a navigation bar at the top which enables movement between the pages. NB: some screen prints might be useful here, including one simply of the navigation bar.

  • Logon
  • Home
  • MySpace Explorer
  • Resource Browser
  • Query Editor
  • Workflow Editor
  • Job Monitor

Go to Contents

Supported Browsers

These are the currently supported browsers. You will need to enable cookies and javascript.

  • Mozilla 1.6 and above
  • Firefox 1.0 and above
  • Internet Explorer 5 and above. NB: Seems to be a bug in 6 regarding displaying the properties page for a file in MySpace.

Go to Contents

Getting a Userid

You require a userid. All the important facilities within Astrogrid involve saving something somewhere - workflows, queries, jobs, configuration files, results. The userid provides a hook for the ownership of these resources. To gain a userid you need to belong to a community: it is one of the fields that has to be typed in alongside your userid and password when you logon. A community is just a convenient administrative grouping of users around an institution that owns underlying computer resources (eg: the Dept of Physics and Astronomy at the University of Leicester). We could easily have a heated debate about what constitutes a community, so I'll leave it at that. Choose the community you wish to join and from the logon screen you can send an email to the system administrator requesting a userid. A userid and password will be sent to you by email. Belonging to a community does not necessarilly place boundaries on what investigations you can undertake. As I said earlier, a community is just a convenient administrative grouping. When you receive a userid, the system administrator should also have allocated you file space within the MySpace virtual file system. Without this your logon may succeed but you wont be able to do a great deal and may be subject to unexpected failures.

Go to Contents

Getting Started

A successful logon will place you within your home page. The home page provides hooks to acquire the new desktop applications. If this is your first time in the portal, it may be worth leaving that aspect of the page to later. You will notice on the right hand side of the page a rather blank area with the title "Current Jobs". Usually it will display brief details of those workflows recently submitted for execution: the idea is that you see these immediately you logon. If you also visit the Job Monitor page,where complete details of all jobs are displayed, you will have a similarly blank experience until you have submitted a workflow for execution. Within the Help box on the left hand side of the home page there are the Portal Usage and Cook-book topics which are specifically aimed at portal usage. The navigation bar at the top of the page there also contains a Help button which displays a number of links to help topics. For your very first entry into the portal, it is a good idea simply to navigate between the pages and have a brief look at each in turn. I would certainly look at the MySpace Explorer, since it is here all your personal resources will reside. Try creating some subfolders to your home folder, uploading files into them from your desktop (don't make them too large!), and navigating between folders by point and click. The experience should be reasonably familiar to users of a graphical desktop. I would then have a look at exploring the registry using the Resource Browser (the Resources button in the navigation bar and then the Catalog Search within the page itself). You can have plenty of fun trying to find suitable catalogs; fairly normal first experiences are either to find nothing or to find far too much, but in any case interacting with the rather powerful set of selection controls is itself an interesting experience. The Resource Browser is currently purely for exploring what is available for use within the VO. If you want to get serious, you must delve into the Query Editor and the Workflow Editor. Start with building a simple query: use one of the examples on the Query Editor page. Then try selecting a table: this will place you into one of the minor pages where you can interact with the Registry to choose a table from a suitable astronomical data source. Try the Workflow Editor and build a simple one step workflow. But now we are straying into areas which are beyond just getting started. You need then to look at the individual user guides for each of these areas.

Go to Contents