AstroGrid DataCenter

Follow the links on the left to subprojects for

  • client delegates
  • server engine
  • backend plugin implementations
  • Publishers Astrogrid Library

Build

The datacenter build is quite byzantine - partly because it does a lot, partly to work-around maven flaws and bugs. Is should be possible to rationalize it a bit, but its probably best to wait for the proper maven v1.0 release.

Project Structure

The datacenter is made up of a set of sub-projects. Each is in a separate subdirectory - /client, /server, /pal, /implementations/*.

Each subproject requires a project.xml project.properties and maven.xml. To save replication and simplify modifications, each project extends templates in the /build directory. Common resources - such as checkstyle and license files - can also be shared between projects by placing them in build directory.

Nightly Build

This section is now out of date and will be updated shortly

The nightly build script calls the following maven goals

  1. java:compile - compile up sources
  2. clover:html-report - dunno. probably a work-around for a bug
  3. site - build project website
  4. site:fsdeploy - pubish project website on server
  5. jar:depoy-snapshot - generate jar, publish on central repository
  6. war:deploy-snapshot - generate war, publish central repository

These work fine for building a single project. However, the datacenter comprises a group of separate sub-projects - each with their own maven project.xml files,etc. There are dependencies between the sub-projects, so they need to be run in the correct order.

The subprojects are tied together by the top-level datacenter project (that's the one you're reading now). This contains no code, provides a bit of additional site documentation, and the maven script that binds the sub-projects together.

Building the correct sequence of sub-projects is shoe-horned into the nightly build by defining 'preGoals' for the important goals in the list above. These are defined for this top-level project in (datacenter/maven.xml)

These pregoals use the multiproject plugin, or the reactor tag - both do the same thing - attain a single goal in a group of interrelated-projects. Furthermore, if subproject A has a dependency on subproject B, multiproject/reactor ensures that the goal is executed on B before A.

What really happens

So what really happens during the nightly build is:

  1. java:compile - Fails, as its being run in the top-level datacenter project, where there's no code. we don't care.
  2. clover:html-report - Fails with errors. We don't care.
  3. site - Builds a simple site based only on the top-level project. This is later overwritten
  4. site:fsdeploy - has a pregoal to attain multiproject:site. This builds a composite project website site from all the sub-projects. It is this website that is then deployed to the webserver by the site:fsdeploy
  5. jar:deploy-snapshot - has a pregoal that runs jar:deploy-snapshot,dist for each of the subprojects (except pal):        
    • jar:deploy-snapshot generates a jar for the subproject and places it in the central repository.
      •        
    • dist generates source and binary zipfiles containing jars, sources and javadoc for each subproject.
    The pregoal then calls goals in the pal project
             
    • webapp-site - fills in webapp configuration files, generates pal webapp documentation
      •        
    • war:deploy-snapshot - assembles the war file, publishes on central repository. As part of the assembly, the source and binary zipfiles created by the previous dist goal are grabbed and inserted into the war.
  6. war:deploy snapshot - this runs in the top level project. there's no sources here, so it just generates an empty war file, which is published to the central server.

In summary, the main goals are site, followed by jar:deploy-snapshot

Doing it better

Clearly, there's a lot of redundancy and wasted computatioin here, due to the mis-match between what the nightly build thinks the datacenter project should do, and what we want it to do.

I've been talking to folk about replacing the long list of predefined goals in the nightly build by a list of goals such as:

  1. astrogrid-site
  2. astrogrid-deploy-snapshots
then it would be up to each project to define these goals to do whatever was necessary for itself. This would make the datacenter build clearer and more efficient.

Building by Hand

cd into any sub-project (client,server,implementations/*)and issue commands there to build just that project. They behave just like any other maven project. e.g.

  • cd datacenter/client
  • maven site builds single site for the client project.
  • maven jar:install-snapshot builds snapshot jar for client, places in local repository where it can be found by other projects.

PAL

To build PAL by hand, you need to build all its prerequisites - the jars it depends on, plus the javadocs and dist bundles it grabs. Do the following:

cd datacenter

maven multiproject:clean ; maven multiproject:site ; maven jar:install-snapshot

then go and have a cup of coffee

multiproject:site is optional (IIRC)- pal grabs some extra documentation from the site build for each project. I think it will manage without it though.

jar:install-snapshot has a preGoal in maven.xml, which is similar to that for jar:deploy-snapshot used in the nightly build.

  • it calls jar:install-snapshot dist on each subproject in turn. This places each subproject's snapshot jar in the local repository, where they can be found by the other subproject builds.
  • it then calls the goals in the pal project that grabs the resources, configure and assemble the war file.

The result is a pal-SNAPSHOT, which is left in your local repository - usually ~/.maven/repository/astrogrid-datacenter/wars/pal-SNAPSHOT.war

See documentation for the PAL project for further build instructions.