wiki:BuildSystem

Version 24 (modified by welberge, 7 years ago) (diff)

--

The ANT based build system

Starting point

Clone the hmibuild repository (see GitRepositories for repository locations). To make the build system work out-of-the box with our other code, this code must reside in the same top level directory as hmibuild.

e.g.

<gitroot>/hmibuild
<gitroot>/AsapDemo
<gitroot>/HmiDemo
<gitroot>/AsapResource
<gitroot>/HmiResource
...etc

And, if you'd like to compile the Asap/Hmi sources:

<gitroot>/HmiCore
<gitroot>/HmiEnvironment
<gitroot>/Asap
....etc

Goal of the build system

The modules in Asap are dependent on many external and internal libraries, sometimes in different versions. Furthermore, each machine (32bit, 64bit, mac, windows, linux) may require its own version of a specific library. The build system was created to:

  • collect all dependent external libraries, and all internal releases, in a structured repository that allows to keep track of versions and dependencies
  • allow one to resolve, from this structured repository, exactly those versions of dependent libraries that are required for your project, on your machine and your operating system
  • make it easy to release new versions of our own modules
  • make it easy to use, in module A, the current latest compile of module B, instead of the released version of module B

Quickstart: resolving libraries for and running a Java demo

In command prompt the module directory (e.g. AsapDemo/AsapRealizerDemo?):

  • ant resolve to collect all dependent released and external libraries in the lib directory
  • ant main to select the main class
  • ant run to compile and run the project

Creating your own Java project

The build system assumes the following directory layout: yourproject/build.xml (ant build file) yourproject/ivy.xml (specification of jar dependencies) yourproject/build.properties (build properties) yourproject/src (source directory) yourproject/lib (directory for resolved jars (auto filled, do not add custom jars here)) yourproject/lib.local directory (directory for custom jars for your project) yourproject/resource (directory for data resources for your project) yourproject/test/ivy.xml (specification of test specific jar dependencies) yourproject/test/src (test sources) yourproject/test/lib (directory for resolved test jars (auto filled, do not add custom jars here)) yourproject/test/resource (test specific data)

To use the build system with your own Java project, you'll need to set up a build.xml, build.properties and ivy.xml file. The build.xml file provides the project name and normally simply refers to the default build.xml in hmibuild: e.g.:

<?xml version="1.0" encoding="UTF-8"?>
<project name="HmiNeuroPhysicsDemo"  default="run">
   <import file="../../hmibuild/build.xml" />
</project>

The ivy.xml file is used to specify the library dependencies of the project. For example: <ivy-module version="2.0">

<info organisation="HMIDemo" module="HmiNeuroPhysicsDemo?"/> <configurations>

<include file="${ivy.settings.dir}/configurations.xml"/>

</configurations> <dependencies>

<dependency org="HMI" name="HmiRenderEnvironment?" rev="latest.${resolve.status}" /> <dependency org="HMI" name="HmiAnimationEmbodiments?" rev="latest.${resolve.status}" /> <dependency org="HMI" name="HmiAnimation" rev="latest.${resolve.status}" /> <dependency org="HMI" name="HmiGraphics?" rev="latest.${resolve.status}" /> <dependency org="HMI" name="HmiMath?" rev="latest.${resolve.status}" /> <dependency org="HMI" name="HmiNeurophysics?" rev="latest.${resolve.status}" /> <dependency org="HMI" name="HmiUtil?" rev="latest.${resolve.status}" />

<dependency org="jcip" name="jcip-annotations" rev="latest.release" /> <dependency org="slf4j" name="slf4j-api" rev="latest.release" /> <dependency org="logback" name="logback-classic" rev="latest.release" /> <dependency org="logback" name="logback-core" rev="latest.release" /> <dependency org="Sun" name="jlfgr" rev="latest.release" />

</dependencies>

</ivy-module> rev="latest.release" means: resolve the latest version of this external library rev="latest.${resolve.status}" means: resolve the latest version of this library of ${resolve.status} resolve.status is set in the build properties, valid values are beta (for daily release) and release (for stable version).

The full list of available external libraries and Hmi libraries can be found here:  http://hmirepo.ewi.utwente.nl/. Asap's libraries names and jars can be found here: http://asap-project.ewi.utwente.nl/repo/.

The build.properties file specifies several build and run options. For example resource path and runtime arguments for the jvm:

resolve.status=beta
resource.path=${shared.resources}/DefaultShaders/resource;${shared.resources}/LogbackConfigs/resource;${shared.resources}/HmiHumanoidEmbodimentArmandia/resource;
run.jvmargs= -Xms128m -Xmx512m  -Xss5M -Dlogback.configurationFile=LogbackConfigs/warnlogstdout.xml

Use cases

Running and compiling against released versions

  • ant resolve to collect all dependent released and external libraries in the lib directory
  • ant main to set the class that is to be run
  • ant run to compile and run the project

Testing

  • ant junit to compile and run the test cases
  • ant junitreport to compile and run the test cases and generate an html report in the test/report directory

Resolving dependencies

Dependencies are resolved from

  1. the local.lib directory of a project
  2. the local repository directory of a project (can be set up in its repository directory)
  3. the repositories in http://asap-project.ewi.utwente.nl/repo/ and  http://hmirepo.ewi.utwente.nl/

`ant resolve' and similar resolve commands empty the lib directory, so use local.lib to store local libraries instead. Local resources are, by convention, stored in a projects resource directory.

Advanced: new/alternative default repositories can be added in !hmibuild/ivy/ivysettings.xml

Advanced

Running and compiling against source code

It is possible (but not recommended) to run and compile your project against the source of other projects. To this end, define the projects source dependencies using rebuild.list:

e.g.

rebuild.list=Hmi/HmiUtil, Hmi/HmiMath, Hmi/HmiAnimation, Hmi/HmiNeurophysics, Hmi/HmiRenderEnvironment, Hmi/HmiAnimationEmbodiments
  • ant resolve to collect all dependent released and external libraries in the lib directory
  • ant compilelist to compile dependent modules
  • ant main to set the class that is to be run
  • ant run to compile and run the project

Fixing java.lang.OutOfMemoryError: PermGen space

For large (re)builds, especially of HmiDemo projects with lots of dependencies, the default ant settings do not suffice, and ant might crash because of memory problems; typically

java.lang.OutOfMemoryError: PermGen space

This error can be fixed by explicitly setting the permgensize in the ANT_OPTS environment variable. In linux this is done using:

export ANT_OPTS="-XX:MaxPermSize=1024M"

in e.g. ~/.bashrc. In windows,ant calls the antrc_pre.bat file (if it exists) before running ant, so the environment variable can be set there using:

set ANT_OPTS=-XX:MaxPermSize=1024M

ant_rc.bat should be placed in your HOME directory.

Using a local repository

By default, the Hmi, Asap and external libraries are checked out from the https server and it is strongly advised that you resolve them from there. However, if you (temporarily, e.g. at a conference) don't have an internet connection, you can set up the build system to check out from a local directory (here: d:/hmirepo). This is done as follows:

Create a hmi.build.properties file in your home directory and add:

hmi.repository.filesystem.root=d:/hmirepo
repository.mode=filesystem

Copy the hmirepository to your directory of choice. Rsync can be used to create incremental copies:

rsync -v -a --delete loginname@hmirepo.ewi.utwente.nl:/local2/hmirepo .

Compiling to/as a different Java version

In build.properties:

javac.version=1.6 (both source and target are 1.6)

or

javac.target=1.6  (source may e.g. be Java 1.7, target is 1.6 class code)

Releasing external jar files

hmibuild/publishexternal/publish.py is a python script to release external jar files into e.g. hmirepo. This script will create the ivy xml file, setup up the correct directory structure in the repository and copy over a specified jar. The script assumes that a private key for the server is available in ~/.ssh/id_rsa. Windows users will need to install the winrandom and Crypto packages for Python, in linux (at least on the new techfak netboot machines in Bielefeld) it just works. Run the script as follows:

python ./publish.py -v 1.1 -o myorganization -m mymodule --host welberge@hmirepo.ewi.utwente.nl mymodule-1.1.jar

Releasing under a different user name

If you need to release under a different name than your username (e.g. at home), use

publish.user.name=yourpublishusername

in your ~/hmi.build.properties file.

More

Detailed explanation of the build system is still being written. For specific questions, ask below.

Discussion

BuildSystem
 unsolved

Reply here to give additional information or ask questions.