Hudson Installation Procedures

Overview


Hudson is a Continuous Integration (CI) server which polls a version control repository and runs an automated build soon after a change is detected in the version control repository. Hudson organizes CI jobs into projects. Each project has its own build script and its build results are stored within Hudson's work area.



This page lists the steps necessary to configure and operate Hudson at CBIIT. The steps can be summarized as

  • Prepare your project to use BDA's Hudson management scripts
  • Create a CI user account on the Linux machine assigned as the Hudson host
  • Configure CI user's environment
  • Obtain the BDA Hudson configuration scripts


Prepare Your Project for Hudson CI


These scripts will help you install and configure Hudson automatically.

  1. Obtain the BDA Hudson configuration (hudson-manager) scripts by checking out the following two files from BDA's SVN to your project's private workspace, by typing the following:
    svn co http://gforge.nci.nih.gov/svnroot/automation/trunk/software/hudson-manager/
    There are two files of note:
    • Build File - The Hudson management build script. Installs and configures tomcat/hudson, copies a Hudson installation config to SVN and copies Hudson configuration from SVN to a Hudson installation
    • Properties File - The Hudson management properties file
  2. Add the two files to your project's SVN repository (For example, PROJECT_ROOT/tools/hudson-manager)
  3. Add a config directory under the location where you stored the two files above (For example, PROJECT_ROOT/tools/hudson-manager/config). This will be used to store configurations once defined in the Hudson CI machine
    Remove the .svn directory from your PROJECT_ROOT/tools/hudson-manager private workspace prior to committing Hudson files to SVN.
  4. Commit all new files/directories to your project's SVN repository.


Install Hudson


The following instructions assume a (nearly) blank Linux environment with only Ant, Java SDK and perhaps a database installed.


Create CI User


  1. Login with your user credentials, as supplied by the CBIIT Systems Team
    Your user must have sudo rights in the CI box
  2. Type
    sudo bash

    to obtain temporary superuser rights

  3. In this step you'll create a CI user. Since Hudson will need large disk space, make sure to set user's home to a partition with enough free space. Type:
    df -h

    Select a suitable folder for the CI user's home. From the df output below, it is apparent that /usr/local will be a good home candidate for the CI user.

    -bash-3.00$ df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/roog-rvol 7.9G 3.8G 3.8G 51% /
    /dev/sda1 99M 23M 71M 25% /boot
    none 1.9G 0 1.9G 0% /dev/shm
    /dev/mapper/roog-lvol 16G 247M 15G 2% /local
    /dev/mapper/roog-cvol 20G 4.3G 15G 24% /usr/local
    /dev/mapper/roog-rvol 4.0G 148M 3.6G 4% /var
    cbionfsb:/nfsb/admin 185G 176G 9.2G 96% /admfs
    cbionfsc:/nfsc/admrch 50G 30G 20G 61% /admarch
    /nfsa/home/duvallp 785G 744G 41G 95% /h1/duvallp
    /nfsa/home/duvallp 785G 744G 41G 95% /h1/duvallp
    1. To create the user, type:
      # login as your self
      sudo bash
      mkdir /local/home
      /usr/sbin/useradd -d /local/home/hudsonuser hudsonuser
  4. To change the CI user's password, type:
    passwd hudsonuser

    and enter a password.



Configure CI User's Environment


  1. Login as the CI user (the Shell account you created above (e.g. hudsonuser)
  2. Add the following code to the .bash_profile file, by typing:
    vi .bash_profile
    Validate that folders listed below reflect what's in your system. If not, apply changes as necessary.
    ANT_HOME=/usr/local/ant-1.7.0
    JAVA_HOME=/usr/jdk1.5.0_10/
    GLOBUS_LOCATION=/usr/local/ws-core-4.0.3
    MAVEN_HOME=/usr/local/apache-maven-2.0.8
    HUDSON_HOME=$HOME/hudson_data
    export ANT_HOME JAVA_HOME GLOBUS_LOCATION MAVEN_HOME HUDSON_HOME
    export PATH=$ANT_HOME/bin:$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH
    # From the command line, set this to allow FireFox to run on xframebuffer for Selenium tests.
    export DISPLAY=:1.0
  3. Source the profile to setup the environment, by typing the following from the command line.
    $ . .bash_profile
  4. Log off and login. Validate the setup by typing java -version and ant -version. Both commands should return meaningful results.


Checkout Project's Hudson Management Scripts into the CI Machine


  1. Login to the Hudson machine as the CI user you created above (i.e. hudsonuser)
  2. cd
  3. mkdir hudson
  4. cd hudson
  5. mkdir src
  6. cd src
  7. svn co http://gforge.nci.nih.gov/svnroot/*_yourproject_*/trunk/tools/hudson-manager
    This is a sample output generated by the svn co command
    [hudsonuser@xxxxxx-xxxxx src]$ svn co http://gforge.nci.nih.gov/svnroot/automation/trunk/software/hudson-manager
    A hudson-manager/install.properties
    A hudson-manager/bash_profile
    A hudson-manager/build.xml
    Checked out revision 653.


Review the Hudson Installation Properties


  1. Login to the Hudson machine as the CI user you created above (i.e. hudsonuser)
  2. cd
  3. cd hudson/src/hudson-manager
  4. Edit install.properties (original file) as necessary
    The following properties from install.properties may require your special attention
    application.base.path=${user.home}/hudson/application
    hudson.dir=${user.home}/hudson/data
    backup.dir=${user.home}/hudson/backup

    A complete list of properties is provided in the table below.

    Property Description Value
    application.base.path Directory where Tomcat is extracted to ${user.home}/hudson/application
    binary.repository.url Base Url for downloading Tomcat http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk
    tomcat.binaries.file Tomcat file Name jakarta-tomcat-5.0.28.zip
    tomcat.src.url Absolute path for download file ${binary.repository.url}/techstack-2006/os-independent/${tomcat.binaries.file}
    tomcat.binaries.relative.dir Root directory of extracted Tomcat jakarta-tomcat-5.0.28
    tomcat.home Tomcat Home Dir ${application.base.path}/${tomcat.binaries.relative.dir}
    tomcat.port.http Tomcat HTTP Port 48080
    tomcat.port.ssl Tomcat SSL port 48443
    tomcat.port.ajp Tomcat AJP port 48009
    tomcat.port.shutdown Tomcat Shutdown port ? 48005
    hudson.base.url Base URL for Hudson ${binary.repository.url}/other/os-independent/hudson
    hudson.plugin.list List of Plug-ins to download build-timeout.hpi,checkstyle.hpi,cobertura.hpi,findbugs.hpi,hudsontracplugin.hpi,jabber.hpi,pmd.hpi,scp.hpi,svncompat13.hpi,svn-tag.hpi,violations.hpi,vmware.hpi,xvnc.hpi,findbugs.hpi
    hudson.file.name Hudson War File hudson.war
    hudson.dir Hudson home dir ${user.home}/hudson/data
    backup.dir Directory where backups will be backed up to and restored from ${user.home}/hudson/backup


Run the Hudson Installation Script


  1. Login to the Hudson machine as the CI user
  2. cd
  3. cd hudson/src/hudson-manager
  4. Type ant install
    If everything goes well the Ant script will download, configure and launch Hudson. You should see the following output:
    install:tomcat:configure:

    install:tomcat:



    retrieve:hudson:
    [echo] Checking for hudson hudson.war
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/hudson.war
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/hudson.war
    [echo] Checking for plugin checkstyle.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/checkstyle.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/checkstyle.hpi
    [echo] Checking for plugin findbugs.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/findbugs.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/findbugs.hpi
    [echo] Checking for plugin pmd.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/pmd.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/pmd.hpi
    [echo] Checking for plugin scp.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/scp.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/scp.hpi
    [echo] Checking for plugin svncompat13.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/svncompat13.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/svncompat13.hpi
    [echo] Checking for plugin svn-tag.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/svn-tag.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/svn-tag.hpi
    [echo] Checking for plugin violations.hpi
    [get] Getting: http://gforge.nci.nih.gov/svnroot/commonlibrary/trunk/other/os-independent/hudson/violations.hpi
    [get] To: /usr/local/hudsonuser/hudson/src/hudson-manager/target/working/violations.hpi

    install:hudson:binaries:
    [copy] Copying 1 file to /usr/local/hudsonuser/hudson/application/jakarta-tomcat-5.0.28/webapps
    [mkdir] Created dir: /usr/local/hudsonuser/hudson/data/plugins
    [copy] Copying 7 files to /usr/local/hudsonuser/hudson/data/plugins

    install:hudson:configure:

    install:hudson:

    start:tomcat:
    [echo] Starting Tomcat instance at /usr/local/hudsonuser/hudson/application/jakarta-tomcat-5.0.28

    install:
    [echo] You can access it at http://localhost:48080/hudson

    BUILD SUCCESSFUL
    Total time: 45 seconds
    [hudsonuser@xxxxxx-xxxxx hudson-manager]$
  5. Verify the installation by opening a browser and navigating to http://YOUR_HUDSON_HOSTNAME:48080/hudson. You should see the Hudson dashboard with this message: Welcome to Hudson! Please create new jobs to get started.


Setting Hudson to start at boot


  • Log into server as your user account
  • Switch to root
    sudo bash
  • cd to start about boot directory
    cd /etc/init.d
  • create a file called tomcat with the following contents (vi or pico or emacs?)
    #!/bin/bash
    #
    # tomcat
    #
    # chkconfig: - 71 43
    # description: Start up the Tomcat servlet engine.

    # Source function library.
    . /etc/init.d/functions

    RETVAL=$?
    CATALINA_HOME="/local/home/hudsonuser/hudson/application/apache-tomcat-5.5.20"

    case "$1" in
    start)
    if [ -f $CATALINA_HOME/bin/startup.sh ];
    then
    echo $"Starting Tomcat"
    /bin/su - hudsonuser $CATALINA_HOME/bin/startup.sh
    fi
    ;;
    stop)
    if [ -f $CATALINA_HOME/bin/shutdown.sh ];
    then
    echo $"Stopping Tomcat"
    /bin/su - hudsonuser $CATALINA_HOME/bin/shutdown.sh
    fi
    ;;
    *)
    echo $"Usage: $0 {start|stop}"
    exit 1
    ;;
    esac

    exit $RETVAL


  • Change permisions on file
    chmod 750 tomcat
  • Link your script (manually or using chkconfig) to the runlevel you want it to start at:
    • cd /etc/rc3.d
      ln -s ../init.d/tomcat S71tomcat

      or

      /sbin/chkconfig --add hudson
      /sbin/chkconfig hudson on


http://www.raibledesigns.com/tomcat/boot-howto.html


Create a New Hudson Job


  1. Open a browser at http://YOUR_HUDSON_HOSTNAME:48080/hudson
  2. Click New Job from the menu on left
  3. Give a name to the new job (all lowercase, no spaces)
  4. Select Build a free-style software project for Ant projects and Build a maven2 project for Maven2 projects
  5. Click OK
  6. In the following job configuration screen, locate the Source Code Management section and enter the project SCM properties, i.e. SVN URL, Local module, etc.
  7. Similarly, modify other properties such as which artifacts to store
    The following is a non-exhaustive list of the job properties as they appear on the Hudson configuration page. Use this is a guide to edit your own project.
    • Project Name
    • Description
    • Advanced Project Options
      • Quiet Period - 60 (seconds)
    • Source Code Management
    • Build Triggers
      • Poll SCM - Schedule * * * * *
      • Build Periodically - Schedule 0 * * * *
    • Build
      • Invoke Ant
      • Ant Version- Select version from pull down
      • Targets - build:all static:analysis
    • Email Notifications
      • Recipients - yoru list
      • Send email for every unstable build - checked
      • Send separate emails to individuals that broke the build
    • Publish PMD Analysis Results (checked)
      • PMD Results - projectname/reports/pmd/*/.xml
    • Publish CheckStyle Analysis Results (checked)
      • PMD Results - projectname/reports/checkstyle/*/.xml
    • Publish FindBugs Analysis Results (checked)
      • FindBugs Results - projectname/reports/findbugs/*/.xml
    • Report Violations (checked)
      • Failure counts and how to set them
        • Sunny
          • Set this value to your target failure rate
          • Set it to at least 1 above your current failure rate
          • Failure counts higher than this but below stormy will show as cloudy on the dashboard
          • This does not affect build status
        • Stormy
          • Set this value to your target failure rate
          • Set it to at least 1 above your current failure rate
          • Failure counts higher than this but below unstable will show as stormy on the dashboard
          • This does not affect build status
        • Unstable
          • Set this value to the your target failure rate to break the build
          • Set it to at least 1 above your current failure rate
          • Failures greater than this will cause the build status show unstable (which indicates a failed build)
      • Xml patterns
        • checkstyle - projectname/reports/checkstyle/*/.xml
        • findbugs - projectname/reports/findbugs/*/.xml
        • pmd - projectname/reports/pmd/*/.xml
        • simian - projectname/reports/simian/*/.xml
  8. Click Save


Setup database



Lower Case Table names


  1. ssh into the machine as your account
  2. sudo bash # provide password
  3. /etc/init.d/mysql.server stop # stop db
  4. edit /etc/my.cnf - add 'lower-case-table-names=1' under '[mysqld]' section
  5. /etc/init.d/mysql.server start # start db


Privileged user


When you receive the CI server the mysql user will not have a password set

  1. Set a password on the mysql database user
    1. /usr/bin/mysqladmin -u mysql password 'yourpassword'
  2. Verify the password
    1. mysql -u mysql -p # enter password from above when prompted



Configuring Hudson


  • Launch browser to http://localhost:48080/hudson
  • Manage Hudson System Configuration and set (as defined above)
    • ANT HOME - /usr/local/ant-1.7.0
    • JAVA HOME - /usr/jdk1.5.0_10/
    • Maven - /usr/local/apache-maven-2.0.8
    • CVS - copy from other user to /var/lib/hudson/.cvspass, change owner
    • Mail
      • SMTP server - mailfwd.nih.gov
      • Default user e-mail suffix- @mail.nih.gov
      • System Admin E-mail Adresss - buildmaster@cbv-ciweb-base.nci.nih.gov


SVN and Hudson configuration


  • Create a hudson-manager area in your scm repository
  • Copy Hudson configuration to svn - 'ant -Dsvn.user.name=svnusername -Dsvn.user.pass=svnpassword'
  • Copy Hudson configuration from svn - 'ant -Dsvn.user.name=svnusername -Dsvn.user.pass=svnpassword deploy2hudson'


Hudson Management Files



Build File


  • Build Script Targets
    • save-hudson-configs
      • init - checks that user name and password are passed in as arguments
      • copy2repo- copies files from Hudson to svn directory
      • svn-add - svn --force add files to svn (will add missing files)
      • svn-commit - commits the files to the repository
    • deploy2hudson - copies files from svn to hudson directory
    • install - wrapper target to install/configure tomcat/hudson based on properties form prop file below
      • install:tomcat - Wrapper target that installs and configures Tomcat
        • init
        • retrieve:tomcat - Retrieves Tomcat distribution from binary repository based on properties, if file is not present
        • install:tomcat:binaries - Extracts Tomcat distribution to destination directory
        • install:tomcat:configure - Configures the ports in the Tomcat installation
      • install:hudson - Wrapper target that installs and configures Hudson
        • init
        • retrieve:hudson - Retrieves Hudson war file and plugins (*.hpi) from binary repository
        • install:hudson:binaries - Deploys war and plugins to Tomcat and Hudson directories
        • install:hudson:configure - empty right now
      • start:tomcat - starts tomcat after installation
    • artifacts:backup - backups build artifacts and history
    • artifacts:restore - restores build artifacts from existing backup file


Usage


  • Command line properties (-D_propname_=value)
    • svn.user.name - user to commit files to svn under
    • svn.user.pass - user's password
    • hudson.dir - directory to read or write Hudson configuration
  • Usage
    ant -Dsvn.user.name=[svnusername] -Dsvn.user.pass=[svnpassword] [targetName]


Operating Hudson



Cleaning Files


  • Finding Artifacts*

    Replace jar with whichever file type you're searching


find /builds/*/archive -type f  -name ".jar"

Reading Hudson RSS Feeds from Firefox


You can use Hudson's nifty RSS feeds to get notified of any failures or get updates of all builds. When you couple RSS feeds with Firefox's Live Bookmarks you don't even have to visit Hudson's page to get a quick status of the builds.