From Wikibooks, open books for an open world
Jump to navigation Jump to search

Creating a Nightly Build[edit | edit source]

The Nightly Build[edit | edit source]

Prerequisites[edit | edit source]

Terms Used[edit | edit source]

The location the nightly scripts are stored in
<Sourceforge SSH Account Name>
A valid SSH account on sourceforge with access to AROS.
<Nightly-Build User Name>
The local user account you will build binaries with.
<Dev Mailing List eMail Address>
An email account with access to the AROS Dev mailing list.
This will be used to send the log to the mailing list.
<Your Servers Config-Name>

Nightly Package Names[edit | edit source]

The build script uses package names to identify what to build for a given server. the packages tell the build script which files in <SRCDIR>/pkg/ to run, which in turn creates the chosen binaries.

Valid Package Names[edit | edit source]

Files which are common to all platform generally have a name that is just their target, with all as the build target. Currently defined ones are

Common Package Names[edit | edit source]
Arch Specific Package Names[edit | edit source]

Disallowed Names[edit | edit source]

  bootiso                            generic routine used by the arch specific nightly targets

Prepairing the system[edit | edit source]

Firstly you will want to set up a user account on your system that will be used to perform the build. create a user

  adduser <Nightly-Build User Name>

The first step is to obtain the nightly build scripts. cd to the location you will perform the build in and checkout the scripts as follows -:

  cd /home/<Nightly-Build User Name>
  mkdir -p MyNightlyBuildDir
  cd MyNightlyBuildDir
  svn checkout

you will have a new subdirectory called nightly that contains all the scripts used to build the nightly binaries. from here on in we will refer to this directory as <SCRIPTDIR>.

now cd into that directory and make a directory tree to hold the AROS source

  mkdir -p Source
  mkdir -p Source/Contrib
  mkdir -p Source/Documentation
  mkdir -p Source/Necessary

Finaly create an initial cache of the svn files used to build AROS

IMPORTANT: You must set the correct LOCALE for your system at this point!!

type the following -:

  cd Source
  export LANG=en_US.ISO8859-1

then checkout all the source repositories needed to perform your nightly build packages.


  svn checkout
  cd Contrib
  svn checkout
  cd ../Necessary
  svn checkout

You may omit complete contrib if you don't want to bundle it within your archive. Just leave the corresponding directory empty.

Prepairing your servers config file[edit | edit source]

Create a plain textfile in the cfg directory called <Your Servers Config-Name> Open it in a text editor and add the packages you'd like to build.

For example ..

  CFG_PACKAGES=(sources.svn.all documentation.svn.all sdk.svn.pc-x86_64 bootiso-grub2.svn.pc-x86_64)

To enable sending an email report of the build results to the dev mailing list add the following lines ..

  CFG_MAIL_CC=<Dev Mailing List eMail Address>

And to enable uploading the binaries so they may be listed on add ..

  CFG_DS_USER=<Sourceforge SSH Account Name>

Performing your first run[edit | edit source]

The first time you check the system is operational ... perform a manual dry run by doing the following

  cd <SCRIPTDIR>; ./build <Your Servers Config-Name>

And watch as it displays all your build output.

If everything went well and you have enabled the email logging in your server config, you should see a confirmation email on the AROS developer mailing list confirming the build completed successfully. If you have also enabled uploading the files, and have a valid AROS SSH account on sourceforge, the binaries should be uploaded into the temporary storage location ready to be moved into the live folders on the download pages

Setting it up to run automatically[edit | edit source]

You will need to create a cron job for the build step we performed doing the dry run. You will want to add a command that is performed once every 24hrs, which runs the build script with your server's configuration and uploads the created archives to

   0 2 * * * ( cd <SCRIPTDIR> && ./build <Your Server's Config-Name> ) > /dev/null

This will make the build machine attempt to build and upload AROS at 2am. All output will be directed to NULL (except of course the generated logfiles)

Adding the binaries to the site[edit | edit source]

If you are simply providing binaries for existing AROS targets, the previous steps should be enough to add the files to the site. If however you wish to provide new flavours of AROS then you will need to update the site with information about them. Read on...

The scripts and page definitions to list the binaries on are maintained in svn at

To edit the files you'll need to checkout the repository and customise to suit.

listing new downloads[edit | edit source]

If you are providing binaries for AROS targets that don't currently exist, you will need to update the files which list them on

Edit the db/download-descriptions/<Language> files to list descriptions of the packages you build. make sure and follow the naming convention used by the existing packages.

Troubleshooting[edit | edit source]

Corrupt localised filenames in the binaries[edit | edit source]

This usually means the source files used have utf8 filenames in the source repository caches resulting from the initial checkout being done when the system locale was set to utf8.

to fix it change directory into each source directory in turn and delete all files and directories EXCEPT the top level .svn directory.

now type ..

  export LANG=en_US.ISO8859-1

.. and finally do an "svn update" in each directory.

Maintaining the Main Nightly Build Machine[edit | edit source]

The main nightly build machine has the additional responsibility of running scripts to move the uploaded binaries to the live download pages and clean up the upload directory. Other build machines don't need to run these scripts.

Step1[edit | edit source]

Moving the uploaded binaries to the live download pages.

You will need to create a cron job that runs at regular intervals during the day, currently every hour. This job however isn't performed locally, and will require an AROS SSH account on sourceforge. Its goal is to move the archives uploaded by the various build machines into the correct folder on the site so that they become available for download. It also deletes any old versions of these files so that only the most up-to-date version is available (there isn't enough space to keep multiple versions of each file).

   00 * * * * ssh <Sourceforge SSH Account Name> 'bash -l -c aros/scripts/nightly-update' > /dev/null

This will make the server transfer any files present in the upload dirextory to the site once an hour. Again all output is sent to NULL.

Step2[edit | edit source]

This step removes incomplete or broken nightly build files from the upload directory.

   51 7 * * * ssh <Sourceforge SSH Account Name> 'bash -l -c aros/scripts/nightly-clean' > /dev/null

This will clean up the uploads directory on the server at 7:51am each day.