ROSE Compiler Framework/Obtaining ROSE

From Wikibooks, open books for an open world
Jump to: navigation, search


Git Repositories[edit]

ROSE's source files are managed by git, a distributed revision control and source code management system. There are several ways to download the source tree:

Internal Repos[edit]

  • Private Git Repositories Within LLNL
    • Private git repository hosted within Lawrence Livermore National Laboratory: the internal file path is /usr/casc/overture/ROSE/git/ROSE.git: central repository of ROSE, in most cases this is automatically updated by Jenkins only after incoming commits pass all regression tests
    • Private git repository hosted by github.llnl.gov: used for daily pushes and code review, accessible in LC. LC does not permit SSH, so you'll have to use HTTPS to clone repos there.

Note: You may encounter SSL certificate problems. If you do, simply disable SSL verification in cURL using either export GIT_SSL_NO_VERIFY=false or configuring git:

 $ git config --global http.sslVerify false


HTTP 401 is unauthorized access. If you're not prompted for your username/password in the Shell, then you need to use X-forwarding so the authentication windows will popup:

 $ ssh -X <LC machine>

Otherwise, create a ~/.netrc file with:


machine github.llnl.gov
  login <username>
  password <password>

EDG 4.x internal branch[edit]

We are working on a branch using more recent version of EDG frontends to have better C++ support.

If you work from a workstation from your office. You should generate your public-private key pair and send Justin your public key for authentication. After that, try

 git clone rose-dev@rosecompiler1:rose/edg4x/rose.git

If you work from an LC machine, try to use your OUN name via https (SSH is restricted) and make sure you have git 1.7.10 or later.

 git clone https://OUN@rose-github.llnl.gov/rose-compiler/edg4x-rose

You may have to modify the URLs to submodules if you clone ROSE from LC machines.

aztec1@too1:git clone https://rose-github.llnl.gov/rose-compiler/edg4x-rose.git
Initialized empty Git repository in /tmp/edg4x-rose/.git/
remote: Counting objects: 184706, done.
remote: Compressing objects: 100% (46828/46828), done.
remote: Total 184706 (delta 133529), reused 184562 (delta 133385)
Receiving objects: 100% (184706/184706), 242.37 MiB | 25.92 MiB/s, done.
Resolving deltas: 100% (133529/133529), done.

aztec1@too1:cd edg4x-rose/

aztec1@too1:vi .gitmodules

aztec1@too1:cat .gitmodules
[submodule "src/frontend/CxxFrontend/EDG"]
        path = src/frontend/CxxFrontend/EDG
        url = https://rose-github.llnl.gov/rose-compiler/edg.git
[submodule "projects/vulnerabilitySeeding"]
        path = projects/vulnerabilitySeeding
        url = https://rose-github.llnl.gov/rose-compiler/vulnerabilitySeeding.git

aztec1@too1:git submodule update --init
Submodule 'projects/vulnerabilitySeeding'
(https://rose-github.llnl.gov/rose-compiler/vulnerabilitySeeding.git)
registered for path 'projects/vulnerabilitySeeding'
Submodule 'src/frontend/CxxFrontend/EDG'
(https://rose-github.llnl.gov/rose-compiler/edg.git) registered for path
'src/frontend/CxxFrontend/EDG'
Initialized empty Git repository in
/tmp/edg4x-rose/projects/vulnerabilitySeeding/.git/
remote: Counting objects: 5992, done.
remote: Compressing objects: 100% (1121/1121), done.
remote: Total 5992 (delta 3178), reused 5992 (delta 3178)
Receiving objects: 100% (5992/5992), 958.13 KiB, done.
Resolving deltas: 100% (3178/3178), done.
Submodule path 'projects/vulnerabilitySeeding': checked out
'847d1d884c8b5098b4b2895259058f91f98a525e'
Initialized empty Git repository in
/tmp/edg4x-rose/src/frontend/CxxFrontend/EDG/.git/
remote: Counting objects: 3351, done.
remote: Compressing objects: 100% (1113/1113), done.
remote: Total 3351 (delta 2574), reused 2939 (delta 2167)
Receiving objects: 100% (3351/3351), 23.01 MiB | 10.51 MiB/s, done.
Resolving deltas: 100% (2574/2574), done.
Submodule path 'src/frontend/CxxFrontend/EDG': checked out
'8950d363a12d1071f600c1b51c9a6d2b41b0dae1'

aztec1@too1:echo $?
0

Public repos[edit]

EDG 4x Public Branch[edit]

The ROSE team has been working on a new EDG4x branch of ROSE from 2012-2013. This branch adds a great deal of robustness, especially for the C/C++ language support. This work is currently in a separate repository, publicly accessible in Github:

   https://github.com/rose-compiler/edg4x-rose

A note about this has also been added to the official ROSE website:

   http://rosecompiler.org/?page_id=14

We hope to have this new branch merged into the mainline soon, so stay tuned!

Here are some rough instructions to get you started with building and installing ROSE on your Linux machine:

1. Install the Boost C++ libraries. You can use my quick-and-dirty script: install-boost.sh.

2. Configure your environment

3. Run the $ROSE/build script to perform the Autotools bootstrap

4. Run the $ROSE/configure script with "--with-boost=/path/to/boost", "--prefix=/path/for/installation". Optionally, you should configure --with-gomp_omp_runtime_library=/usr/apps/gcc/4.4.1/lib64/ if you want to play with OpenMP support of ROSE and link with GCC 4.4's OpenMP runtime library.

5. Run "make –j24 install-core" to build and install ROSE

Please note that we only support GCC 4.2.4 and GCC 4.4.5 currently on the ROSE-EDG4 repository. Full documentation is available on our website, http://rosecompiler.org, though instructions specific to ROSE-EDG4 are generally lacking.

Current Limitations[edit]

Currently, the EDG4x branch of ROSE is supporting GCC version 4.0 to 4.8.

Autoconf Configuration[edit]

The EDG4.7 frontend is enabled by default, you can just configure ROSE by using the following options

   $ $ROSE/configure --with-boost=/path/to/boost/installation --prefix=/path/for/rose/installation

Example Environment[edit]

   # Example setup.sh file to source relevant environment settings for building ROSE
   #
   # Update the paths to fit your environment…
   export BOOST_ROOT=/home/too1/local/opt/boost/1_45_0
   export BOOST_HOME="${BOOST_ROOT}"
   export JAVA_HOME=/home/too1/local/workspace/rose/edg3/keep_going/opt/java/jdk/1.7.0_15
   export LIBTOOL_HOME=/usr/apps/libtool/2.2.4
   export AUTOMAKE_HOME=/usr/apps/automake/1.9.6
   export LD_LIBRARY_PATH=":${JAVA_HOME}/lib:${JAVA_HOME}/jre/lib/amd64/server:${BOOST_ROOT}/lib:"
   export PATH="${JAVA_HOME}/bin:${AUTOMAKE_HOME}/bin:${LIBTOOL_HOME}/bin:$PATH"
   export PATH="/usr/apps/git/latest/bin:$PATH"
   # GCC 4.4.1
   source /nfs/apps/mpfr/2.4.1/setup.sh
   source /nfs/apps/gmp/4.3.1/setup.sh
   source /nfs/apps/gcc/4.2.4/setup.sh

Virtual Machine Image[edit]

It can take quite some time to install ROSE for the first time. A virtual machine image is provided with a Ubuntu 10.04 OS with ROSE already installed.

You can download it and play it using VMware Player

Downloading The Virtual Machine Image[edit]

More information is at ROSE Virtual Machine Image

git 1.7.10 or later for github.com[edit]

github requires git 1.7.10 or later to avoid https cloning errors, as mentioned at https://help.github.com/articles/https-cloning-errors

Ubuntu 10.04's package repository contains git 1.7.0.4. So building later version of git is needed. But you still need an older version of git to get the latest version of git.

 apt-get install git-core

Now you can clone the latest git

 git clone https://github.com/git/git.git

Install all prerequisite packages needed to build git from source files(assuming you already installed GNU tool chain with GCC compiler, make, etc.)

 sudo apt-get install gettext zlib1g-dev asciidoc libcurl4-openssl-dev
 $ cd git  # enter the cloned git directory
 $ make configure ;# as yourself
 $ ./configure --prefix=/usr ;# as yourself
 $ make all doc ;# as yourself
 # make install install-doc install-html;# as root

EDG source code[edit]

If you have an EDG license, we can provide you with ROSE's EDG source code. The original, official EDG source code does NOT work with ROSE since we have modified EDG to better serve our purposes.

Note: We provide you with a snapshot of our Git revision controlled ROSE-EDG source code repository. This way, you can more easily contribute your EDG modifications back into ROSE.

1. Send your EDG (research) license to two ROSE staff members, just in case one is on vacation or on travel.

2. Provide ROSE staff with a drop-off location for the EDG source code (ssh or ftp server, etc.)

3. Once you receive the EDG source code, you have two options:

As a submodule[edit]

a. Use ROSE-EDG as a submodule (assuming you have ROSE's Git source tree):

This is the recommended way to use the EDG git repo we provide. So the assumption is that you use a local git clone of ROSE($ROSE).

Edit submodule path in $ROSE/.gitmodules to point to your ROSE-EDG repository:

[submodule "src/frontend/CxxFrontend/EDG"]
       path = src/frontend/CxxFrontend/EDG
-       url = ../ROSE-EDG.git
+       url = <path/to/your/ROSE-EDG.git>
-[submodule "projects/vulnerabilitySeeding"]
-       path = projects/vulnerabilitySeeding
-       url = ../vulnerabilitySeeding.git

Run git-submodule commands:

$ cd $ROSE
$ git submodule init
$ git submodule update

The commands above will check out a version of the EDG submodule and save it into ROSE/src/frontend/CxxFrontend/EDG

As a Drop-in[edit]

b. As a Drop-in

Move ROSE-EDG tarball into its correct location within the ROSE source tree: $ROSE/src/frontend/EDG

  $ tar xzvf ROSE-EDG-b12158aa2.tgz
  $ ls
  EDG  ROSE-EDG-b12158aa2.tgz
  $ mv EDG $ROSE/src/frontend/EDG

Warning: This method may not work because EDG is a submodule of ROSE and therefore, requires a version synchronization between the two. For example, the latest version of ROSE may not use the latest version of ROSE's EDG.

The remaining steps[edit]

4. In ROSE, run the $ROSE/build script from the top-level of the ROSE source tree, i.e. $ROSE. This script bootstraps Autotools, including the Makefile.ams in the EDG source tree.

5. Configure and build ROSE: Normally, during this process ROSE would attempt to download an EDG binary tarball for you, but since you have the source code, this step will be skipped.


EDG tarball[edit]

Process[edit]

If you don't have access to the EDG source code, you will be able to automatically download a packaged EDG binary tarball during the ROSE build process. The download is triggered during make in $ROSE_BUILD/src/frontend/CxxFrontend.

The EDG binary version is a computed binary compatibility signature relative to your version of ROSE. You can check this version by running the $ROSE/scripts/bincompat-sig, for example:

$ ./scripts/bincompat-sig 
7b1930fafc929de85182ee1a14c86758

You may encounter this error:

$ ./scripts/bincompat-sig 
Unable to find a remote tracking a canonical repository.  Please add a
canonical repository as a remote and ensure it is up to date.  Currently
configured remotes are:

   origin => https://github.com/rose-compiler/rose

Potential canonical repositories include:

   anything ending with "rose.git" (case insensitive)

If you do, simply add ".git" to the end of your origin's URL path. In our example, this translates to:

https://github.com/rose-compiler/rose.git

List of binaries[edit]

View the list of available EDG binaries here: http://www.rosecompiler.org/edg_binaries/edg_binaries.txt.

EDG binaries are generated for these platforms (Last updated on 12/22/2012):

Platform EDG 3.3 EDG 4.0
amd64-linux GCC 3.4.6, 4.0.4, 4.1.2, 4.2.4, 4.3.2, 4.4.1 GCC 3.4.6, 4.0.4, 4.1.2, 4.2.4, 4.3.2, 4.4.1
i686-linux GCC 3.4.6, 4.0.4, 4.1.2, 4.2.4, 4.3.2, 4.4.1 GCC 3.4.6, 4.0.4, 4.1.2, 4.2.4, 4.3.2, 4.4.1
32bit-macos-10.5 GCC 4.0.4
64bit-macos-10.6 GCC 4.2.4
64bit-x86_64-macos-10.6 GCC 4.2.4
34bit-debian GCC 3.4.6, 4.0.4, 4.1.2, 4.2.4, 4.3.2, 4.4.1 GCC 3.4.6, 4.0.4, 4.1.2, 4.2.4, 4.3.2, 4.4.1