ROSE Compiler Framework/Virtual Machine Image

From Wikibooks, open books for an open world
Jump to: navigation, search

Overview[edit]

The goal of this page is to document

  • How users can download the virtual machine image (or virtual appliance) and use ROSE out of box.

We have three virtual machine images right now

  • V3: the newest VM using Ubuntu 16.04 (Xenial Xerus) with ROSE installed with EDG 4.12 frontend
  • V2: the VM using Ubuntu 14.04 (Trusty Tahr) and ROSE based on EDG 4.x frontend (no longer maintained)
  • V1: the very old VM using Ubuntu 10.04 (Lucid Lynx) and ROSE based on EDG 3.x frontend (no longer maintained)

V3[edit]

Download[edit]

Download the virtual machine image created by using VMware Player:

  • http://www.rosecompiler.org/Ubuntu-ROSE-Demo-V3.tar.gz
  • Warning: it is a huge file of 4.4 GB (20.2 GB if fully uncompressed). It may take ~1 hour to download depending on your High Speed Internet Connection.
  • Demonstration user account (sudo user in Ubuntu):
    • account: demo
    • password: password

Warning: LLNL users may not be able to download it due to limitations to max downloaded file size within LLNL. It may also be against LLNL's security policy to run a virtual machine without authorization. So this image should not be used inside LLNL.


On windows, you can install 7-zip (http://www.7-zip.org/) to untar the tar ball (.tar.gz file) into a folder.

  • It may take ~ 20 minutes on a desktop PC to fully uncompress it in two steps (.tar.gz to .tar, then .tar to the folder)
  • The final folder size is around 20.2 GB

Content[edit]

demo@ubuntu:~$ cat readme

This is a Ubuntu 16.04 virtual machine with the ROSE Compiler installed.

cloned rose-develop on 2/22/2017
version 0.9.7.188

Directory List
~/rose-develop  : git clone https://github.com/rose-compiler/rose-develop
~/build-rose    : build tree of rose
~/opt/rose_inst : installation path of rose
~/tests         : a simple c file, processed by identityTranslator and dotGeneratorWholeASTGraph. 

type zgrviewer -f filename.dot to view a dot file of the AST graph

gcc-4.9.3 is the default gcc

---------- using ROSE ----------

To use the rose translator, you need to first setup the environment.
source ~/set.rose

---------- bashrc ----------

bash env in .bashrc has the following variables by default

# add jdk to PATH and LD_LIBRARY_PATH
PATH=/home/demo/opt/jvm/jdk1.7.0_51/bin:$PATH
LD_LIBRARY_PATH=/home/demo/opt/jvm/jdk1.7.0_51/jre/lib/amd64/server:/home/demo/opt/jvm/jdk1.7.0_51/lib:$LD_LIBRARY_PATH
# add boost to LD_LIBRARY_PATH
LD_LIBRARY_PATH=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default/lib:$LD_LIBRARY_PATH
# create alias for zgrviewer
alias zgrviewer='/home/demo/opt/zgrviewer-0.10.0/run.sh'
export PATH LD_LIBRARY_PATH

---------- configuration of ROSE ----------

CC=/usr/bin/gcc-4.9 CXX=g++-4.9 FC=/usr/bin/gfortran-4.9
CXXFLAGS='-g -rdynamic -Wall -Wno-unused-local-typedefs -Wno-attributes' 
/home/demo/rose-develop/configure 
--enable-assertion-behavior=abort 
--prefix=/home/demo/opt/rose_inst 
--with-CFLAGS=-fPIC --with-CXXFLAGS=-fPIC 
--with-C_OPTIMIZE=-O0 --with-CXX_OPTIMIZE=-O0 
--with-C_DEBUG='-g -rdynamic' --with-CXX_DEBUG='-g -rdynamic' 
--with-C_WARNINGS='-Wall -Wno-unused-local-typedefs -Wno-attributes' 
--with-CXX_WARNINGS='-Wall -Wno-unused-local-typedefs -Wno-attributes' 
--with-ROSE_LONG_MAKE_CHECK_RULE=yes 
--with-boost=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default 
--with-gfortran='/usr/bin/gfortran-4.9' 
--with-python='/usr/bin/python3' 
--with-java=/home/demo/opt/jvm/jdk1.7.0_51/bin/javac 
--enable-languages=all 
--enable-projects-directory 
--with-doxygen 
--without-sqlite3 
--without-libreadline 
--without-magic 
--with-yaml='/home/demo/opt/yaml/0.5.3/boost-1.61.0/gcc-4.9.3-default' 
--with-dlib='/home/demo/opt/dlib/18.18' 
--without-wt 
--without-yices 
--without-pch 
--enable-rosehpct 
--with-gomp_omp_runtime_library=/usr/lib/gcc/x86_64-linux-gnu/4.9/ 
--without-haskell
--enable-edg_version=4.12

make core
make install-core

Installation Notes[edit]

The installation procedures followed directions from http://rosecompiler.org/ROSE_HTML_Reference/installation.html except for changing the gcc version to 4.9.3.

demo@ubuntu:~$ cat installation_notes

ubuntu 64-bit 16.04.1 amd64
made with VMware Workstation 12 player

name: Ubuntu-ROSE-Demo-V3
username: demo
password: password
Virtual machine name: Ubuntu-ROSE-Demo-V3

hard disk: 30 GB, split
memory 4096 MB
processors 2

cloned rose-develop on 2/22/2017
version 0.9.7.188
installed with: 
gcc 4.9.3
Boost 1.61.0
EDG 4.12

-------------------- INSTALLATION PROCEDURES --------------------

>$ sudo apt-get update
>$ sudo apt-get upgrade
>$ sudo apt-get install git wget build-essential libtool automake flex bison python3-dev

---------- change gcc to 4.9.3 ----------

>$ sudo apt-get install gcc-4.9 g++-4.9 gfortran-4.9
>$ sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 100
>$ sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.9 100
>$ sudo update-alternatives --install /usr/bin/gfortran gfortran /usr/bin/gfortran-4.9 100

check that gcc -v says version 4.9.3

---------- install jdk ----------

# download jdk-7u51-linux-x64.tar.gz

>$ mkdir ~/opt/jvm
>$ cd ~/opt/jvm
>$ tar xf ~/Downloads/jdk-7u51-linux-x64.tar.gz 

# add to .bashrc
PATH=/home/demo/opt/jvm/jdk1.7.0_51/bin:$PATH
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/demo/opt/jvm/jdk1.7.0_51/jre/lib/amd64/server:/home/demo/opt/jvm/jdk1.7.0_51/lib
export PATH LD_LIBRARY_PATH

open new terminal
check that javac -version says javac 1.7.0_51

---------- installing Boost ----------

# download boost from
# https://sourceforge.net/projects/boost/files/boost/1.61.0/

>$ cd ~/Downloads
>$ wget -O boost-1.61.0.tar.bz2 http://sourceforge.net/projects/boost/files/boost/1.61.0/boost_1_61_0.tar.bz2/download
>$ tar xf boost-1.61.0.tar.bz2
>$ cd boost_1_61_0
>$ ./bootstrap.sh --prefix=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default --with-libraries=chrono,date_time,filesystem,iostreams,program_options,random,regex,serialization,signals,system,thread,wave
>$ ./b2 -sNO_BZIP2=1 install

# add to .bashrc
export LD_LIBRARY_PATH=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default/lib:$LD_LIBRARY_PATH

---------- install zgrviewer ----------

Download and untar the ZGRViewer distribution.
wget -O zgrviewer-0.10.0.zip https://sourceforge.net/projects/zvtm/files/zgrviewer/0.10.0/zgrviewer-0.10.0.zip/download

>$ cd ~/opt/
>$ unzip ~/Downloads/zgrviewer-0.10.0.zip 

Edit it's "run.sh" script so that the ZGRV_HOME variable has the correct value. The scripts/zgrviewerExampleScript has some additional java switches that are useful.
ZGRV_HOME=/home/demo/opt/zgrviewer-0.10.0

Edit ~/.bashrc and add an alias that allows you to run ZGRViewer by typing "zgrviewer":

alias zgrviewer='/home/demo/opt/zgrviewer-0.10.0/run.sh'

>$ sudo apt-get install graphviz

Run zgrviewer and edit the preferences to point to graphviz bins

---------------------------

# doxygen
>$ sudo apt-get install doxygen 

---------------------------

# latex
>$ sudo apt-get install texlive

---------------------------

# needed to install yaml 
>$ sudo apt-get install cmake

# yaml-cpp, for reading YAML or JSON configuration files and storing results. 
# Yaml-cpp must be compiled against the same version of boost used
# download yaml-cpp-yaml-cpp-0.5.3.tar.gz from https://github.com/jbeder/yaml-cpp/releases

# Download source code to ~/Downloads/yaml-cpp-0.5.3.tar.gz
>$ BOOST_ROOT=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default
>$ YAMLCPP_ROOT=/home/demo/opt/yaml/0.5.3/boost-1.61.0/gcc-4.9.3-default
>$ cd ~/Downloads
>$ tar xzvf yaml-cpp-yaml-cpp-0.5.3.tar.gz
>$ mkdir yaml-cpp-yaml-cpp-0.5.3/_build
>$ cd yaml-cpp-yaml-cpp-0.5.3/_build
>$ cmake .. -DBOOST_ROOT=$BOOST_ROOT -DCMAKE_INSTALL_PREFIX=$YAMLCPP_ROOT
>$ make install

---------------------------

# Dlib.
# --with-dlib='/home/demo/opt/dlib/18.18'
# Download tarball from http://dlib.net/  
# or https://sourceforge.net/projects/dclib/files/dlib/v18.18/
# unpack into desired installation directory

>$ mkdir ~/opt/dlib
>$ cd ~/opt/dlib
>$ tar -xf ~/Downloads/dlib-18.18.tar.bz2
>$ mv dlib-18.18 18.18

---------------------------

# For various analysis algorithms that use cryptographic functions
>$ sudo apt-get install libssl-dev libgcrypt11-dev 

---------------------------

# For parsing XML files in certain tools such as roseHPCT and BinaryContextLookup.
>$ sudo apt-get install libxml2-dev 

---------------------------

>$ sudo apt-get install libdwarf-dev

---------- bashrc ----------

.bashrc should have

# add jdk to PATH and LD_LIBRARY_PATH
PATH=/home/demo/opt/jvm/jdk1.7.0_51/bin:$PATH
LD_LIBRARY_PATH=/home/demo/opt/jvm/jdk1.7.0_51/jre/lib/amd64/server:/home/demo/opt/jvm/jdk1.7.0_51/lib:$LD_LIBRARY_PATH
# add boost to LD_LIBRARY_PATH
LD_LIBRARY_PATH=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default/lib:$LD_LIBRARY_PATH
# create alias for zgrviewer
alias zgrviewer='/home/demo/opt/zgrviewer-0.10.0/run.sh'
export PATH LD_LIBRARY_PATH

---------- install rose ----------

>$ cd ~/
>$ git clone https://github.com/rose-compiler/rose-develop

>$ cd rose-develop
>$ ./build
>$ cd ..
>$ mkdir build-rose
>$ cd build-rose
>$ CC=/usr/bin/gcc-4.9 CXX=g++-4.9 FC=/usr/bin/gfortran-4.9 CXXFLAGS='-g -rdynamic -Wall -Wno-unused-local-typedefs -Wno-attributes' /home/demo/rose-develop/configure --enable-assertion-behavior=abort --prefix=/home/demo/opt/rose_inst --with-CFLAGS=-fPIC --with-CXXFLAGS=-fPIC --with-C_OPTIMIZE=-O0 --with-CXX_OPTIMIZE=-O0 --with-C_DEBUG='-g -rdynamic' --with-CXX_DEBUG='-g -rdynamic' --with-C_WARNINGS='-Wall -Wno-unused-local-typedefs -Wno-attributes' --with-CXX_WARNINGS='-Wall -Wno-unused-local-typedefs -Wno-attributes' --with-ROSE_LONG_MAKE_CHECK_RULE=yes --with-boost=/home/demo/opt/boost/1.61.0/gcc-4.9.3-default --with-gfortran='/usr/bin/gfortran-4.9' --with-python='/usr/bin/python3' --with-java=/home/demo/opt/jvm/jdk1.7.0_51/bin/javac --enable-languages=all --enable-projects-directory --with-doxygen --without-sqlite3 --without-libreadline --without-magic --with-yaml='/home/demo/opt/yaml/0.5.3/boost-1.61.0/gcc-4.9.3-default' --with-dlib='/home/demo/opt/dlib/18.18' --without-wt --without-yices --without-pch --enable-rosehpct --with-gomp_omp_runtime_library=/usr/lib/gcc/x86_64-linux-gnu/4.9/ --without-haskell --enable-edg_version=4.12
>$ make core
>$ make install-core

---------- set.rose ----------

# create a file to set the rose environment

>$ cd ~/
>$ cat > set.rose
ROSE_INS=/home/demo/opt/rose_inst
PATH=$PATH:$ROSE_INS/bin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ROSE_INS/lib
# Don't forget to export variables !!!
export PATH LD_LIBRARY_PATH

------- using ROSE ----------

>$ source set.rose.edg
>$ mkdir tests/
>$ cd tests/
>$ cat > sample.c
void foo()
{
        int a = 0;
        a += 1;
        return a;
}

>$ identityTranslator -c sample.c
>$ dotGenerator -c sample.c 

V2[edit]

Download[edit]

Download the virtual machine image created by using VMware Player:

  • http://www.rosecompiler.org/Ubuntu-ROSE-Demo-V2.tar.gz
  • Warning: it is a huge file of 6.7 GB (18.2GB if fully uncompressed). It may take ~1 hour to download depending on your High Speed Internet Connection.
  • Demonstration user account (sudo user in Ubuntu):
    • account: demo
    • password: password

Warning: LLNL users may not be able to download it due to limitations to max downloaded file size within LLNL. It may also be against LLNL's security policy to run a virtual machine without authorization. So this image should not be used inside LLNL.


On windows, you can install 7-zip (http://www.7-zip.org/) to untar the tar ball (.tar.gz file) into a folder.

  • It may take ~ 20 minutes on a desktop PC to fully uncompress it in two steps (.tar.gz to .tar, then .tar to the folder)
  • The final folder size is around 18.2 GB

Content[edit]

demo@ubuntu:~$ cat readme

This is a Ubuntu 14.04 virtual machine with installed ROSE-edg4.x. 

Directory List
* ~/rose-edg4x.git  : github.com rose edg4.x-based, checked out on Jan 24, 2015

* ~/buildtree  : build tree of rose, using the following configure command: 

../rose-edg4x.git/configure --prefix=/home/demo/opt/rose_inst --with-boost=/home/demo/opt/boost_1.45.0_inst --with-C_OPTIMIZE=-O0 --with-CXX_OPTIMIZE=-O0 --with-gomp_omp_runtime_library=/usr/lib/gcc/x86_64-linux-gnu/4.8/

* ~/opt/rose_inst : installation path of rose (--prefix value)

* ~/rose-project-template.git:  project templates using the installed rose as a library.

*~/tests: a simple c file, processed by identityTranslator and dotGeneratorWholeASTGraph. 
  type run.sh filex.dot to view a dot file of AST graph

bash env in .bashrc has the following variables by default
----------------------
export PATH=$PATH:/home/demo/opt/jdk1.8.0_25/bin:/home/demo/opt/zgrviewer-0.8.2
export LD_LIBRARY_PATH=/home/demo/opt/boost_1.45.0_inst/lib:/home/demo/opt/jdk1.8.0_25/jre/lib/amd64/server:$LD_LIBRARY_PATH
export JAVA_HOME=/home/demo/opt/jdk1.8.0_25/


To use rose translator , you need to type

source ~/set.rose

Installation Notes[edit]

At the time of writing, ROSE does not officially support Ubuntu 14.04 and its default gcc 4.8, mostly due to boost and other portability issues. Fortunately, the release process does generate gcc 4.8 EDG binaries.

There are a few tweaks used to successfully compile ROSE on Ubuntu 14.04

Prepare the prerequisite stuff for ROSE installation

  • sudo apt-get install gcc g++ gfortran
  • sudo apt-get install libtool flex bison automake
  • sudo apt-get install zlibc zlib1g zlib1g-dev libbz2-dev // mostly for boost iostreams library

Hack 1 to the system header path:

  • sudo ln -s /usr/include/x86_64-linux-gnu/sys /usr/include/sys
    • This is a hack since rose uses an absolute path to find some system headers. Ubuntu 14 has a different path. A better fix should be available.

Boost 1.45 has to be patched to work with gcc 4.8 so the required thread lib can be installed

Boost 1.45 Patch 1:

Error: /home/demo/development/install/gcc-4.4.7/boost-1.45.0/include/boost/thread/xtime.hpp:23: error: expected identifier before numeric constant

WORKAROUND: in file boost/thread/xtime.hpp WORKAROUND THAT DOES WORK FOR THREAD AND WAVE:

only undefine the C11 macro: (the following 3 lines are new, followed by the existing troublesome enum in the file)

#ifdef TIME_UTC
#undef TIME_UTC
#endif

enum xtime_clock_types
{
    TIME_UTC=1
};

Boost 1.45 Patch 2: force boost to recognize threading is actually supported by gcc 4.7 and after.

patch for boost/config/stdlib/libstdcpp3.hpp

33      33       
34      34      #ifdef __GLIBCXX__ // gcc 3.4 and greater: 
35      35      #  if defined(_GLIBCXX_HAVE_GTHR_DEFAULT) \ 
36                      || defined(_GLIBCXX__PTHREADS) 
        36              || defined(_GLIBCXX__PTHREADS) \ 
        37              || defined(_GLIBCXX_HAS_GTHREADS) 
37      38            // 
38      39            // If the std lib has thread support turned on, then turn it on in Boost 
39      40            // as well.  We do this because some gcc-3.4 std lib headers define _REENTANT 


Patch 3 to ROSE, bump up the gcc version allowed for boost filesystem

demo@ubuntu:~/rose-edg4x.git$ git diff
diff --git a/src/util/support/FileHelper.h b/src/util/support/FileHelper.h
index d2ca5b6..142e509 100644
--- a/src/util/support/FileHelper.h
+++ b/src/util/support/FileHelper.h
@@ -5,7 +5,8 @@
 // Non-windows support should used boost filesystem 2 if using GNU version less than 4.7.
 #ifndef _MSC_VER
 // #if ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7))
-#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7)))
+// Liao, 1/24/2015. Not sure why GCC version is checked when we are taling about boost filesystem version. bumped up to 9 so gcc 4.8 can be supported  (<8 do
+#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 9)))
   #define BOOST_FILESYSTEM_VERSION 2
 #endif
 #else
@@ -56,8 +57,8 @@ public:
 
 // DQ (3/8/2014): Adding use of BACKEND_CXX_IS_INTEL_COMPILER to support Intel compiler for backend use.
 #ifndef _MSC_VER
-// #if (defined(_MSC_VER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7)))
-#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7)))
+// #if (defined(_MSC_VER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 9)))
+#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 9)))
  // DQ (2/10/2014): I think this is the older BOOST_FILESYSTEM_VERSION 2 specific code.
     static string getFileName(const string& aPath) {
         path boostPath(aPath);
@@ -132,7 +133,7 @@ public:
 #ifndef _MSC_VER
 // DQ (3/8/2014): Adding use of BACKEND_CXX_IS_INTEL_COMPILER to support Intel compiler for backend use.
 // #if ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7))
-#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7)))
+#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 9)))
          // DQ (2/10/2014): I think this is the older BOOST_FILESYSTEM_VERSION 2 specific code.
             relativePath += *toPathIterator; //The first path element comes without the leading path delimiter
 #else
@@ -147,7 +148,7 @@ public:
             while (toPathIterator != boostToPath.end()) {
 #ifndef _MSC_VER
 // #if (defined(_MSC_VER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7)))
-#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 7)))
+#if (defined(BACKEND_CXX_IS_INTEL_COMPILER) || ((BACKEND_CXX_COMPILER_MAJOR_VERSION_NUMBER == 4) && (BACKEND_CXX_COMPILER_MINOR_VERSION_NUMBER < 9)))
              // DQ (2/10/2014): I think this is the older BOOST_FILESYSTEM_VERSION 2 specific code.
                 relativePath += pathDelimiter + *toPathIterator;
 #else

V1[edit]

For the old VM using ROSE based on EDG 3.x, see ROSE_Compiler_Framework/Virtual_Machine_Image_V1