ROSE Compiler Framework/ROSE tools

From Wikibooks, open books for an open world
Jump to: navigation, search

Overview[edit]

ROSE is a compiler framework to build customized compiler-based tools. A set of example tools are provided as part of the ROSE release to demonstrate the use of ROSE. Some of them are also useful for daily work of ROSE developers.

We list and briefly explain some tools built using ROSE. They are installed under ROSE_INSTALLATION_TREE/bin .

Any ROSE tool(translator) works like a compiler. You have to provide all necessary compilation flags to make it work properly. You have to specify some include paths and/or some macro to be defined.

  • One way to use rose tools is to replace your default compiler (gcc for example) with the rose tool command (identityTranslator, outline, etc) in your Makefile. So the tool will be called with all correct flags when you type "make".
  • Alternatively, if you are only interested in processing a single file. You can manually watch the full compilation command line (e.g. gcc .... -c ) used to compile that single file during a normal "make". Then replace your compiler (gcc) with the rose tool you are interested in (e.g. outline) in the command line.

Prerequisites[edit]

You have to install ROSE first, by typing configure, make, make install, etc.

You also have to set the environment variables properly before you can call ROSE tools from command line.

For example: if the installation path (or --prefix path in configure) is /home/opt/rose/install, you can have the following script to set the environment variables using bash:

ROSE_INS=/home/opt/rose/install
export ROSE_INS 

PATH=$ROSE_INS/bin:$PATH
export PATH

LD_LIBRARY_PATH=$ROSE_INS/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH

All ROSE tools are installed under the path specified by the --prefix option.

identityTranslator[edit]

Source: http://www.rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf (chapter 2)

This is the simplest tool built using ROSE. It takes input source files , builds AST, and then unparses the AST back to compilable source code. It tries its best to preserve everything from the input file.

Uses[edit]

Typical use cases

  • without any options, test if ROSE can compile your code: replace the compiler used by your Makefile with identityTranslator
  • turn on some built-in analysis, translation or optimization phases, such as -rose:openmp:lowering to support OpenMP
    • type "identityTranslator --help" to see all options
  • debug a ROSE-based translator: the first step is often to use identityTranslator to rule out if it is a compilation problem using ROSE
  • use the source of the identityTranslator as a start point to add custom analysis and transformation. The code in the identityTranslator is indeed the minimum code required for almost all kinds of ROSE-based tools.

Source code[edit]

identityTranslator.c

#include "rose.h"
int main(int argc, char *argv[]){
	// Build the AST used by ROSE
	SgProject *project = frontend(argc, argv);
 
	// Run internal consistency tests on AST
	AstTests::runAllTests(project);
 
	// Insert your own manipulation of the AST here...
 
	// Generate source code from AST and call the vendor's compiler
	return backend(project);
}

Limitations[edit]

Due to limitations of the frontends used by ROSE and some internal processing, identityTranslator cannot generate 100% identical output compared to the input file.

Some notable changes it may introduce include:

  • "int a, b, c;" are transformed to three SgVariableDeclaration statements,
  • macros are expanded.
  • extra brackets are added around constants of typedef types (e.g. c=Typedef_Example(12); is translated in the output to c = Typedef_Example((12));)
  • Converting NULL to 0.

AST dot graph generators[edit]

Tools to generate AST graph in dot format. There are two versions

  • dotGenerator: simple AST graph generator showing essential nodes and edges
  • dotGeneratorWholeASTGraph: whole AST graph showing more details. It provides filter options to show/hide certain AST information.

command line:

 dotGeneratorWholeASTGraph  yourcode.c  // it is best to avoid include any header into your sample code to have a small enough tree to visualize!
dotGeneratorWholeASTGraph --help
   -rose:help                     show this help message
   -rose:dotgraph:asmFileFormatFilter           [0|1]  Disable or enable asmFileFormat filter
   -rose:dotgraph:asmTypeFilter                 [0|1]  Disable or enable asmType filter
   -rose:dotgraph:binaryExecutableFormatFilter  [0|1]  Disable or enable binaryExecutableFormat filter
   -rose:dotgraph:commentAndDirectiveFilter     [0|1]  Disable or enable commentAndDirective filter
   -rose:dotgraph:ctorInitializerListFilter     [0|1]  Disable or enable ctorInitializerList filter
   -rose:dotgraph:defaultFilter                 [0|1]  Disable or enable default filter
   -rose:dotgraph:defaultColorFilter            [0|1]  Disable or enable defaultColor filter
   -rose:dotgraph:edgeFilter                    [0|1]  Disable or enable edge filter
   -rose:dotgraph:expressionFilter              [0|1]  Disable or enable expression filter
   -rose:dotgraph:fileInfoFilter                [0|1]  Disable or enable fileInfo filter
   -rose:dotgraph:frontendCompatibilityFilter   [0|1]  Disable or enable frontendCompatibility filter
   -rose:dotgraph:symbolFilter                  [0|1]  Disable or enable symbol filter
   -rose:dotgraph:emptySymbolTableFilter        [0|1]  Disable or enable emptySymbolTable filter
   -rose:dotgraph:typeFilter                    [0|1]  Disable or enable type filter
   -rose:dotgraph:variableDeclarationFilter     [0|1]  Disable or enable variableDeclaration filter
   -rose:dotgraph:variableDefinitionFilter      [0|1]  Disable or enable variableDefinitionFilter filter
   -rose:dotgraph:noFilter                      [0|1]  Disable or enable no filtering
Current filter flags' values are: 
         m_asmFileFormat = 0 
         m_asmType = 0 
         m_binaryExecutableFormat = 0 
         m_commentAndDirective = 1 
         m_ctorInitializer = 0 
         m_default = 1 
         m_defaultColor = 1 
         m_edge = 1 
         m_emptySymbolTable = 0 
         m_expression = 0 
         m_fileInfo = 1 
         m_frontendCompatibility = 0 
         m_symbol = 0 
         m_type = 0 
         m_variableDeclaration = 0 
         m_variableDefinition = 0 
         m_noFilter = 0 

More information about how to use the tools can be found at How_to_visualize_AST

AST Outliner[edit]

Basic concept: outlining is the process of replacing a block of consecutive statements with a function call to a new function containing those statements. Conceptually, outlining the inverse of inlining.

ROSE provide a builtin translator called AST outliner, which can outline a specified portion of code and generate a function from it.

Official documentation for the AST outliner is located in Chapter 37 Using the AST Outliner with the ROSE Tutorial. pdf.

Supplemental information can be found here at ROSE Compiler Framework/outliner

Inliner[edit]

The ROSE Inliner inlines functions at function callsites.

Official documentation about the Inliner is Chapter 36 Calling the Inliner of ROSE tutorial: http://rosecompiler.org/ROSE_Tutorial/ROSE-Tutorial.pdf

Test directory with an example translator and test input files

By looking into Makefile.am, the example translator's source code will generate an executable named "inlineEverything" in your buildtree.

This is the tool you can try to inline your sample code.

The same Makefile.am's make check rules contain sample command lines to use the tool.

call graph generator[edit]

Command line:

 buildCallGraph -c yourprogram.cpp

Control flow graph generator[edit]

Command line:

 virtualCFG -c yourprogram.c

autoPar[edit]

This is a tool which can automatically insert OpenMP pragmas into input serial C/C++ codes

This tool is an implementation of automatic parallelization using OpenMP. It is used to explore semantics-aware automatic parallelization, as described in one of our paper* .

  • The source files are currently located in rose/projects/autoParallelization.
  • A standalone executable program (named autoPar ) is generated and installed to the installation tree of ROSE (under ROSE INS/bin).
  • you can test the tool in rose_build_tree/projects/autoParallelization by typing "make check"
  • There is a section in ROSE manual pdf: 12.7 Automatic Parallelization

Publications

  • Chunhua Liao, Daniel J. Quinlan, Jeremiah J. Willcock and Thomas Panas, Semantic-Aware Automatic Parallelization of Modern Applications Using High-Level Abstractions, Journal of Parallel Programming, Volume 38, Numbers 5-6, 361-378, August 23. 2010 LLNL-JRNL-421803
  • Chunhua Liao, Daniel J. Quinlan, Jeremiah J. Willcock and Thomas Panas, Extending Automatic Parallelization to Optimize High-Level Abstractions for Multicore, In Proceedings of the 5th international Workshop on OpenMP: Evolving OpenMP in An Age of Extreme Parallelism (Dresden, Germany, June 03 - 05, 2009).

More info is at autoPar

TODO[edit]

refactor the tool translators[edit]

Refactor the tools into a dedicated rose/tools directory. So they will always be built and available by default, with minimum dependency on other things, like which languages are turned on or off (when applicable of course)

Our current idea is we should separate translators used as examples or tutorials AND translators used for creating end-user tools.

  • For tutorial translators, they should NOT be installed as tools by default. Their purpose is to be included in Manual or Tutorial pdf files to illustrate something to developers by examples. Examples should be concise and to the point.
  • On the other hand, translators used to build end-user tools should have much higher standard to accept command options for different, even advanced features. These translators can be very sophisticated since they don't have the page limitation as tutorial examples do.