ROSE Compiler Framework/outliner

From Wikibooks, open books for an open world
Jump to navigation Jump to search

Overview[edit]

Basic concept: outlining is the process of replacing a block of consecutive statements with a function call to a new function containing those statements. Conceptually, outlining the inverse of inlining.

Use: Outlining is widely used to generate kernel functions to be executed on CPU and/or GPU.

  • help implementing programming models such as OpenMP
  • support empirical tuning of a code portion by first generating a function out of the code portion.

ROSE provide a builtin translator called AST outliner, which can outline a specified portion of code and generate a function from it.

Official documentation for the AST outliner is located in Chapter 37 Using the AST Outliner with the ROSE Tutorial. pdf.

There are two basic ways to use the outliner.

  • Command line method: You can use a command (outline )with options to specify outlining targets, there are two ways to specify which portion of code to outline
    • use a special pragma to mark outline targets in the input program, and then call a high-level driver routine to process these pragmas.
    • using abstract handle strings (detailed in Chapter 46 of ROSE tutorial) as command line options
  • Function call method: call “low-level” outlining routines that operate directly on AST nodes you want to outline

Command line[edit]

The tool rose/bin/outline comes from tutorial/outliner/outline.cc, which relies on 1) pragmas in input codes or 2) abstract handles specified as command line options to find the target code portion to be outlined.

  • The pragma: put #pragam rose_outline right in front of the code portion you want to outline in your input code
  • abstract handle: -rose:outline:abstract_handle your_handle_string

options[edit]

./outline --help


Usage: outline [OPTION]... FILENAME...
Main operation mode:
        -rose:outline:preproc-only                     preprocessing only, no actual outlining
        -rose:outline:abstract_handle handle_string    using an abstract handle to specify an outlining target
        -rose:outline:parameter_wrapper                use an array of pointers to pack the variables to be passed
        -rose:outline:structure_wrapper                use a data structure to pack the variables to be passed
        -rose:outline:enable_classic                   use parameters directly in the outlined function body without transferring statement, C only
        -rose:outline:temp_variable                    use temp variables to reduce pointer dereferencing for the variables to be passed
        -rose:outline:enable_liveness                  use liveness analysis to reduce restoring statements if temp_variable is turned on
        -rose:outline:new_file                         use a new source file for the generated outlined function
        -rose:outline:output_path                      the path to store newly generated files for outlined functions, if requested by new_file. The original source file's path is used by default.
        -rose:outline:exclude_headers                  do not include any headers in the new file for outlined functions
        -rose:outline:use_dlopen                       use dlopen() to find the outlined functions saved in new files.It will turn on new_file and parameter_wrapper flags internally
        -rose:outline:enable_debug                     run outliner in a debugging mode

Example use[edit]

  • outline test.cpp // outline code portions in test.cpp. These code portions are marked by the special rose_outline pragma
  • outline -rose:skipfinalCompileStep -rose:outline:new_file test.cpp // skip compiling the generated rose_? file, put the generated function into a new file

Using abstract handles at command lines, no need to insert pragmas into your input codes anymore

  • outline -rose:outline:abstract_handle ”ForStatement<position,12>” test3.cpp // outline the for loop located at line 12 of test3.cpp
  • outline -rose:outline:abstract_handle ”FunctionDeclaration<name,initialize>::ForStatement<numbering,2>” test2.cpp // outline the 2nd for loop within a function named "initialize" within the test2.cpp file.

/home/liao6/workspace/masterDevClean/buildtree/tests/roseTests/astOutliningTests/outline -rose:outline:new_file -rose:outline:temp_variable -rose:outline:exclude_headers -rose:outline:abstract_handle 'ForStatement<numbering,1>' -c /home/liao6/workspace/masterDevClean/sourcetree/tests/roseTests/astOutliningTests/complexStruct.c

Programming API[edit]

You can build your own translators leveraging the outlining support in ROSE. The programming API is defined in

  • Header file: src/midend/programTransformation/astOutlining/
  • Namespace: Outliner

A few functions and options are provided:

  • Functions: Outliner::outline(), Outliner::isOutlineable()
  • Options

Advanced features[edit]

Some details for outlining can be specified by using command line options or internal flags of the programming API.

List

  • wrap all variables into a data structure: Outliner::useStructureWrapper


dlopen[edit]

use_dlopen option tells the outliner to use the dlopen() to find and call the outlined function stored into a dynamically loadable library.

This option will turn on several other options

  • -rose:outline:exclude_headers
  • useNewFile= true;
  • useParameterWrapper = true;
  • temp_variable = true;

outline -rose:outline:use_dlopen -I/home/liao6/workspace/outliner/build/../sourcetree/projects/autoTuning -rose:outline:output_path . -c /path/to/input.c

Testing[edit]

The ROSE AST outliner has a dedicated testing directory: rose/tests/nonsmoke/functional/roseTests/astOutliningTests

  • Some C, C++ and Fortran test input files are prepared there.
  • Sample command line options are available in the Makefile.am file within this test directory.


full command line example

  • /home/liao6/workspace/autoPar/buildtree/tests/nonsmoke/functional/roseTests/astOutliningTests/outline -rose:outline:use_dlopen -rose:outline:temp_variable -I/home/liao6/workspace/autoPar/buildtree/../sourcetree/projects/autoTuning -rose:outline:exclude_headers -rose:outline:output_path . -c /home/liao6/workspace/autoPar/sourcetree/tests/nonsmoke/functional/roseTests/astOutliningTests/array1.c

Known issues[edit]

List

  • the message "error in side effect analysis!" when setting Outliner::useStructureWrapper to true. This also happens in the outlineIfs example from the tutorial directory.
    • you can ignore this warning message if your translator still works. The outliner uses quite some analyses internally if Outliner::useStructureWrapper is turned on. But some of the analyses may not always handle all situations so they just give up and notify the outliner. The outliner is designed to make conservative decisions in this case and to generate less optimal translated code.

Publications[edit]

A paper describing the internals of the AST outliner, the default paper to cite if you happen to use outliner for your research work

  • Chunhua Liao, Daniel J. Quinlan, Richard Vuduc, and Thomas Panas. 2009. Effective source-to-source outlining to support whole program empirical optimization. In Proceedings of the 22nd international conference on Languages and Compilers for Parallel Computing (LCPC'09)

To support generating multi-threaded kernels for CPUs and GPUs

  • Chunhua Liao , Daniel J. Quinlan , Thomas Panas , Bronis R. de Supinski, A ROSE-Based OpenMP 3.0 research compiler supporting multiple runtime libraries, Proceedings of the 6th international conference on Beyond Loop Level Parallelism in OpenMP: accelerators, Tasking and more, June 14-16, 2010, Tsukuba, Japan
  • C. Liao, Y. Yan, B. R. de Supinski, D. J. Quinlan, and B. Chapman, “Early experiences with the openmp accelerator model,” in Openmp in the era of low power devices and accelerators, Springer, 2013, pp. 84-98.

Being used to support empirical tuning or autotuning

  • Shirley Moore, Refactoring and automated performance tuning of computational chemistry application codes, Proceedings of the Winter Simulation Conference, December 09-12, 2012, Berlin, Germany
  • Nicholas Chaimov , Scott Biersdorff , Allen D Malony, Tools for machine-learning-based empirical autotuning and specialization, International Journal of High Performance Computing Applications, v.27 n.4, p.403-411, November 2013