ROSE Compiler Framework/Source Code Repository

From Wikibooks, open books for an open world
Jump to: navigation, search

Motivation[edit]

Why are we considering this?

  • A single monolithic git repository for ROSE becomes too large
    • not productive to clone big repo if people only care about a single portion of it
  • We have many collaborative projects using ROSE. Many project contributors are forced to live with the same high-standard as the core ROSE contributors for commits
    • Same rigid regression tests by Jenkins for different platforms, different versions of GCC, boost, etc. This is not always necessary for some projects.
  • We have some LaTeX documentation and HTML web pages within ROSE's git repository. They essentially can be standalone. But even minor fixing typo for them will require lengthy Jenkins tests (~10 hours or more) to be made effective. This really turns down our appetite to add/update our documents.

Goals[edit]

Have lean, clean, and well organized source code repositories for the ROSE project

  • lean (<=100 MB): only essential things should be stored in the central core repository
  • clean: only code-reviewed and tested things are allowed be added into central repositories
  • well organized: intuitive, standardized directory layout

Facilitate collaborations

  • allowing incremental/alternative testing requirements for collaborative, experimental projects

Other requirements: mostly already taken care of by using git

  • secure
  • backup

TODO[edit]

  • Find out how other big projects manage multiple repositories ??
  • remove unnecessary big files: remove from the repository and from the history
  • split the single repository into smaller ones.
    • Allow users to download/clone only the things they care.
    • make sure they all work together and tested as a whole when needed
  • prepare sample repositories to demonstrate how different types of repositories should be used

Survey of Best Practices[edit]

Risks and Mitigation[edit]

The biggest concern is that by having multiple repositories,

  • it is hard to make sure non-core repositories always work with the core ROSE repository
  • Or changes to the core repository may break other repositories without being noticed
  • make sure the accurate linking between a project and the version of librose.so

How to prevent these problem?

  • Commits to ROSE core repo will trigger tests of all official projects. Only commits passing all project tests will be accepted into the core repo.
  • Similarly, changes to any official project will trigger tests against latest ROSE. Only commits passing all ROSE tests will be accepted into the individual project
  • Better versioning of librose.so: non-official (external) projects can easily detect if a version of librose.so is compatible.

Plan[edit]

Core repository[edit]

The absolutely necessary things for 80% ROSE users should be put into the core repository of ROSE.

It should contain only

  • the source files and headers used to build librose.so
  • built-in tools for users to use out-of-box
  • Doxygen documentation

Submodule:

  • Core tests

Questions

  • How about some essential documents: installation guide for example
  • and some tests which are tightly coupled with some features in ROSE?

Docs repository[edit]

Why do we want to separate some documentations out of the ROSE repository?

  • Right now, any changes (even fixing typos) to documentations within ROSE require the same rigid regression tests by Jenkins. This is not necessary and wasteful.

Projects repository/repositories[edit]

We may have two choices:

  • Having a single monolithic repository for all projects.
  • Allowing individual repository for each project. So people can just clone what they want. Each project can have its own lifecycle (abandoned, merged into rose, separated for ever, etc.)

I think in a long run, most projects should be treated as

  • plugins (or augmentative library) of ROSE (librose.so).
  • additional tools, as compared to built-in tools of ROSE

They are important and useful, but they somehow are not merged into the core of ROSE due to various reasons.


The current idea:

  • Individual projects should be separate repositories, configurable "--with-rose=/path/to/rose/installation" to make the ROSE-to-project connection.
  • Collections of all official projects should be contained in a monolithic repository (e.g. "rose-projects"), where each project is simply a submodule

To be discussed

  • We may allow users to contribute projects which only use simple Makefile. Staff members or paid interns can be in charge of converting the build system to use autotools.

Tests repository[edit]

Common test input files, benchmarks we use to test robustness of the core of ROSE.

Projects can also leverage the same set of tests.

References[edit]