ROSE Compiler Framework/Source Code Repository
Motivation[edit | edit source]
Why are we considering this?
- A single monolithic git repository for ROSE becomes too large
- not productive to clone big repo if people only care about a single portion of it
- We have many collaborative projects using ROSE. Many project contributors are forced to live with the same high-standard as the core ROSE contributors for commits
- Same rigid regression tests by Jenkins for different platforms, different versions of GCC, boost, etc. This is not always necessary for some projects.
- We have some LaTeX documentation and HTML web pages within ROSE's git repository. They essentially can be standalone. But even minor fixing typo for them will require lengthy Jenkins tests (~10 hours or more) to be made effective. This really turns down our appetite to add/update our documents.
Goals[edit | edit source]
Have lean, clean, and well organized source code repositories for the ROSE project
- lean (<=100 MB): only essential things should be stored in the central core repository
- clean: only code-reviewed and tested things are allowed be added into central repositories
- well organized: intuitive, standardized directory layout
- allowing incremental/alternative testing requirements for collaborative, experimental projects
Other requirements: mostly already taken care of by using git
TODO[edit | edit source]
- Find out how other big projects manage multiple repositories ??
- remove unnecessary big files: remove from the repository and from the history
- split the single repository into smaller ones.
- Allow users to download/clone only the things they care.
- make sure they all work together and tested as a whole when needed
- prepare sample repositories to demonstrate how different types of repositories should be used
Survey of Best Practices[edit | edit source]
Risks and Mitigation[edit | edit source]
The biggest concern is that by having multiple repositories,
- it is hard to make sure non-core repositories always work with the core ROSE repository
- Or changes to the core repository may break other repositories without being noticed
- make sure the accurate linking between a project and the version of librose.so
How to prevent these problem?
- Commits to ROSE core repo will trigger tests of all official projects. Only commits passing all project tests will be accepted into the core repo.
- Similarly, changes to any official project will trigger tests against latest ROSE. Only commits passing all ROSE tests will be accepted into the individual project
- Better versioning of librose.so: non-official (external) projects can easily detect if a version of librose.so is compatible.
Plan[edit | edit source]
Core repository[edit | edit source]
The absolutely necessary things for 80% ROSE users should be put into the core repository of ROSE.
It should contain only
- the source files and headers used to build librose.so
- built-in tools for users to use out-of-box
- Doxygen documentation
- Core tests
- How about some essential documents: installation guide for example
- and some tests which are tightly coupled with some features in ROSE?
Docs repository[edit | edit source]
Why do we want to separate some documentations out of the ROSE repository?
- Right now, any changes (even fixing typos) to documentations within ROSE require the same rigid regression tests by Jenkins. This is not necessary and wasteful.
Projects repository/repositories[edit | edit source]
We may have two choices:
- Having a single monolithic repository for all projects.
- Allowing individual repository for each project. So people can just clone what they want. Each project can have its own lifecycle (abandoned, merged into rose, separated for ever, etc.)
I think in a long run, most projects should be treated as
- plugins (or augmentative library) of ROSE (librose.so).
- additional tools, as compared to built-in tools of ROSE
They are important and useful, but they somehow are not merged into the core of ROSE due to various reasons.
The current idea:
- Individual projects should be separate repositories, configurable "--with-rose=/path/to/rose/installation" to make the ROSE-to-project connection.
- Collections of all official projects should be contained in a monolithic repository (e.g. "rose-projects"), where each project is simply a submodule
To be discussed
- We may allow users to contribute projects which only use simple Makefile. Staff members or paid interns can be in charge of converting the build system to use autotools.
Tests repository[edit | edit source]
Common test input files, benchmarks we use to test robustness of the core of ROSE.
Projects can also leverage the same set of tests.