EMF Views Developer Guide

How EMF Views works
Hacking on EMF Views

How EMF Views works

We create virtual elements that stand-off for each element in the contributing models.

EMF Views is first and foremost an Eclipse plugin (or rather, a suite of Eclipse plugins). Its purpose is to let users to build views from EMF-based models. The core plugin provides the logic for creating these views, and the other plugins provide extra conveniences: DSLs for creating views easily, an experimental viewer, extension facilities, etc.

Most of the critical code thus resides in the core plugin (org.atlanmod.emfviews), and in the Ecore-generated virtual links model. However, we also have an extensive manual and examples.

Repository hierarchy

If we take a look into the EMF Views repository, here is what we will see:

.
├── doc
│   ├── org.atlanmod.emfviews.doc
│   └── src
├── dsls
│   ├── mel
│   └── vpdl
├── examples
│   ├── emfviews-tutorial
│   ├── programmatic-view-tutorial
│   ├── traceability-demo
│   ├── view-to-html-tutorial
│   └── vpdl-tutorial
├── features
│   ├── org.atlanmod.emfviews.feature
│   ├── org.atlanmod.emfviews.mel.feature
│   └── org.atlanmod.emfviews.vpdl.feature
├── plugins
│   ├── org.atlanmod.emfviews
│   ├── org.atlanmod.emfviews.ui.editors
│   ├── org.atlanmod.emfviews.virtuallinks
│   └── org.atlanmod.emfviews.virtuallinksepsilondelegate
├── tests
│   └── org.atlanmod.emfviews.tests
└── update

In order, we have:

The doc folder which contains the Org source for the EMF Views manual and the Eclipse help plugin. See the section The manual.
The dsls folder contains the Xtext-based projects for our two languages that help build views, as well as their tests. See VPDL and MEL.
The examples folder contains the source code and models for the tutorials and demos. The tutorials are documented in the user manual.
The features folder contains the Eclipse features for the update site. There is one feature for the core plugin, and one per DSL. An Eclipse feature is just a group of plugins, to simplify installation and dependency management.
The plugins folder contains the core plugins, namely org.atlanmod.emfviews and the virtual links model which is generated from Ecore. The ECL delegate plugin (see Adding new matching models) is used to create views from an ECL file. There is also an experimental views browser in org.atlanmod.emfviews.ui.editors.
The tests folder contains the tests for the core plugins. These tests are run by our Maven build and on our continuous integration.
Finally, the update folder contains the update site configuration. An Eclipse update site is just a bunch of JARs and XML description files. This is a convenient way for users to install multiple plugins from the same vendor. Our update site simply regroups all the features.

Building and running the project manually

See the Development setup instructions in the README.

While you can use Maven to build and run the tests, most of the interesting functionality of the plugins comes from actually using it in an Eclipse application, in concert with other EMF-based tools like the Sample Ecore Editor, the MoDisco Model Browser or ATL.

We thus recommend that you import the projects in Eclipse. You can then run another Eclipse instance with the plugins loaded; if you use a debug instance (Debug As → Eclise Application) instead, you will be able to tweak the code while the client instance is running, which can make for a more pleasing development experience.

Maven build overview

The Maven build is there mostly for our continuous integration, since it's about the only sane way to build Eclipse plugins in batch.

The way it is setup, we have a pom.xml file at the root which declares a bunch of modules:

<modules>
  <module>plugins</module>
  <module>features</module>
  <module>update</module>
  <module>tests</module>
  <module>dsls</module>
  <module>doc</module>
</modules>

In each module, you will find another pom.xml file with its own modules, etc. The leaves of this tree are the actual plugins, which use the eclipse-plugin packaging:

<packaging>eclipse-plugin</packaging>

Invoking mvn install (or any other target) in the root directory will make use of all the pom.xml files.

The plugins are built using Tycho, a Maven extension for building Eclipse plugins. The benefit of Tycho is that it is pulling the plugin dependencies from the plugin.xml files, so we don't have to duplicate our list of dependencies. Most of the Tycho configuration is in the root pom.xml and shouldn't need much tweaking.

The most likely source of adjustment is in bumping the top-level dependencies. You can easily find them at the top of the root pom.xml:

<properties>
  <tycho-version>1.1.0</tycho-version>
  <eclipse-oxygen>
    http://download.eclipse.org/releases/oxygen/201712201001
  </eclipse-oxygen>
  <eclipse-epsilon>
    http://download.eclipse.org/epsilon/updates/
  </eclipse-epsilon>
  <xtext-version>2.14.0</xtext-version>
</properties>

Should you require a more recent version of Eclipse, or Xtext, this is where to change it.

The Maven builds runs with JUnit, as running them in Eclipse would do. Note that running all the tests require additional dependencies, as stated by the pom.xml of org.atlanmod.emfviews.tests:

<repositories>
  <repository>
    <id>obeo-bpmn2</id>
    <layout>p2</layout>
    <url>
https://s3-eu-west-1.amazonaws.com/obeo-bpmndesigner-releases/3.5.0/repository/
    </url>
  </repository>

  <repository>
    <id>eclipse-reqif</id>
    <layout>p2</layout>
    <url>http://download.eclipse.org/rmf/updates/releases/0.13.0/</url>
  </repository>
</repositories>

Building the DSLs using Maven required more pipework, which see.

VPDL and MEL

VPDL and MEL are two domain-specific languages (DSLs) that are used to simplify the creation of views, or more precisely, the creation of a view weaving model (virtual links). For more on how to use these DSLs, see the user manual.

We have developed these DSLs with Xtext, which conveniently generates a parser and an editor plugin with auto-completion and error reporting directly from a source grammar. Here are the VPDL and MEL grammars.

Xtext grammars are essentially BNF-based, but instead of giving you parse trees, they give you graphs as EMF models.

We then turn a parsed graph into a virtual links model using an ATL transformation. This is done by the extending the AbstractGenerator class. We also create eview and eviewpoint files, accordingly. See VpdlGenerator and MelGenerator for how this is done. Note that these are .xtend files: this is a Java-like language, but more dynamic, and well-suited for creating templates. These get compiled to Java files, and the use of Xtend is transparent to the end-user.

We also enhance the validation and auto-completion of these DSLs by extending the AbstractScopeProvider of Xtext. In VPDL for instance, we want to make sure only valid classes or attributes are used, with respect to the declared metamodels. This is done in VpdlScopeProvider; see MelScopeProvider for MEL.

Maven build of Xtext DSLs

Build the DSLs using Maven is more involved than simply using Tycho. We need to:

Add a goal to regenerate the plugins from the Xtext grammar.
Compile the Xtend files to Java.
Compile the ATL transformations to EMFTVM bytecode.
Cleanup these files for the clean goal.

The first two tasks are documented, and we can use the xtend-maven-plugin for the first one, and the exec-maven-plugin for the second. Generating the plugins from the grammar can take a while though, and usually you don't want to do it unless the grammar has been changed. You can skip this step by passing -Dmwe2-skip-generate=true to mvn.

Running ATL EMFTVM in Maven is possible only since ATL 4.0, but it works well.

Cleaning up generated files is verbose, but straightforward. See the pom.xml for details.

Building the manual

The EMF Views manual is what you are reading right now. There are two versions of it: the online HTML version published here, and the Eclipes Help version, published as an Eclipse help plugin (org.atlanmod.emfviews.doc) and available from the Help → Contents menu inside Eclipse if you have the plugin installed. Both versions are identical in content, but the Eclipse plugin is easy to access offline.

See Building the manual in the README for basic build instructions.

The manual is written in Org. The syntax is akin to Markdown, but Org provides a very convenient and customizable export to HTML. In this respect, it is closer to Sphinx, but the Org syntax is more sensible, and we can customize the whole pipeline in Emacs Lisp instead of Python.

The manual is built simply by invoking make, which in turn invokes emacs to build both versions. All the setup for the HTML export is done in the single doc/publish.el file, which is well commented.

Note that as the org.atlanmod.emfviews.doc plugin is part of the update site, if you want to build the update site manually with mvn install, you should first build the manual. You shouldn't need to build the update site manually though, as it is built automatically by our continuous integration.

Continuous integration

We use Travis to run our tests after each commit. This is done using the Maven build. See the Travis configuration file for details.

After each successful build on the master branch, we build the manual and deploy it on the EMF Views website, on the gh-pages branch. The way we do this is more involved than what is suggested by the Travis documentation, with good reasons documented in the deploy script.

Technically, building the online manual could be a separate Travis job, but since we need the manual to build the org.atlanmod.emfviews.doc plugin, and the doc plugin is used for the update site, we might as well do everything in the same job.

On success, we also deploy the built update site to atlanmod/updates, on the master branch. The latest snapshot is thus always up to date, and can be installed in Eclipse directly.

Rerolling the deployment keys

In order for Travis to push to our repositories, we generate repo-specific deployment keys. We keep the (encrypted) private keys in the repo, and add the public keys to the repositories where Travis must deploy.

Be careful about never committing the private deployment keys to the repository, or leaking them anywhere. These grant push access to the deployment repositories, and not to a specific branch!

In the event the deployment keys were leaked, unauthorize these keys from the depolyment repositories via the Github interface immediately. Then, reroll new deployment keys following this procedure.

These are the steps to generate the deployment key to the atlanmod/emfviews repository and for the atlanmod/updates repository. You usually want to reroll both keys because of the way the Travis client handles file encryption.

Generate SSH key pairs using the email address configured in the deployment scripts:
```
ssh-keygen -t rsa -b 4096 -C "deploy@travis.org"
```
(Be sure to strengthen the security as years go by.)

When prompted for a name, input deploy-key-manual. Leave the password empty (press ENTER twice at the prompt).

This created a private key deploy-key-manual and a public key deploy-key-manual.pub in the current directory.
Do the same to create a deploy-key-updates key pair.
Encrypt the private keys using the Travis client. For the first key:
```
travis encrypt-file -p deploy-key-manual
```
The -p flag tells the client to print the key/iv to the command line. We need to re-use these values for encrypting the second key:
```
travis encrypt-file -p --key <KEY> --iv <IV> deploy-key-updates
```
Replace <KEY> and <IV> by the corresponding values given by the first encrypt-file call.

If you do not reuse the same key/iv values for the second encryption, the Travis client will use random values and will overwrite them in the configuration of your repository, leading to one of the two private key to be undecypherable. Always use the same key/iv pair for encrypting both keys.
The Travis client should have updated the environment variables for the emfviews repository automatically. If not, add them there.

You should have two environment variables of the form:
```
encrypted_LABEL_key
encrypted_LABEL_iv
```
Check that the LABEL matches the one used in the .travis/deploy.sh file.
Authorize the public deploy keys on Github.

Go to atlanmod/emfviews key settings and paste the contents of the deploy-key-manual.pub file here. Call it Travis deploy key and be sure to check Allow write access.

Do the same for atlanmod/updates.
Commit deploy-key-manual.enc and deploy-key-updates.enc to the repository and push.

If all went well, you should have deployed successfully to atlanmod/updates.
Delete the private keys deploy-key-manual and deploy-key-updates. You don't need them anymore. If you need access, just reroll new keys.