CONTRIBUTING.md

layout	title	category	tags
developer-doc	Development & Contributing Guide	summary	summary contributing

Development & Contributing Guide

Thank you for your interest in contributing to the Enso IDE! We believe that only through community involvement can Enso be the best it can be! There are a whole host of ways to contribute, and every single one is appreciated.

Reporting Issues

If you are concerned that your bug publicly presents a security risk to the users of Enso, please contact security@enso.org.

While it's never great to find a bug, they are a reality of software and software development! We can't fix or improve on the things that we don't know about, so report as many bugs as you can! If you're not sure whether something is a bug, file it anyway!

Even though GitHub search can sometimes be a bit hard to use, we'd appreciate if you could search for your issue before filing a bug as it's possible that someone else has already reported the issue. We know the search isn't the best, and it can be hard to know what to search for, so we really don't mind if you do submit a duplicate!

Opening an issue is as easy as following this link and filling out the fields. The template is intended to collect all the information we need to best diagnose the issue, so please take the time to fill it out accurately.

The reproduction steps are particularly important, as the more easily we can reproduce it, the faster we can fix the bug! It's also helpful to have the version of the IDE, as that will let us know if the bug is Operating System or Architecture specific.

Development Environment

The project builds on macOS, Windows, and Linux. Cross-platform targets work well on all of these platforms, however, macOS package will miss the right application icon if built on Linux or Windows due to non-trivial icon generation on these platforms. To develop the source code you will need the following setup:

• The Rust Toolchain

  This project uses several features available only in the nightly Rust toolchain. Please use the Rust toolchain installer to manage Rust toolchains. It will automatically download the toolchain needed to build the project.

  In addition, some custom CLI programs need to be installed manually:

  rustup toolchain install stable   # Stable toolchain required for the following tools.
  cargo +stable install wasm-pack   # Install the wasm-pack toolkit.
  cargo +stable install cargo-watch # To enable `./run wasm watch` utility

  Make sure that your PATH environment variable is set up correctly, so that the binaries installed by cargo (typically to $HOME/.cargo/bin) can be run from the command line; verify this by running wasm-pack --version.

• Node and Node Package Manager LTS

  To build the web and desktop applications you will need the latest LTS version of node and npm. Even minor release changes are known to cause serious issues, thus we provide support for the latest LTS version only. Please do not report build issues if you use other versions. The easiest way to set up the proper version is by installing a Node version manager that automatically picks up the correct version, like fnm.

• (Optional) FlatBuffer compiler flatc

  This dependency is needed only if you need to update files generated by the FlatBuffer from the Engine Services binary protocol description. Otherwise, relying on the generated files that are being stored in this repository is fine.

  flatc must be in the version newer than 1.12 due to this bug. As of writing this text there are no official releases with this issue fixed, however current binaries can be obtained from the project's CI build artifacts. flatc builds from 8 May 2020 onwards have been confirmed to work.

  After placing flatc in PATH you need to define the ENSO_IDE_ENABLE_FLATC environment variable to explicitly enable regeneration of the interface files. The flatc is run as part of build.rs script of the `enso-protocol package.

• (Optional) The Latest version of wasm-opt

  The version of wasm-opt that comes bundled with wasm-pack can be out of date and slow. We recommend to install a newer version from the binaryen releases, or compile the newest version from scratch. wasm-pack will pick up the locally installed version automatically and use it instead of the bundled one. Binaryen packages are also available through some system package managers (apt/pacman/brew), check whether the version there is newer than the bundled version of wasm-pack (which is true for all of them as of wasm-pack 0.10.2, which bundles version 90 which was released Dec 2019).

Working with sources

Please be sure to carefully read the Rust style guide 1 and the Rust style guide 2 before contributing to the codebase.

We do also use prettier for the JavaScript and Markdown files in our code base. If you have not installed it already you can do so via npm install prettier. To use it manually via command line run prettier --write to all JavaScript files in the project. Alternatively, there are plugins for many IDEs available to do this for you.

Repository Structure Overview

Note: Currently, the Enso repository is going through a process of refactoring, and it is not finished yet: the Engine files are still not in the app/engine where they ought to be, but in the root directory instead.

The root directory contains Cargo.toml and build.sbt files, allowing to open all rust or scala code as a single project in your favorite IDE. There is also a run and run.cmd scripts used for building and running the Enso IDE (see next section for details).

The subdirectories of interests are:

• app: The actual products delivered in this repository:
  • gui: A rust crate compiled to a WASM library with all the logic of the GUI layer. The library is used by both the desktop application and the cloud environment. For further documentation see the documentation of the crate (at the top of the src/lib.rs file).
  • ide-desktop: The desktop version of the Enso IDE. Implemented as an electron application which spawns backend services, loads the WASM gui library and runs the main entry point.
  • engine: (In the future: see the note at the section beginning). The implementation of the language itself: CLI tools like compiler or interpreter, as well as the services used as a backend for the Enso IDE (Language Server and Project Manager).
• lib: All libraries not being the main components of our application. They are grouped by language. The most prominent are:
  • rust/prelude: A library containing the most popular utilities and imports. Should be imported in each rust module - see Contributing guidelines.
  • rust/ensogl: EnsoGL Framework for creating efficient GUI applications in WASM.
  • rust/frp: The library allows following the Functional Reactive Programming paradigm in rust.
• build: Build script that is wrapped by the run script.
• integration-test: A single crate with all integration tests of our applications.

Other directories are auto-generated dist and target, or (currently) are the Engine files, which will be moved to app/engine soon.

Development

As this is a multipart project with many complex dependencies, it is equipped with a build script which both validates your working environment and takes care of providing the most suitable compilation flags for a particular development stage.

The build-script is invokable by invoking the following script from the working copy root:

• ./run in bash-compatible environments, like Linux or macOS;
• .\run.cmd on Windows.

For brevity, this guide will use ./run form from here onwards.

In general, ./run should also work on Windows ports of bash (like the ones provided by git or MSYS2). These configurations are not tested though.

Run ./run --help to learn about available commands and options. Some subcommands allow passing additional arguments following -- argument to the underlying call. For example ./run ide build -- FLAG will pass the FLAG flag to wasm-pack (Rust WASM build tool). The most common options are presented below:

• Interactive mode Run ./run ide watch to start a local web-server and a source-file watch utility which will build the project on every change. Open http://localhost:8080 (the port may vary and will be reported in the terminal if 8080 was already in use) to run the application, or http://localhost:8080/?entry to open example demo scenes list. Please remember to disable the cache in your browser during the development! By default, the script disables heavyweight optimizations to provide interactive development experience.
• Production mode In order to compile in a production mode (enable all optimizations, strip WASM debug symbols, minimize the output binaries, etc.), run ./run gui build. To create platform-specific packages and installers use ./run ide build instead. The final executables will be located at dist/ide.
• Selective mode In order to compile only part of the project, and thus drastically shorten the incremental compile time, you are advised to use the selective compilation mode by passing the --crate-path option to the build or watch command, e.g. ./run ide watch --crate-path ensogl/examples to compile only the renderer-related example scenes. Please note, that in order to run a scene in a web-browser, the scene has to be compiled and has to expose a public function with a name starting with entry_point_. Thus, if you compile only selected crate, you will have access only to the example scenes that were defined or re-exported by that crate. In particular, the ide crate exposes the entry_point_ide function, so you have to compile it to test your code in the Enso IDE.

Using IDE as a library.

In case you want to use the IDE as a library, for example to embed it into another website, you need to first build it using ./run gui build and find the necessary artifacts located at dist/gui. Especially, the dist/gui/assets/index.js defines a function window.enso.main(cfg) which you can use to run the IDE. Currently, the configuration argument can contain the following options:

• entry - the entry point, one of predefined scenes. Set it to empty string to see the list of possible entry points.
• project - the project name to open after loading the IDE.

Testing, Linting, and Validation

After changing the code it's always a good idea to lint and test the code. We have prepared several scripts which maximally automate the process:

• Size Validation Use ./run wasm build to check if the size of the final binary did not grew too much in comparison to the previous release. Watching the resulting binary size is one of the most important responsibility of each contributor in order to keep the project small and suitable for web-based usage. In case the size will exceed the limits:

  • If the PR does not include any new libraries, you are allowed to increase the limit by 10KB. In case the limit will be exceeded by more than 10KB, check which part of the code contribute to it, and talk about it with the code owner.
  • If the PR does include new libraries, you are allowed to increase the limit by 10KB, but you should also consider if it is possible to get the same results without using a new library (even by implementing few lines of code from the library in sources of the project).
  • If the PR does include new libraries, and the limit is exceeded by more than 10KB, check which part of the code contributed to it, and talk about it with the code owner.
  • If the PR does include new libraries, and the limit is exceeded by more than 50KB, it would probably not be merged. Research possible alternatives before talking with code owner about this case.

• Testing For the test suite to run you need a current version of Chrome installed. Use ./run wasm test run both unit and web-based visual test.

• Integration Tests The integration tests are gathered in integration-test crate. You can run them with ./run ide integration-test command. The script will spawn required Engine process.

  • To run une test suite add -- --test <suite-name> at end of command options. The <suite-name> is a name of the file in integration-test/tests directory without extension, for example graph_editor.
  • The integration test can create and leave new Enso projects. Keep it in mind when running the script with your own backend (the --external-backend option). The Engine spawned by the script will use a dedicated workspace created in temporary directory, so the user workspace will not be affected.

• Linting Please be sure to fix all errors reported by ./run lint before creating a pull request to this repository.

Development Branches

The following branches are used to develop the product:

• wip/[github_user_name]/[feature] Feature branches. These are temporary branches used by the team to develop a particular feature.

• develop Contains the most recent changes to the product. After successful review, the feature branches are merged here. Each commit to this branch will result in a nightly build of the product accessible as CI artifacts.

• unstable Contains only those commits which can be considered unstable product releases. Each commit to this branch will result in an unstable release of the product and will be published on GitHub as a pre-release. The build version and build description will be automatically fetched from the newest CHANGELOG.md entry and will fail if the version will not be of the form [major].[minor].[patch]-[sfx], where [sfx] is one of alpha.[n], beta.[n], or rc.[n], where [n] is an unstable build number.

• stable Contains only those commits which can be considered stable product releases. Each commit to this branch will result in a stable release of the product and will be published on GitHub as a release. The build version and build description will be automatically fetched from the newest CHANGELOG.md entry and will fail if the version will not be of the form [major].[minor].[patch].

Forcing CI builds

By default, CI would not build artifacts from wip and develop branches in order to save time and resources. If you want the artifacts to be build for your PR, simply add [ci build] anywhere in the PR description.

Skipping CHANGELOG.md change assertions

By default, CI would fail if the CHANGELOG.md file does not need to be updated. However, sometimes there are PRs that does not change anything significant in the final product. You can then simply add the 'CI: No changelog needed' label to the PR to skip this assertion.

Publishing Results

All new changes should be proposed in the form of Pull Requests (PRs) to this repository. Each PR should contain changes to documentation and CHANGELOG.md if applicable.

Changelog

Please remember to update the CHANGELOG.md on every new bug fix or feature implementation. Please note that CHANGELOG.md is used to establish the current product version (the run script extracts it from the newest changelog entry). Thus, be sure to always increase the newest version in the changelog after a release, otherwise CI will fail. Please use the docs/CHANGELOG_TEMPLATE.md as the template to create new changelog entries. Please note, that there is a special syntax for defining features of the upcoming release. The newest changelog entry can have a title "Next Release". In such a case, the build version will be 0.0.0 and CI would fail when trying to publish it as a release.