Background
At Platogo, we develop 2 mobile apps – Slotpark and Gaminator. The 2 apps are native, however they share the same codebase, which allows us to build unique releases per brand.
Most of the development happens in TypeScript, which allows us to also maintain a desktop browser version of our apps. To make the native apps possible, we use a cross-platform framework called Apache Cordova. Our app has a lot of plugins, and over the years this number has grown to nearly 40! However, we noticed that this made fresh builds often extremely slow, so this is where the investigation begins.
How Cordova works
The bulk of our app is essentially a React-based app. However, we rely on a lot of native platform features, which is where the plugins come in. These plugins are declared in a single manifest file.
https://cordova.apache.org/docs/en/12.x/config_ref/index.html#plugin
The plugin tag declares a plugin as a dependency, and the spec attribute allows you to get the plugin from 3 different kinds of sources:
- A local directory
- a Git remote
- from NPM (if it is published there)
In our project, we actually have multiple combinations of cordova projects we generate based on a few key variables:
- The project (whether it is Slotpark or Gaminator)
- The platform (iOS, Android or Huawei)
- The target environment (staging, production, local)
During development, we regularly make multiple different build variations, and often the cordova folder needs to be wiped. Since we need to “generate” a new Cordova project for each combination, we have our own custom pre-processing step where this is done through various templating.
Investigating the build times
The first thing I noticed, was that the build was really heavy on network traffic, so I decided to measure the data volume over the time of the build. Unfortunately, measuring this is pretty difficult: you cannot really just watch one process, since the build spawns a bunch of ephemeral processes (mostly ssh ones cloning repositories over SSH), so the simplest thing is to simply kill as many background network processes as you can, and measure the difference between the starting and end value of the network Rx in something like Activity Monitor. The result was shocking: a whopping ~4,5GB of data received! What’s going on here?
Anatomy of a Cordova folder
At its core, a Cordova project looks almost like a typical npm project. However, you will notice that it also includes a few extra files and folders:
config.xml declares various Cordova configuration and plugin specs
platforms is where the generates platform-specific projects go
plugins where the cloned plugins live
For a Cordova project to build, it needs to gather and “restore” all the plugins necessary either from npm, direct Git clones, or from the file system.
The solution
One way to reliably ensure that the plugins persist, and that they are guaranteed to be pinned to the right versions, is to simply add them as submodules, and then reference them from the config-template.xml
The migration process looks like this:
- Find the source of the plugin (the right fork, branch and commit)
- Clone the plugin as a submodule into plugins
- Checkout the correct tag/commit
- Update the spec in config-template.xml
We decided to stick to the convention of naming the directories of the cloned plugins to match the name of the plugin itself. However, there are some limitations to this.
For example, certain plugins are needed only for a specific platform, so even for Android builds, you must clone everything. The bigger issue is when we use the same plugin, but checkout different versions for different platforms! We have not found a good solution to this problem yet, but luckily the flexibility of config-template.xml allows us to still keep a few plugin specs defined the old way. As long as the majority of the heavy plugins use submodules, we already gain huge wins.
Results
In the end, we reduced our build times by 3-4x, and also simplified the process of testing new changes rapidly when making patches to plugins.
There are some caveats to working with submodules of course – it is still a less extensively used feature of Git, and without proper care can be mishandled. Because of this, some Git clients like the one integrated into VSCode can be buggy, and e.g. not allow you to stage changes to submodules on certain occasions.
by Daniils Petrovs
