Performance comparison

In the following, we'd like to present some of the build performance comparisons that we've made when building Android applications. The measurements will be performed for saker.build as well as for Gradle, the official build system for Android.

All the comparisons are done on the same and single machine. The comparions are done on Windows 10, with the Windows Defender being turned off all times. When doing the comparisons, we'll be using the build daemons for both tools when applicable.

We'll perform 8 runs for a given case and take the lowest (fastest) time result for each tool. This lowest measurement will be the result for a given test case.

We'll use the Measure-Command PowerShell cmdlet to measure the time it takes to execute a build. (We use the time the cmdlet displays instead that the given tool displays to have a consistent measuring practice.) Note that when the tools are used inside an IDE, the startup cost of the Java processes may be omitted.

When clean builds are measured, the previous outputs are manually cleaned from the output directory rather than running clean commands for a given tool.

The Gradle wrapper wasn't used for running Gradle, it was invoked directly from the installation directory.

Medium sized project

For this test case we've constructed a medium sized Android application that has:

• 3000 Java source files
• 500 layout resource files
• 500 drawable resource files
• 115 PNG image files
• 1000 string declarations in a single resource XML

The test case is available on GitHub. The measurements are performed for building the debug APK for the Android application. It includes compilation of the Android resources, the associated Java code, converting it to dex format, and finally creating and signing the APK.

The results of our measurements are the following:

We can see that saker.build performed the builds 15 - 28% faster, averaging around 22% (not taking no-op into account).

Clean builds are mostly self explanatory, while for incremental builds we've done the following:

• Java incremental: We modified a string value (literal) inside the method body of MainActivity.onCreate().
• Modify Android resource: Modified the value of a string resource in strings.xml.
• Add Android resource: Added a new string resource to strings.xml.

We believe that these kind of modifications are often done when developing Android applications and reflect the build times one should expect when building the apps.