Optimising Flutter CI by caching packages

Reducing package download time by almost 5x by caching it

A while back I was working on reducing the time required to build and test my Flutter app. Besides reworking my CI pipeline I also tried to cut down the by speeding up the linting and reducing the package download time.

Compared to the 1min I gained by combining CI steps, this is a tiny win. But did cut the package download time almost to 1/5th.

The linting time, however, I couldn't master. It seems to somehow cache the results but, I have been able to find out how to reproduce this in the CI build.

Cache Flutter packages for quicker pub get

By caching the packages across jobs it is possible to save the entire download time between builds.

I’ve timed the pub update two jobs here, one where the cache is new (not cached yet) and secondly where it is:

Running pub get without cache

Running pub get after introducing cache

How to cache Flutter packages in Gitlab CI

Gitlab CI only allows caching from within the project folder. By manually specifying the pub cache path you put it inside the project folder and get it cached.

Define a path inside the project folder by using the predefined variable $CI_PROJECT_DIR

Add cache to the top of the gitlab-ci.yaml file and GitLab will start caching. Here’s an example of the full file I use:

The attempt to speed up linting

I found it really strange that on CI, the linting takes about 40s compared to the 2–4s on my local machine. I still haven’t succeeded to track down why but, I will share what I managed to find this far.

The key is to use flutter analyze with the -v for verbose logging. This will give you a timed output for each step and it’s clear to see that in GitLab, compared to local run, some files take far longer.

Output from local run

Output from GitLab CI

The processing time difference for each file (curiously the opposite for app_root.dart) is almost at the order of magnitude between the two pictures above.

I’ve tried adding the analysis_options.yaml with excludes but even with this file below, the build and android folders are read and takes some time:

analyzer:
  exclude:
    - "build/**"
    - "android/**"

And the output from flutter analyze -v:

[+3017 ms] <== {"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/android/app/src/profile/AndroidManifest.xml","errors":[]}}
[   +5 ms] <== {"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/android/app/src/main/AndroidManifest.xml","errors":[]}}
[   +3 ms] <== {"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/android/app/src/debug/AndroidManifest.xml","errors":[]}}
[  +65 ms] <==
{"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/build/path_provider/intermediates/merged_manifest/release/AndroidManifest.xml","errors":[]}}

I found that it was possible to exclude some folders using the --no-current-package setting but it required digging into the code of flutter analyze in github to figure out how it worked. Regardless it didn’t impact much.

So, unfortunately, the linting time is still a mystery.

Conclusion

I would’ve hoped to give a better speedup than just a few seconds for the package but, it’s something and it is easy to add. Just copy the lines from the gist file above.

If anyone has ideas or insights on the linting I would be really happy to hear it! Cutting those ~40s I have down to the 2–10s that it takes locally would speed things up a lot.

Thanks for reading!

Slow lint