Optimising Flutter CI by caching packages

Reducing package download time by almost 5x by caching it

A while back I was working on reducing the time required to build and test my Flutter app. Besides reworking my CI pipeline I also tried to cut down the by speeding up the linting and reducing the package download time.

Compared to the 1min I gained by combining CI steps, this is a tiny win. But did cut the package download time almost to 1/5th.

The linting time, however, I couldn't master. It seems to somehow cache the results but, I have been able to find out how to reproduce this in the CI build.

Cache Flutter packages for quicker pub get

By caching the packages across jobs it is possible to save the entire download time between builds.

I’ve timed the pub update two jobs here, one where the cache is new (not cached yet) and secondly where it is:

Running pub get without cache
Running pub get after introducing cache

How to cache Flutter packages in Gitlab CI

Gitlab CI only allows caching from within the project folder. By manually specifying the pub cache path you put it inside the project folder and get it cached.

Define a path inside the project folder by using the predefined variable $CI_PROJECT_DIR

Add cache to the top of the gitlab-ci.yaml file and GitLab will start caching. Here’s an example of the full file I use:

The attempt to speed up linting

I found it really strange that on CI, the linting takes about 40s compared to the 2–4s on my local machine. I still haven’t succeeded to track down why but, I will share what I managed to find this far.

The key is to use flutter analyze with the -v for verbose logging. This will give you a timed output for each step and it’s clear to see that in GitLab, compared to local run, some files take far longer.

Output from local run
Output from GitLab CI

The processing time difference for each file (curiously the opposite for app_root.dart) is almost at the order of magnitude between the two pictures above.

I’ve tried adding the analysis_options.yaml with excludes but even with this file below, the build and android folders are read and takes some time:

analyzer:
exclude:
- "build/**"
- "android/**"

And the output from flutter analyze -v:

[+3017 ms] <== {"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/android/app/src/profile/AndroidManifest.xml","errors":[]}}
[ +5 ms] <== {"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/android/app/src/main/AndroidManifest.xml","errors":[]}}
[ +3 ms] <== {"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/android/app/src/debug/AndroidManifest.xml","errors":[]}}
[ +65 ms] <==
{"event":"analysis.errors","params":{"file":"/Users/ddikman/code/jreader/build/path_provider/intermediates/merged_manifest/release/AndroidManifest.xml","errors":[]}}

I found that it was possible to exclude some folders using the --no-current-package setting but it required digging into the code of flutter analyze in github to figure out how it worked. Regardless it didn’t impact much.

So, unfortunately, the linting time is still a mystery.

Conclusion

I would’ve hoped to give a better speedup than just a few seconds for the package but, it’s something and it is easy to add. Just copy the lines from the gist file above.

If anyone has ideas or insights on the linting I would be really happy to hear it! Cutting those ~40s I have down to the 2–10s that it takes locally would speed things up a lot.

Thanks for reading!

Slow lint

Active reader, developer, manager, entrepreneur & sporadic writer here and at https://greycastle.se. Currently working at https://styler.link.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Mysql Gui Tools 5.0r12 Free Download For Mac

Galaxy Shooter 2D — UI Text

CI/CD Practices In Software And App Development: How To Successfully Implement Them

Getting Sidekiq to Debounce

Sidekiq debounce

Rock’n’Rule and the challenge of introducing code analysis across the DB DevOps process

Industry Use cases of MongoDB

How to Implement LRU Cache Using Doubly Linked List and a HashMap

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
David Dikman

David Dikman

Active reader, developer, manager, entrepreneur & sporadic writer here and at https://greycastle.se. Currently working at https://styler.link.

More from Medium

Flutter Measure Formatter Plugin

Flutter Golden Testing with Bloc

A Better Flutter App #1

Document type storage in Flutter using Hive