Build cache experience, best practices?

I’m working on a project that uses microservices architecture, so we have about 40 deliverables. Of course, there’s a CI pipeline for all 40 projects, all shared libraries and CI feels slow (I think developers find CIs always slow though :slight_smile: ). The individual deliverables share a lot of code - the microservices communicate over GraphQL, so there’s an HTTP server and client, GraphQL server and client, JSON library, etc. in all 40 projects, they are compiled in each CI pipeline for each PR and I feel it’s not optimal to compile the very same version of cowboy 100 times a day. There are also lot of internal infrastructure-like libraries shared between projects, about 20-30 separate Erlang apps. The external dependencies and maybe half of the internal dependencies change only a few times a year, other dependencies change more often. The whole build is run via rebar3.

I wonder - is there already a solution out there that somehow caches the built artifacts between builds and projects? Would it really speed up build? I mean downloading and unziping 90 pre-built depedencies individually might not be that much slower than git fetching them, then compiling the Erlang. Is there a best practice on how to do it? Does it worth optimizing build time - after all, the whole pipeline runs over 30 minutes, compilation is only 2-3 minutes, so shaving a few(?) seconds there might not worth the hassle…

1 Like