Dependency Manager — An Approach to Multiple Repositories in Flutter
As a code base begins to grow, in an environment where you are developing multiple applications, it is normal to have some dependencies that are shared across projects. This means that if we don’t want to copy and paste the same functionality, we need to create individual packages that can be used by multiple projects, an approach that has be discussed in this article - Modular Flutter Apps — Design and Considerations.
In one company I worked with, we were using multiple repositories in our development pipelines, in order to create multiple applications with Flutter Web. However this approach created a new problem- if each package can have different versions, how can we version them and reuse them in all other packages and projects?
In this article we will discuss some of the possible approaches to how we can manage dependencies across multiple repositories and present one solution we use at our company. For this purpose, we will consider an application called
app_a that will depend on 3 libraries -
lib_c. Furthermore, both
lib_b depend on
Approach 1: Path Dependencies
The easiest approach to set up a project is to create a project folder in which we place all the libraries and applications, so that we can reference them by relative path. For this to work, we will need to create a strict folder structure, so that we can reference each project:
. ├── app_a ├── lib_a ├── lib_b └── lib_c
Now we can add the dependencies on
pubspec.yaml using the
name: app_a version: 1.0.0+1 dependencies: # Other dependencies lib_a: path: ../lib_a lib_b: path: ../lib_b lib_c: path: ../lib_c
And the same would apply to the
name: lib_a version: 1.0.0+1 dependencies: # Other dependencies lib_c: path: ../lib_c
If we run
flutter pub get, we can successfully get the dependencies and if we run our app via
flutter run -d chrome it will run our application. However, this approach has several problems when we use it in an environment where we are working in a team.
First, this approach means that each developer will have to update to the correct version of each library when developing. This can lead to situations where two developers are developing on
app_a can be using different versions of
lib_c, leading not only to conflicts but also to other development issues since each developer will be running a different version of the same application.
In summary, this solution may fail since we are not able to stipulate strict versioning of the libraries. And for this reason, we explore our next proposition - submodules.
Approach 2: Submodules
With submodules we are able to use the benefits of a
path dependency alongside a strict version reference, since each submodule can point to a specific git reference.
When adding a submodule, we are adding a git repository to our parent repository, in which each have its own commit tree. We can stipulate then what is the reference - commit, branch or tag - that we are going to use for each submodule, meaning that we can have developers using the same library version.
Let’s start by adding the git submodule to
git submodule add https://github.com/Vanethos/dm_lib_c.git
This creates a new folder called
dm_lib_c in the root of our libraries and adds a
.gitmodules file. In this file, we can specify the git branch or reference that we want to use, for this example we will use the
master branch for
[submodule “dm_lib_c”] path = dm_lib_c url = https://github.com/Vanethos/dm_lib_c.git branch = master
Since this submodule is added as a new path, we can reference it in the
pubspec.yaml file as a path reference:
# ... dependencies: # ... lib_c: path: ./dm_lib_c
We use the same methodology in
app_a to add
lib_c as dependencies, and for each one we specify the correct branch that we want to use in the
[submodule "dm_lib_a"] path = dm_lib_a url = https://github.com/Vanethos/dm_lib_a.git branch = submodules [submodule "dm_lib_b"] path = dm_lib_b url = https://github.com/Vanethos/dm_lib_b.git branch = submodules [submodule "dm_lib_c"] path = dm_lib_c url = https://github.com/Vanethos/dm_lib_c.git branch = master
And update the
pubsec.yaml file accordingly:
name: app_a version: 1.0.0+1 dependencies: #... lib_a: path: ./dm_lib_a lib_b: path: ./dm_lib_b lib_c: path: ./dm_lib_c
Finally, we can run
flutter pub get in our
app_a to retrieve the dependencies. However, when we do so we are presented with the following error message:
Because app_a depends on lib_b from path which depends on lib_c from path, lib_c from path dm_lib_b/dm_lib_c is required. So, because app_a depends on lib_c from path dm_lib_c, version solving failed. Running "flutter pub get" in app_a... pub get failed (1; So, because app_a depends on lib_c from path dm_lib_c, version solving failed.)
So what is happening here?
When pubspec tries to resolve its dependencies it has a rule: all dependencies must have the same version, path or git reference. This is why when adding a new dependency to our projects, we usually add it with the
^ symbol - this allows
pubspec to find an appropriate version the dependency in a strict set of rules that you can find in the official documentation.
In reality, when we are adding submodules to each library, we are specifying different
paths to our dependencies. Let’s take a closer look at our tree when we add the submodules on
. ├── dm_lib_a │ └── dm_lib_c ├── dm_lib_b │ └── dm_lib_c └── dm_lib_c
app_a will have a dependency on
lib_c with the path
lib_a will have the path dependency on
/dm_lib_a/dm_lib_c, leading to conflicts on
pubspec since it cannot decide which version to use.
Despite the fact that initially this solution sounded like a good approach since we could define a strict reference for each library, it falls short since it creates issues with the dependencies management of
However, this approach gives us an idea to how we can solve our issues - using git references for each package. Let’s explore that approach.
Approach 3: Using git references
As seen in the Dart dependencies documentation, we can add a dependency to
pubspec.yaml via a git reference:
dependencies: kittens: git: url: firstname.lastname@example.org:munificent/kittens.git ref: some-branch
ref can be any git reference - a tag, a branch or a commit. In the case of this example, we are using branch references to identify our app, so we can use the following to identify
lib_c in the
name: lib_a version: 1.0.0+1 dependencies: lib_c: git: url: https://github.com/Vanethos/dm_lib_c.git ref: git_reference_tag
And in our
app_a we can reference all the other libraries:
name: app_a version: 1.0.0+1 dependencies: lib_a: git: url: https://github.com/Vanethos/dm_lib_a.git ref: git_reference_tag_a lib_b: git: url: https://github.com/Vanethos/dm_lib_b.git ref: git_reference_tag_b lib_c: git: url: https://github.com/Vanethos/dm_lib_c.git ref: git_reference_tag
And if we try to run our app, our app can get all the dependencies, compile and run! So we overcome our last approach’s problem of not being able to compile and applied its idea of using git references to designate strict versions for each library.
With this approach each developer will be able to run and work on the same version of the app, leading to less conflicts and errors down the line.
Nonetheless there is a new problem with this method. As we said in the previous approach, all dependencies must have the same reference, which means that if we want to update the git reference on any library, we will need to create a commit in each library that references it. This can be a problem if we are working in a project with more than 20 libraries that may have dependencies between themselves and are constantly being updated. The process of tagging and updating each library can be time consuming and error-prone, which may lead us to discard this solution too.
One way we can solve this issue is by having our dependencies being set in a remote repository - introducing the Dependency Manager.
Proposed Solution - Dependency Manager
From our previous approaches, we now know two things:
- If we use git references, all the developers on the team are going to use the same version of each library;
- The problem with this approach is that each time that once there is a new update on a specific repository, we have to update all
pubspec.yamlfiles with that new version.
So what if there was a way to have a remote repository where we control all our dependencies? That’s the main idea behind the dependency manager.
Dependency Manager (also known as Bill of Materials), is an outside repository where we stipulate the versions for all libraries that we use. We can have as many Dependency Managers as we want, but we can use the following rule of thumb:
- If we have libraries that are used in different projects we use a specific Dependency Manager for them
- For each app we have a specific dependency manager.
But how can we create the dependency manager?
First, in the
pubspec.yaml file we add all the dependencies that we want:
name: dependency_manager version: 1.0.0+1 dependencies: lib_a: git: url: https://github.com/Vanethos/dm_lib_a.git ref: git_reference_tag_a lib_b: git: url: https://github.com/Vanethos/dm_lib_b.git ref: git_reference_tag_b lib_c: git: url: https://github.com/Vanethos/dm_lib_c.git ref: git_reference_tag
Then, inside the
lib folder we have to create one
dart file whose purpose is to export all our dependencies, making them available for all projects that depend on this one.
library dependency_manager; export 'package:lib_a/lib_a.dart'; export 'package:lib_b/lib_b.dart'; export 'package:lib_c/lib_c.dart';
By doing this, when a file imports
package:dependency_manager/dependency_manager.dart it will automatically have access to
Finally, to make our dependencies available, we add the
dependency_manager as a git reference for each library, for example:
name: lib_a version: 0.0.1 dependencies: flutter: sdk: flutter dependency_manager: git: url: https://github.com/Vanethos/dm_dependency_manager.git ref: master
There is something odd in this process - we have our own library as a dependency of itself and we will have a lot of code in our dependencies that we will not use. This is where Tree Shaking comes into place - all the code that we will not use will be discarded when we compile our binaries.
dependency_manager to work, all our libraries need to point to the same reference of it so that there is no conflict. But what if I want to compile my app pointing to a specific version of the dependency manager? That is where the
dependency_overrides come into place.
As the name implies, we can use
dependency_overrides to override specific versions of libraries we are using. As an example - though we are using remote dependencies, what happens if we are working locally on a feature on
lib_b and want to see how it behaves when integrated on
app_a? If we adhere to the same folder structure as stipulated in the
path dependencies, we can do the following:
name: app_a version: 0.0.1 dependencies: flutter: sdk: flutter dependency_manager: git: url: https://github.com/Vanethos/dm_dependency_manager.git ref: master dependency_overrides: lib_b: path: ./dm_lib_b
pubspec compiles the dependencies, it’s going to retrieve all of them from the
dependency_manager minus the
lib_b dependency which is going to reference from
This has a huge advantage when working with distributed teams - when several developers are working on the same application, if one of them creates a Pull Request (PR), she can use the
dependency_overrides to specify the PR branch so that’s easier for a reviewer to compile the app on their machine:
name: app_a version: 0.0.1 dependencies: flutter: sdk: flutter dependency_manager: git: url: https://github.com/Vanethos/dm_dependency_manager.git ref: master dependency_overrides: lib_b: git: url: https://github.com/Vanethos/dm_lib_b.git ref: feature/PR-1-adds_calculator
In this example when we use
dependency_overrides, we will be able to compile the app with the new feature developed in branch
If we are changing multiple libraries, we can also use
dependency_overrides with the
First, we create a new branch with the same feature name:
feature/PR-1-adds_calculator. In it, we specify the
ref for all the branches we are working on, for example we are only going change
name: dependency_manager version: 1.0.0+1 dependencies: lib_a: git: url: https://github.com/Vanethos/dm_lib_a.git ref: feature/PR-1-adds_calculator lib_b: git: url: https://github.com/Vanethos/dm_lib_b.git ref: feature/PR-1-adds_calculator lib_c: git: url: https://github.com/Vanethos/dm_lib_c.git ref: dm_ref_c
Then, when creating a PR, we must tell other developers that they can use the following
dependency_overrides: dependency_manager: git: url: https://github.com/Vanethos/dm_dependency_manager.git ref: feature/PR-1-adds_calculator
And with one override, we basically stipulated that both
lib_b will target the branch
feature/PR-1-adds_calculator that we have worked on. This makes it easier to build and deploy if necessary.
Downsides of using a Dependency Manager
Although we see many upsides with the dependency manager, there are a couple of issues that we need to be aware of.
The first one is that since we are using a lot of
git references, when we try to get new packages via
flutter pub get, it will take longer since
pubspec needs to verify all the dependencies and git references and download them.
The second, is that since we are using a branch reference for the Dependency Manager,
flutter pub get might not always get the latest version, which means that we need to upgrade the packages. However, this can cause an issue - maybe we want to get the latest version of the Dependency Manager but we don’t want to upgrade all of our other packages - so how can we solve this? By specifying directly which package we are upgrading:
flutter upgrade dependency_manager
With this command we will upgrade only the
Finally, as we have seen with the git references, each time we finish a new feature on a library we will need to release a new version of that package, which means that we need to:
- Create a new tag in the relevant repository
- Update the Dependency Manager with the latest tag This process can be time consuming and error-prone, hence why it is recommended that we create scripts to automate this process.
As we have seen, there are many ways for us to manage dependencies in
pubspec, and though some might fit small applications, it is hard to create a solution that is scalable and usable in large codebases.
The solution we propose in this article is an alternative solution to manage our dependencies by creating a remote repository, and though it causes some issues such as long
flutter pub get commands, we feel that it’s the solution that fits our purpose 🏇.
There was another solution that was not included in this article - creating our own pub server in which we could host our dependencies. This would cause other issues, such as the time setting up the server, and the cost of maintaining it, and it would not always give a solution to every problem - how would we update dependencies across libraries? And how could we stipulate the dependencies when someone is manually testing a PR?
As a final thought - these are the kind of problems that arise when we start creating apps that will be developed and maintainable over a long period of time. It’s hard to find or to create good solutions, so that is why I personally advise for sharing the knowledge and discussing it openly - have you ever come across a problem similar to this one? How did you solve it?
The GitHub repositories for this article can be found here: https://github.com/Vanethos/dm_dependency_manager https://github.com/Vanethos/dm_app_a https://github.com/Vanethos/dm_lib_a https://github.com/Vanethos/dm_lib_b https://github.com/Vanethos/dm_lib_c