Continuous delivery for iOS applications using Jenkins and Fastlane

Technical article | Continuous delivery for iOS applications using Jenkins and Fastlane. Find out why you need continuous delivery and how to deliver it.

Why you need continuous delivery

We often want to deploy our mobile applications more frequently without compromising the stability of our apps. To do that we need the process of building and distributing our mobile apps predictable and on-demand. This article will guide you through setting up a CI/CD pipeline using Fastlane, Jenkins, Gitlab and SonarQube for code quality. Hold on tight, it's going to be a long way, but it's definitely worth the time.

First things first, we must make sure that our code is always ready for deployment so that we can deliver with confidence. We can do this by building a deployment pipeline that can help us discover bugs in minutes with the help of unit, integration, and UI tests. Because of this, the team’s focus will be on delivering small, incremental changes that are low risk and help the product with a faster time to market.

Continuous delivery can drive the development costs down even though they require an up-front effort in building those pipelines and writing those unit tests by reducing or completely eliminating the so-called hardening and smoke test phases as well as the unnecessary “code freezes”.

Being able to deploy smaller changes faster while at the same time continuing working on other small, incremental improvements and getting feedback faster from users, development teams are more efficient and happier. Why happier? Because it enables them to do what they care about most: building apps and features that users love.

Delivering with confidence

The first and most important step in building a valuable continuous delivery pipeline is code confidence, meaning having confidence in the code you deploy. How many times have you heard the following excuses for not deploying an app to the AppStore or even for submitting a build for testing?

“We cannot release the application until our lead developer returns from vacation.”

“I have to recreate the provisioning profiles before I send a new testing build but I am currently working on something else, sorry”

“This fix is not critical, we can submit it in our next release, later this year”

Even though some of these excuses may have a seed of truth in them, the fact is that the team may not have any confidence in the code they are about to deliver or the process is just too tedious to do and that they loath going through it. On top of this, Apple’s reviewing process, although necessary, doesn’t encourage implementing a continuous delivery process.

The first step any team should take towards gaining trust in the code they are delivering is making sure it works according to the specs. Nothing helps more than having an extensive unit test suite. Integration and UI tests are also more than welcomed. Investing time in writing and maintaining automated tests not only helps the team identify and fix bugs faster and easier, but it also lowers the team’s dependency on manual testing which can very often be time-consuming, therefore increasing the confidence in the code it delivers.

Automate the development and release process

So, once you have a great suite of automated tests it is time to put it to good use by building a continuous delivery pipeline around it. These are the two main approaches:

Using Xcode Server - that allows you to configure bots that execute your test suites;
Using Fastlane and a CI environment such as Jenkins, Bitrise, CircleCI, etc.

Personally, I prefer the latter one. Even though it involves relatively more effort in setting up and maintaining, Fastlane comes with more flexibility and a much broader set of plugins and actions that will allow your team to do much more with your continuous delivery pipeline.

Fastlane presentation

Fastlane is a suite of open source Ruby scripts that allows your team to build, test and distribute your application. And is one of the most widely used tools iOS developers rely on in their development process. If you didn’t know about it until now, I highly recommend going to their website to find out more.

If you’re not currently using Fastlane, your development process probably looks something like this:

Run your test suite and make sure not a single one is failing.
Build the app for release.
Run into code signing issues.
Update your certificate and provisioning profile.
Try to build the app again and realize you forgot to update some of the application screenshots so you go and do that.
Update your push notification certificate or key.
Build the app again and upload it to AppStore Connect.
Send a build for smoke testing on TestFlight.
Spot a small typo on the first screen and realize you have to start this process all over.

With Fastlane, all you would have to do is push a single button. Your release process could look as simple as this:

 1 lane :appstore do
 2   cocoapods # Installs Cocoapods dependencies
 3   scan # Tests the app
 4   snapshot # Creates automated app screenshots
 5   match # Installs the certificate and provisioning profile
 6   gym # Builds the app
 7   deliver # Uploads the build to AppStore Connect
 8 end

Simple isn’t it?! And more important than that: PREDICTABLE. Every single time you would run this, the result will be the same under the same conditions, with no human error involved.

Now that I’ve got your attention, let’s jump right in and see how we can make your project benefit from what Fastlane has to offer.

1. Setting up Jenkins

You can download and install on a Mac machine Jenkins from the official website but we recommend using brew:

 1 brew update && brew install jenkins

To start Jenkins, just run the following command in your terminal:

 1 jenkins

You should now be able to access Jenkins as [http://localhost:8080](http://localhost:8080) and the first thing I recommend you to do is install a couple of plugins. To do that, go to “Manage Jenkins” and then “Manage Plugins”.

Plugins that I recommend are:

HTML Publisher Plugin - to publish test reports;
AnsiColor Plugin - to color the build logs output;
Rebuild Plugin - trust me, you will need this;
Git Plugin - For accessing your Git repo;
Gitlab - This one allows Gitlab to trigger Jenkins builds.

You can go on and add more anytime you like or think some plugin can ease the process of building your continuous delivery pipeline such as SSH Slaves Plugin, Slack plugin, etc.

We’ll come back to creating our first Jenkins job right after we write our Fastlane files.

2. Setting up Fastlane

Let’s start installing Fastlane. As previously mentioned, Fastlane is a set of open-source Ruby gems. For your Ruby environment, it is recommended to use rbenv but rvm (that comes as a default on Mac OS) is also fine.

Because you do not want to worry about what version of Fastlane is installed on your machine and what version you have on the Jenkins machine, or simply because you do not want to install all the dependencies when moving to another machine we recommend using bundler to manage the Fastlane installation and not only. If you do not have bundler install, go on and install it.

On the root of your project create a file named Gemfile that should have the following contents for starters:

 1 source "https://rubygems.org"
 2 gem 'fastlane'
 3 gem 'cocoapods'
 4 gem ‘CFPropertyList' gem ‘slather'
 5 plugins_path = File.join(File.dirname(__FILE__), 'fastlane', 'Pluginfile')
 6 eval_gemfile(plugins_path) if File.exist?(plugins_path)

As you can see, we also have the Cocoapods dependency in this file along with slather - which is used for generating code coverage and CFPropertyList - that can be used to update .plist files. This means that we don’t have to worry about having cocoapods installed on the machine that we want to run this build on.

To install fastlane, save this file and then just run the following command in your terminal:

 1 bundle install

After it finishes installing all the dependencies, you should be able to use Fastlane, so let’s start using it. First, run:

 1 fastlane init

After this, you should be presented with four options that look like this:

 1 What would you like to do?
 2 1. 📸 Automate screenshots
 3 2. 👩‍✈️ Automate beta distribution to TestFlight
 4 3. 🚀 Automate App Store distribution
 5 4. 🛠 Manual setup - manually setup your project to automate your tasks

Even though Fastlane suggests trying at least one automation if you are a beginner, we recommend going with the fourth option: manual setup. After this, Fastlane will guide you through some frequently asked questions. We recommend reading through them so you get a better understanding of the concepts behind them.

Once finished, you should now have a Fastlane folder that contains two files: Fastfile and Appfile. We will add a couple more to this folder as we move along.

3. Setting up Fastfile

The firsts thing you should focus on is the Appfile. It should look something like this:

 1 app_identifier "com.company.identifier" # The bundle identifier of your app
 2 apple_id "account@developer.com" # Your Apple email address 
 3 team_id "APPLETEAMID" # Developer Portal Team ID

We then want to write our first lane, so we open the Fastfile. It should look something like this:

 1 default_platform(:ios)
 2 platform :ios do
 3   desc "Description of what the lane does" lane :custom_lane do
 4       # add actions here
 5   end 
 6 end

The first thing we want it to do is to rename that lane and add our first action that runs our tests. To do that, we recommend using scan, an action provided by Fastlane.

4. Scan

We open the terminal at the location of our project and run the following command:

 1 fastlane scan init

This should create a new file in our Fastlane folder named Scanfile, go on and open it. The structure should be pretty self-explanatory and we go on and update it.

 1 # The project scheme you want to build and test
 2 scheme("YourAppName")
 3 # If it should clean the project before testing
 4 clean(true)
 5 # The output report types
 6 output_types("html,junit")
 7 # If it should open the reports
 8 open_report(false)
 9 # The output directory for the test reports
10 output_directory("./reports")
11 # Generate code coverage files
12 code_coverage(true)
13 # Device on which the test will be ran
14 device("iPhone XS (12.2)")
15 # Enable skip_build to skip debug builds for faster test performance
16 skip_build(true)

This file is used by Scan to configure how the Xcode command-line tool should run your application tests. We will have similar config files throughout this step-by-step guide so we recommend carefully going through each line’s comments to better understand why we added that setting. For more details and options, we encourage you to go through the official Fastlane documentation.

We can now go on and add this action to our Fastfile and test it. Our file should look like this now:

 1 platform :ios do
 2   desc "Test the application" lane :test_app do
 3     # run tests
 4     scan 
 5   end
 6 end

Now you should see that the lane is called test_app and that we added the scan action. To run this lane, open the terminal and run the following command:

 1 fastlane test

5. Gym

Now that we’ve added our first action, we must dive deeper to take full advantage of what Fastlane has to offer.

We need to let Fastlane know how it should build our project and to do that I recommend you rely on another action provided by it: Gym. To setup Gym you must run the following command in the terminal:

 1 fastlane gym init

This should create a new file, called... (you’ve guessed it) Gymfile, that you should go on and edit:

 1 # If you use Cocoapods you surely have a workspace workspace("YourAppName.xcworkspace")
 2 # Your default project scheme
 3 scheme("YourAppName")
 4 # The project default configuration to use
 5 configuration("Ad-hoc")
 6 # If it should clean the project before building
 7 clean(true)
 8 # The default export method
 9 export_method("ad-hoc")
10 # The path to your build folder
11 build_path("./build")

This configuration file is tailored more around building ad-hoc builds because this will be the most common use of this action: providing builds for testing.

One thing you might have noticed is the Ad-Hoc configuration. We imply that your projects have three configurations:

Debug - for debugging;
Ad-Hoc - for ad-hoc builds that are signed with an ad-hoc provisioning profile;
Release - a configuration for your release build.

This is common if you use a build distribution system such as HockeyApp (now AppCenter) or Firebase Distribution from Firebase. If you are using Testflight, the Release configuration should suffice.

That being said, let’s now jump to one of my favorite topics: managing your certificates and provisioning profile.

6. Match

One of the most useful tools in the Fastlane arsenal is Match. This tool helps teams manage signing certificates and provisioning profiles. Before you begin, please make sure you haven’t reached the maximum limit of signing certificates. If that is the case, please try clearing up one space for Match to use.

To start using Match in your Fastfile, run the following command in the terminal:

 1 fastlane match init

This will guide you through a setup process in which you will be asked to provide a storage option. My recommendation is git and a git repository. Once you finished, a Matchfile will be generated for you. It should look something like this:

 1 # The storage method
 2 storage_mode("git")
 3 # The repo you want to store your cenrificates and provisioning profiles
 4 git_url("https://gitlab.com/your-account/certificates")
 5 # The default type, can be: appstore, adhoc, enterprise or development
 6 type("development")

The best part about Match is that you can create a secure, shared repository of certificates and provisioning profiles that multiple teams can use.

My recommendation is to use git branches for each team to achieve this separation. Each team will have its own branch that will contain the encrypted certificates (development and release) along with their own provisioning profiles: development, ad-hoc, and release.

In order to do this, you must first generate the certificates and provisioning profiles for one team and then clone that branch for the other ones.

 1 # The storage method
 2 storage_mode("git")
 3 # The repo you want to store your cenrificates and provisioning profiles
 4 git_url("https://gitlab.com/your-account/certificates-repo")
 5 # Your team git branch or “master”
 6 git_branch "myteam"
 7 # The default type, can be: appstore, adhoc, enterprise or development
 8 type("adhoc")
 9 # The app identifiers for which the profiles will be needed
10 app_identifier ["com.company.identifier", "com.company.identifier.extension"]
11 # Your Apple Developer Portal username
12 username "account@developer.com"

Once we’ve done this, let's go on and create the certificates and provisioning profiles:

 1 fastlane match development
 2 fastlane match adhoc
 3 fastlane match release

Your branch repo should now have two folders: certs and profiles. The certs folder should contain two folders that have the development and distribution certificates while the profiles folder should contain folders with the provisioning profiles for each of your targets/app identifiers.

To add another team to this repo simply create a new branch from this one and delete the profiles folder and update the branch name in the Matchfile:

 1 # Other team git branch
 2 git_branch "other-team"

Then you can run the commands for generating the provisioning profiles again (Fastlane match development, etc) and the new team will have its own encrypted provisioning profiles. Pretty neat, isn’t it?! Just don’t forget the encryption password or you’ll have to nuke (Fastlane match nuke) everything and start over again.

One last thing you should do before using match in your project is adding the following line in your Matchfile:

 1 readonly true

This will prevent Match from regenerating the certificates and provisioning profiles again when you run Match from your fastfile.

Now that we’ve covered the testing, building, and code signing parts, we can move on to creating a lane that will build and distribute our app for testing.

7. Build and distribute

Now add another lane to our Fastfile:

 1 desc "Build and distribute the application"
 2 lane :build_and_distribute do
 3   # test the application
 4   test_app
 5   # get the last commit message
 6   change_log_message = changelog_from_git_commits(commits_count: 1)
 7   # install the certificate and provisioning profile
 8   match(type: "adhoc")
 9   # build and archive the application
10   gym
11   # distribute the app via Firebase Distribution
12   firebase_app_distribution(app: "your-app-id",
13       release_notes: "#{change_log_message}",
14       debug: true,
15       groups: "testing"
16   )
17   # upload dsyms to Firebase Crashlytics
18   upload_symbols_to_crashlytics(dsym_path: "./YourAppName.app.dSYM.zip")
19 end

Let’s go through it line by line:

One of Fastlane's capabilities is the so-called “lane switching”. We use this to run our previously defined test_app lane. It’s just a ruby function called from another function.
The second one creates a string with the latest git commit message.
The next one installs the certificates and provisioning profiles we just created. As you can see, we specify what type of profile we want to use.
We then build our app. Gym uses the configuration in the Gymfile.
After the build and archive succeeds, we upload it to Firebase Distribution, but you can use any tool that might suit you better.
Don't forget to upload the dSYM zip file for Crashlytics to symbolicate your crash reports.

Now run this function locally using the command:

 1 astlane build_and_distribute

Ok, this is it for the basic concepts, it's time to go and set up our first Jenkins jobs.

8. Setting up the Jenkins jobs

Open Jenkins in your browser accessing http://localhost:8080, and go “Manage Jenkins” and click on “Configure System”. Under the Gitlab section, you should add your Gitlab configuration and credentials, where the host URL is the base URL of your [Gitlab instance](https:// gitlab.com/).

Next click on “New item” and you should see a form similar to the one below.

Let’s name this one YourAppName-Test because it will be responsible for testing your app. In the “General” section select the Gitlab connection that you have just created.

Under the “Source Code Management” section please select the Git option.

Type in your repo URL in the “Repository URL”, add or select your Gitlab credentials from the “Credentials” option and specify what branch would you like to be tested - let’s go with origin/develop - which should check out the develop branch.

You might want certain commits not to be built. To do that you could add an additional behavior that can ignore commits with a specific message. Let say you want all builds that contains skip-ci in the commit message to be ignored. You can add (?s).*skip-ci.* in the Exclude Messages and all commits containing skip-ci will be ignored (for example: [skip-ci] Bump version number).

In the “Build Environment” section please select “Delete workspace before build starts”.

Finally, we are at the crucial part: building and testing the app. In the “Build section” add an “Execute Shell” build step that contains the following:

 1 source ~/.bashrc
 2 export LANG="en_US.UTF-8"
 3 bundle install
 4 SNAPSHOT_FORCE_DELETE=1 fastlane snapshot reset_simulators fastlane test_app

The first file loads your bash configuration file. The second one is a requirement for Fastlane to properly display the run logs. Keep in mind that at this point the job has already checked out your develop branch, so the next line is for bundler instal the project dependencies: cocoapods, etc.

We then reset all the simulators, to run our tests on a fresh environment (someone might have used them before our job is ran) and after that, we run the test_app lane just like we did on our machine.

It should look something like this:

You can add a Post-build Action to publish the JUnit test result report. As you might remember, when we configured Scan, we added two options: one to publish JUnit and html reports and one that specifies where to publish them. Now go on and add the path in the Test report XMLs to reports/cobertura.xml.

And that’s it! Save your job and go ahead and run it and at the end of the log you should see something similar:

 1 [08:20:04]: fastlane.tools just saved you 11 minutes! 🎉 Recording test results
 2 Finished: SUCCESS

Now go on and create another job to automate the distribution of the app. Let’s do that right now only this time we will use what we have already configured so in the Copy from section type the name of the job you have just created.

Let’s name it YourAppName-Distribute.

This will create a copy of the job you have just set up and we’ll only change two things: enable “Build when a change is pushed to GitLab” - Push events in the “Build trigger” section and change the following line in the Execute Shell Build Phase: fastlane test_app to fastlane build_and_distribute.

And you’re done. You have a job that can build, test, and distribute your app.

We’re on a roll here, so let’s add one more. This one will be a little bit fancier but will have the same base as the previous one. Start by creating a new job from the test one that we have created the first time. Go on and name it YourAppName-MR because it will be responsible for building merge requests open in Gitlab.

Under the Source Code Management section change the “Branch specifier” section from origin/develop to origin/${gitlabSourceBranch}. This will checkout the branch specified as a source in the merge request.

Now let’s add an additional behavior to “Merge before build” that should look like this:

The name of the repository: origin

Branch to merge to: ${gitlabTargetBranch}

Merge strategy: default

Fast-forward mode: —ff

In the “Build trigger” section check the “Build when a change is pushed to GitLab. GitLab webhook URL” and select push events, Open merge Request events, and Comments. In the Comment field you can add a comment that will trigger a re-build of the merge request, Jenkins please retry for example.

We did a lot of work in Jenkins in this chapter and to finish it all up we will need to link everything up with some Gitlab webhooks.

9. Triggering Jenkins jobs from GitLab

Gitlab has to communicate with Jenkins when different events happen: a push, a merge request, a comment on a merge request to rebuild the merge request, etc. To do that we can use Gitlab webhooks. To add a webhook, go to your Gitlab project and click “Integrations” under the “Settings” menu. It should look something like this:

Here you can add your webhooks, we’ll need two:

one for changes pushed to the develop branch - to distribute builds
one for opened merge request - to merge, build and test the merge request.

Merge requests (to the develop branch) that are accepted will then trigger the distribute job as it will result in another push to the develop branch.

Hope this flow is clear so let’s jump straight to it.

The first webhook you should add is the one that will trigger your distribute job. In the URL section, you should add the URL to the Jenkins job, is should look like this:

 1 http://localhost:8080/project/YourAppName-Test/

If you have Jenkins installed on a different machine, please use its address instead. The only thing to check from the list of options is “Push events” and add “develop” in the branch name text input underneath it - this will only trigger for builds on the develop branch. Save it and test it, and it should queue a job in Jenkins.

The second webhook you must create should do a POST request to the merge request job:

 1 http://localhost:8080/project/YourAppName-MR/

And the triggers should be the following:

Comments
Confidential Comments
Merge request events

You can now save and to test that this is working properly open a merge request in your git repository or comment with your re-build comment phrase that you have previously set (“Jenkins please retry”).

By now you should be able to build, test and distribute your app in an automated manner and start saving precious time.

Bonus! Gathering SonarQube metrics using Fastlane

This part is optional but it brings a lot of value to teams that want to keep an eye on the code quality of their project and what to take action towards constantly improving it.

To install SonarQube and its dependencies, please follow the instructions found here.

To start running it just type sonar start in the terminal. You should be able to access it at https://localhost:9000.

Before we dive into updating the sonar-project.properties we should set up Slather - a tool that handles code coverage for your project. Please create a .slather.yml file in your project (yes, with a dot in front) that looks something like this:

 1 coverage_service: cobertura_xml
 2 xcodeproj: YourAppName.xcodeproj
 3 workspace: YourAppName.xcworkspace
 4 scheme: YourAppName
 5 source_files:
 6 - YourAppName/*
 7 output_directory: reports/ ignore:
 8 - YourAppNameTests/* - fastlane/*
 9 - Pods/*

This will be used to generate a code coverage report that will be sent to SonarQube.

Next we should set up our SwiftLint rules. To find out more about that please go to https://github.com/realm/ SwiftLint to find out more about it and how your project’s .swiftlint.yml like should look like. Please check the full set of rules and pick what fits best your project.

Let’s head back to our Fastfile and update the test_app lane first.

 1 desc "Test the application"
 2 private_lane :test_app do |options|
 3   # get the publish_reports option
 4   publish_reports = options[:publish_reports]
 5   # run tests
 6   scan
 7   # check if it must publish reports
 8   if publish_reports
 9     publish_reports_to_sonar
10   end 
11 end

After that add the new private lane called by the test_app lane.

 1 desc "Publish the application reports to sonar"
 2 private_lane :publish_reports_to_sonar
 3   # Run slather to generate code coverage
 4   slather(proj: "YourAppName.xcodeproj", 
 5     workspace: "YourAppName.xcworkspace",
 6     scheme: "YourAppName",
 7     configuration: "Debug",
 8     html: true,
 9     cobertura_xml: true, 
10     output_directory: "./reports"
11   )
12   # Run swiftlint to detect code smells (you might need to specify the executable path
13   swiftlint(output_file: "./reports/swiftlint.txt", ignore_exit_status: true)
14   
15   # Run lizard to check code complexity
16   lizard(source_folder: 'YourAppName',
17     export_type: 'xml',
18     report_file: 'reports/lizard-report.xml')
19   # Run sonar
20   sonar
21 end

Add the new lane that will be called by our merge requests:

 1 desc "Test a merge request"
 2 lane :test_merge_request
 3   # test the app without publishing the reports
 4   test_app(publish_reports: false)
 5 end

And a new lane that will be called from now on by our test job:

 1 desc "Run tests"
 2 lane :run_tests
 3   # Test the app and publish the reports
 4   test_app(publish_reports: true)
 5 end

As you can see above, we have taken advantage of lane parameters to be able to add more complex logic to your Fastline.

The final configuration step is to edit the sonar- project.properties. We won’t go through it here but you should keep in mind that we keep most of the reports in the reports folder. The file provided by the Backelite team should be a good starting point. After all of this has been done, the Jenkins test and merge request jobs should be updated to call the run_tests and test_merge_requests lanes.

There you have it, you should be able to check your project code coverage and code smells in SonarQube.

More articles from Ciprian Redinciuc can be found on Userdesk or our OceanoBe technical article blog section.