How To Use GitLab CI To Run Tests Locally

Overcoming Local Testing Challenges

If there is one thing you can always do, maintain the test suite! In a way, to every bug you patch, add new tests to every new feature.

Running GitLab CI via Docker Locally

Only add your .gitlab-ci.yml file to your project, create a new branch, push it, create a Merge Request, and if you like, the CI will kick in, with several parallel jobs. The CI would not let me fail, even though I forgot to run my tests.

However, keeping a CI up and running allows you to pause and maintain your specs. You will often wonder why a test runs on your local computer, but does the CI server continue to fail?

That is why it is super helpful to run the CI docker image locally, to iron out quirks based on the rest of the world.

And by running the GitLab CI Runner itself, you can do just that. It will pick up the .gitlab-ci.yml of your project and run it via Docker locally. So ultimately, any issue you see on the server will undoubtedly happen locally, ensuring good test coverage locally. There are queues of jobs waiting to run with the added advantage that you don’t need to wait in line in case.

Configuring TestMu AI Tunnel to Test Locally

The TestMu AI Tunnel allows you to connect your local system with TestMu AI servers via an SSH-based integration Tunnel. Once this Tunnel is established, you can perform local page testing on all 3000+ browser environments present on the platform.

Apologies for throwing a few jargons like Gitlab CI Runner / .gitlab-ci.yml / Docker / Lamba Tunnel ahead of any introduction. But do not worry, the guide is intended to deep dive into several of these strategies to circumvent quirks in running tests locally.

How to Test GitLab CI locally?

To configure GitLab CI locally, it is essential to understand .gitlab-ci.yml file creation.

The .gitlab-ci.yml file that resides in your repository root is where you configure what your project is doing with CI. It specifies the structures of the pipelines and their order and determines:

• What to perform using the GitLab Runner.
• What decisions to make when particular situations are encountered. For instance, when a process is successful or fails.

Upon any push to your repository, GitLab will scan for a .gitlab-ci.yml file and start jobs on Runner based on the contents of the file.

Because .gitlab-ci.yml is in the repository and is version managed,

• Old versions are still successfully installed.
• Forks can easily use CI.
• Branches can have multiple pipelines and jobs.
• For CI, you have a single source of reality.

Creating a simple file for .gitlab-ci.yml

In the root directory of your repository, you must create a file called .gitlab-ci.yml. This is a YAML file, so you need to pay special attention to indentation. Always use tabs, not spaces.

A sample .gitlab-ci.yml for a Ruby on Rails project

For most Ruby applications, this is the easiest possible configuration that will work:

• Specify two jobs, RSpec and RuboCop jobs (the names are random), with various commands to execute.
• The script commands are executed before each job.

Jobs are the fundamental part of the .gitlab-ci.yml file. It specifies job sets with restrictions as to how and when to run them. The jobs are classified as top-level elements with a name (RSpec and RuboCop in this case) and must always contain a keyword – ‘script.’ Jobs are used to build jobs, which are then chosen and executed by Runners within the Runner’s environment.

Each job must run separately from the other.

Validate .gitlab-ci.yml-ci file

Each GitLab CI / CD instance has an embedded debug tool that validates the content of your .gitlab-ci.yml files, called the CI Lint tool. Under the ci/lint page in your project namespace, you will find the Lint. HTTPs:/gitlab.example.com/gitlab-org/project-123/-/ci/lint, for example.

For a graphical representation of your .gitlab-ci.yml file, you can use the CI/CD configuration visualization. For more detail and the full .gitlab-ci.yml syntax, please read the reference documentation on .gitlab-ci.yml.

Let’s say you have a .gitlab-ci.yml file, and you want to test this file locally.

Gitlab Runner is the tool to accomplish this! It is an application that works to run jobs in a pipeline with GitLab CI / CD.

Installation:

Install GitLab Runner on the infrastructure you own or control but just make sure that it is installed on a different machine from the one that hosts an instance of GitLab.

GitLab Runner installation is supported by various OS. The following table identifies supporting compatibilities and installation links.

See some best practices on how the GitLab Runner should be used and administered.

Also, we should make sure GitLab Runner should be the same version as GitLab.

Registration:

You need to register individual Runners after installation. When you do so, you establish a communication between your Gitlab instance and the machine which has Runner installed.

Typically, Runners process jobs on the same machine where you installed GitLab Runner. However, you may also have a Runner process job in a container, in a Kubernetes cluster, or in auto-scaled instances in the cloud.

Depending on which OS/platform/binaries Runner is installed, you can refer to the Runner registration guide for registration steps.

Executors:

An Executor decides the environment each job runs in. When a Runner is registered, you must select an executor.

Several Executors supported by Gitlab Runner:

• SSH
• Shell
• Parallels
• VirtualBox
• Docker
• Docker Machine (auto-scaling)
• Kubernetes
• Custom

The table below shows the main facts that will help you determine which executor to use with each executor.

• Using Vagrant
• Dependent on what kind of environment you are provisioning. It can be completely isolated or shared between each build.
• When access to a Runner file system is not secured, jobs can access the entire system, including the Runner token, cache, and code. Executors marked with ‘✓’ do not allow Runner by default to access the file system.

Features supported by different Executors:

Of all the choices mentioned above, below could be best opted.

• You could install GitLab Runner on a Windows server and then register a Runner that uses the shell executor if you want your CI / CD job to run PowerShell commands.
• You could install GitLab Runner on a Linux server and register a Runner that uses the Docker executor if you want your CI / CD job to run commands in a custom Docker container. It is often referred to as a “Docker-in-Docker” configuration when installing GitLab Runner in a Docker container and selecting the Docker executor to run your tasks. We would see a couple of implementations of testing locally using different GitLab Runner & executors in the subsequent section.
• You can install GitLab Runner on a virtual machine and have another virtual machine used as an executor.

GitLab Runner commands

Below are the set commands used to build, test, register, etc., you would see by giving ‘gitlab-Runner’ without any argument.

The ‘exec’ command is our apple of the eye!

‘gitlab-Runner exec’ command is the command that easily lets you test builds locally.

It allows the jobs specified in .gitlab-ci.yml to run locally! This, in turn, allows for quicker test cycles and makes it easier for broken builds to be patched.

The way it works is, it aims to reproduce the CI Environment as similar as possible to the way it should function. Instead of connecting to GitLab instance, it reads the local .gitlab-ci.yml file and creates a new environment where all the build steps are executed.

Since gitlab-Runner exec clones the current state of the local Git repository, be sure to have committed any changes you want to test beforehand.

The exec command should be executed directly from the root directory because .gitlab-ci.yml is present there. And while executing it, you must specify the executor and the name of the job.

The following instruction, for example, executes a job called ‘test’ locally using a shell executor.

‘gitlab-Runner exec shell tests’

Let us now try to use all the above information to test a change locally!

Testing Using Docker as an Executor

In order to use Docker, follow the below-mentioned steps:

• Install Bleeding Edge Runner release locally.
• Run the build: ‘gitlab-Runner exec docker my-local-job.’ ( the job name is arbitrary*)

This will run ‘my-local-job’ defined in the docker container specified in the local .gitlab-ci.yml file.

Note: https://docs.gitlab.com/ee/ci/yaml/README.html : Even though job names are arbitrary, there are specific keywords listed in this link that you cannot use as job names. An excellent link that extensively contains keywords, syntax, and usage.

Using exec with Docker

• Install a docker-machine.
• Create a new docker VM: ‘docker-machine create -d virtualbox Runner.’
• Configure shell settings : eval $(docker-machine env Runner).

For best practices on Docker as an executor, go through the document on GitLab Runner with Docker.

Using Shell as an executor on MacOSX:

• Install gitlab-Runner via brew.
• Register the gitlab-Runner by URL and token in Setup a specific Runner manually section in Project Settings > CI / CD.
• Now we must enter the executor type. In this case, it’s Shell.
• Execute the ‘gitlab-Runner register’ command on the Shell. A wizard opens when this command runs on the shell. Anything written about the project, such as a description, will be shown in Project Settings > CI / CD > Runners tab.
• Assume to have a .gitlab-ci.yml file with two stages – Test & Deploy.

• Executing the ‘gitlab-Runner exec shell my_project_test’ will run the job ‘my_project_test’ under stage ‘test’ locally.

Basically, in both cases, the gitlab-Runner exec command clones the project from the Gitlab repository, installs all related dependencies, and runs the tests locally!!

This was all about introducing GitLab CI to run tests locally and overcome significant challenges of testing locally otherwise.

Run Selenium Test in GitLab CI locally Using TestMu AI Tunnel

The TestMu AI GitLab Integration allows you to create a card directly in your repository from the TestMu AI platform.

Integrate and automate your Selenium test suites with GitLab CI on TestMu AI.

TestMu AI Tunnel App can be used to execute local tests efficiently. This is a confidence booster before deploying any UI change into production because it lets you test your development thoroughly with the Secure Shell(SSH) network protocol hosted by TestMu AI Tunnel. Experience the flexibility to securely test your website on 3000+ browsers and their assorted versions, running on real OS code for cross-platform and cross browser compatibility, even before it gets deployed over the production environment. Before any configurations, download LambdaTunnel for your OS using,

• Download for Windows
• Download for MacOS
• Download for Linux

The below .gitlab-ci.yml file would show you how to leverage TestMu AI’s cloud Selenium Grid by performing a browser test through your testing server contained in your GitLab CI build.

// Example of .gitlab-ci.yml for Lambda Tunnel
 
image: node:7.10
 
before_script:
  - |
    cd /builds/qarachit/protractor-selenium-sample-gitlab
    wget https://s3.amazonaws.com/lambda-tunnel/LT_Linux.zip
    #Extracting tunnel binary
    sudo apt-get install unzip
    unzip LT_Linux.zip
    #Executing tunnel library
    ./LT -user ${LAMBDATEST_EMAIL} -key ${LAMBDATEST_KEY} &
    sleep 30
    #Installing Dependencies
    echo 'export LT_USERNAME="{your_lambdatest_username}"' >> ~/.bashrc
    echo 'export LT_ACCESS_KEY="{your_lambda_access_key}"' >> ~/.bashrc
    npm install
    npm install -g protractor
  
job:
  script:
    #Running Test!
    protractor single.conf.js
 
Once the tunnel is successfully set up, you can start testing in your network by adding the below code.
"desiredCapabilities": 
{ "tunnel":true }

While you get TestMu AI Tunnel configured on your suitable machine, if you are working in a team, you could also configure Tunnel to be operated in shared mode!

This removes any overheads arising in shuffling their SSH Tunnel connection across team members to enable access across the team. Maybe a case where a team has, say, seven members, and everyone configured their Tunnel.

Now, a user can only work on a user-specific SSH Tunnel connection. In case you wish to keep a common SSH Tunnel connection across all your teammates, you will need to establish a shared TestMu AI Tunnel that would work for all of the teammates listed under your organization at TestMu AI.

Integrate and automate your Selenium test suites with GitLab CI on TestMu AI.