Introduction

Welcome to the Samagra DevOps guide!

Whether you're a startup or a large enterprise, your ability to deliver software quickly, reliably, and securely is critical to your success. That's where DevOps comes in.

DevOps is a set of practices that combines development and operations teams to work together in a more agile, collaborative, and automated way.

At Samagra, we're passionate about adopting DevOps practices and improve the software delivery capabilities. In this guide, we'll explore the key concepts, tools, and processes that make up a successful DevOps strategy. Whether you're just getting started with DevOps or looking to improve your existing practices, this guide will provide practical tips and best practices to help you achieve your goals.

Let's get started with the guide.

Infrastructure Overview

image

  1. Development : Developers work on code changes locally and push their changes to a GitHub repository.

  2. GitHub Actions : When a code change is pushed to GitHub, a GitHub Action is triggered to run automated tests on the code. This ensures that any issues are caught early and can be fixed before they are deployed.

  3. GitHub Webhooks : Once the code is merged, GitHub webhooks trigger a Jenkins job to build and deploy the code changes to a staging environment.

  4. Jenkins : Jenkins runs a build pipeline to verify the build.

  5. Deploy : Jenkins has another pipeline (deploy-staging) which deploy the code changes to the production environment using Ansible playbooks.

  6. Hashicorp Vault: It can be used to manage secrets that are required by the various tools and processes involved in the pipeline. Ansible fetches these secrets from here.

Github

Repositories

  • samagra-devops : It is the main repo that contains Jenkins job and Ansible roles for services that are reproducible in other projects at Samagra.
  • Each project then contains a (project-name)-DevOps repo. For example ULP-Devops etc.

Project Structure

For each project which we have as pipeline, In their github repository we follow a common structure for getting the Dockerfile and Jenkinsfile.

root-project-repo/
    ├── Dockerfile
    ├── build/       
            ├── Jenkinsfile

The Dockerfile needs to be tested by developers for any errors and once it is ready, we can use it to deploy the project in private docker registry on docker swarm.

The Jenkinsfile contains all the necessary steps for several stages in a pipeline which eventually builds the project on server and deploys it.

Webhook URLs

image

In each project, we have the webhook which triggers the events occuring on github to the jenkins which helps in detecting the changes going in a repository.

Jenkins View

Dashboard

It is the main Jenkins screen which you would be able to see just after login.

image

Project Space

In project space there are directories for specific projects or job.

image

Pipelines

For each job we have two pipelines.

  • build
  • deploy-staging
  • deploy-prod (third pipeline yet to come) image

Build Pipeline

This is a multibranch pipeline and it can make Docker image builds of all branches and new PRs that are created. Images are tagged by branch name or PR-(Number) and pushed to a private docker registry.

image

Deploy-staging

Whenever commits are made to main/master branch, this job automatically deploys the code to the server and pushes a notification on the discord channel regarding the success and failure of the jobs. image

Deploy-prod

This pipeline will be used to deploy to the prod server and will be manually used to deploy the changes image

Jenkins Configuration

As we have the build pipeline running the stages as defined in the Jenkinsfile for any project, we have a lot more going once the build is succedded. Many things are configured into the deploy staging pipeline.

  • Running the ansible playbook for testing.
  • Sending the build status notification to Samagra discord server.

Jenkins allow us to configure all such tasks.

image

  1. Creating build tags as string parameters.

image

  1. Running Shell commands after each build runs.

image

Here as you can see, the tag parameter which we created in step 1 gets used with ansible-playbook. You can create several parameters if needed.

  1. Setting up discord notifier using webhook URL.

image'

If need arises, you can manually deploy the projects using the build with parameters option.

image

Jenkins Build Runs

You would wish to see the build history and the logs in case if the build fails. This section is all about knowing about where the builds history would be visible and where you can see the output logs.

image

Now, If you click on any build ( Eg #45 ) then you get more clear overview about the job.

image