CENTOS v 6.4, GIT v 1.7.1 and JENKINS v 1.535 CONTINOUS INTEGRATION

Having tuned our Agile process to release iteratively and often I decided it was time to spend some time looking at how we could introduce continuous integration into our PHP and Javascript BDD workflows. Given that Travis isn’t suitable for most of our work (your code must be open source), I chose Jenkins as our CI server and was able to get up and running fairly quickly. I  found a few resources that covered integrating Git with Jenkins but I ended up doing a bit of digging myself so thought I’d quickly share the steps I followed.

Install Jenkins and Git

I provisioned a fresh CENTOS v 6.4 box, which meant I was able to follow the official Jenkins docs without any problems. If you’re using anything other than Ubuntu/Debian or Redhat you may need to look elsewhere. The installation instructions outline simple Apache or Nginx vhost configurations that can be used to serve the Jenkins administration console. You’ll need to install git on the same box so that Jenkins can eventually pull down, or commit to, your git repositories. To install Git I simply ran $ sudo apt-get install git (debian) and $ yum install git (Centos).

Install the Git Plugin

Once you can access your Jenkins console,  goto `Manage Jenkins -> Manage Plugins` from the home screen.
Open the ‘Available’ tab and find the plugin entitled Git Plugin.There is a filter box but it didn’t work particularly well for me, I ended up using Find in Chrome.

Create ssh key pair

Part of the installation process will create the user `jenkins` which will envoke all Jenkins processes, including all git commands. Therefore you’ll need to provision a keypair for this user then add the public key to your git repo. There’s a bit of trick to doing the former which I’ll cover now:

1. Login to your box and switch to the Jenkins user. The installation process doesn’t create a password so you’ll need to have root/sudo permissions to do this. Run the command sudo su - jenkins. The ‘-’ specifies a login shell, and will switch you to jenkins’ home directory (for me this was /var/lib/jenkins’).
2. Create a .ssh directory in the jenkins home directory.
3. Create the public private key pair. There are many tutorials which cover using the ssh-keygen command to do this. The most important thing is not to set a password, otherwise the jenkins user will not be able to connect to the git repo in an automated way.
4. Add the public key to your Git repo. We use bitbucket so this was fairly straightforward for me and I imagine anyone reading this will have performed similar actions for all their devs keys in the past.
5. Set a git user and email address. This is also mentioned in the git plugin documentation. Run:cd /srv/jenkins/jobs/project/workspacegit config user.email "some@email.com"git config user.name "jenkins".

Connect to the Git repo. This is a one time step which will dismiss that ‘Are you sure you want to connect’ ssh message, again jenkins won’t be able to deal with this. Just run ssh git@your_git_server_url info.

Create a new Job in Jenkins

The official docs provide a good level of detail on how to configure a basic Jenkins job, so I’d recommend following them here. The git-plugin docs also provide some useful info, so there’s not too much more for me to say other than list the steps I followed:

1. Select Git in Source Code Management.
2. Enter your git repository URL. This performs a asynchronous check and will give you an error if it can’t connect. Double check you followed the steps in #3 if you get an error.
3. Select the branch to build. This is branch jenkins will pull from when a build is started so enter whatever is suitable. We are building from develop so I entered this here.

Run a test Build

Once you’ve saved the job it’s worth kicking off a build to check everything’s working OK. You can do this in the main dashboard. Have a look at the latest build, you should see the commit id that was pulled down. If you dig around in the docs you’ll find that the plugin will have fetched your branch (in my case develop) and pulled from the remote repo. The repo is checked out into the job’s `workspace` directory and during the build the Jenkins user is cd’d into this directory. By adding your makefiles and build scripts into your repository it’s then a straightforward case of configuring the Jenkins job to execute these upon build. Again, this is covered in good detail in the docs.

Post Receive Hook

Finally you’ll need to setup your Git repository to initialise a build each time you push to your repository. Github and bitbucket both have hooks ready to be used, but if you’re self hosted a good place to start is at the plugin documentation which specifies the HTTP endpoint which can be used to trigger the job.

Git for Windows with TortoiseGit and GitHub

For this project, we are required to use Git for version control and host code on GitHub. As a person who has always preferred SVN (for its linear/incremental nature), and have only had experience with Google Code and SourceForge, setting up a new GitHub repo was a new experience. And so, for future reference, I document the process here. In this article, we will install msysgit and TortoiseGit and host code on GitHub.

Installing TortoiseGit:

http://code.google.com/p/tortoisegit/ (yes, its hosted on Google Code) TortoiseGit is a shell extension for Windows Explorer that allows for easy management of Git projects. Its a port of the popular TortoiseSVN project, but for Git. First step would be installing Git for Windows (msysgit), which will also give you a nice, simplistic bash shell that you can work with (and execute the above commands just like on Linux if you'd prefer that over TortoiseGit). I personally just downloaded and installed Git 1.7.9 with all default settings. Next was installing TortoiseGit itself, which should autodetect the installed Git setup (though you can configure it later if you haven't) and add itself to the Windows Explorer right-click context menu.

Generating your public key:

To associate your computer with GitHub, you will need a "public key/private key" pair. You could do this entire process manually with the bash shell, but TortoiseGit gives you a handy GUI tool for it: Puttygen (PuTTY Key Generator). Run Puttygen via your Start menu: "Start -> Programs -> TortoiseGit -> Puttygen". Click "Generate", move your mouse around the blank box for a bit, and wait for the box to fill with your newly generated key. Copy the entire text in the box (right-click -> Select All, right-click -> Copy). This is your public key. Then click "Save private key" and choose a safe place to store the .pkk file (e.g. "TortoiseGit.ppk"), passphrase optional. This file stores your private key.

Adding your private key to TortoiseGit:

Next you want make your private key accessible to TortoiseGit for authenticating with GitHub. For this, we'll manage our key list with Pageant: "Start -> TortoiseGit -> Pageant". This will add an icon to your taskbar (see picture below). Right click the icon and select "View Keys". Click "Add Keys" and navigate to your previously saved PuTTY key that is storing your private key file (e.g. "D:\Development\Sandbox\TortoiseGit.ppk"). If added properly, you should see the new key added to the list.

Registering for GitHub:

https://github.com/signup/free
GitHub is a project hosting website that focuses mainly on Git. As of the moment, they offer free public repositories and unlimited public collaborators for open source projects. Registration is pretty straightforward - just sign up for a free personal account.

Adding your public key to GitHub:

https://github.com/settings/ssh
Go to your "Account Settings" on GitHub and go to the "SSH Keys" section. Click "Add SSH key" and paste your public key into the box. Give the key a descriptive title (it doesn't really matter) and then click "Add key". After confirming your GitHub password, you should see the new key added to your "SSH Keys" list.

Creating a GitHub repository:

https://github.com/repositories/new Go to your GitHub profile page (click your username on the top right) and click "New repository". Proceed to fill out the project information. Note that the "Project Name" becomes the repos name, so you may want to use a clean, space-less name (e.g. "PlanmanupNewGITProject" for this project). After clicking "Create repository", you'll be presented with a list of commands to execute to create your first commit. Since we'll be using TortoiseGit instead of a command-line to manage files, you can ignore

Creating your local repo

Now that the project has been created on GitHub, its time to create it on your computer! The GitHub page will give you bash commands for creating the new repos, but we'll use TortoiseGit instead. Navigate to the folder/drive where you want your code to be stored (e.g. "D:\Development\Sandbox"). In that folder, right click to open up the Windows Explorer context menu and select "Git clone...". In the popup box, copy-paste the Git repo URL from GitHub (see above picture) and click "OK". Since you've already set up the authentication earlier and its an empty repository, it should be quick (the log will warn you about cloning an empty repo though). If a new folder has been created with the name of your project (e.g. "D:\Development\Sandbox\PlanmanupNewGITProject"), the clone was successful!

Push files/changes to your local branch

What you have now is a clone of a branch of your repository. Your current branch is probably the master branch assuming you just created a clean project. Go ahead and add files and your code to the folder. Once you're happy with your code/changes, it's time to commit these changes. You can do so by right clicking to open up the context menu in the clone's folder and selecting "Git Commit -> "master"...". This will open up a window that will allow you to add the change message and select what changes you want to commit. If its your first commit, you will want to check the "Select/Deselect All" checkbox to add all your new files. Click "OK" when you're happy to commit the changes to your branch. Note that unlike SVN, this commit is LOCAL ONLY!

Synchronizing changes with GitHub

After a few commits, you might want to push your changes to GitHub. Open up the context menu in the clone's folder and selecting "TortoiseGit -> Push". In the new window, if you also added tags, don't forget to check the "Include Tags" checkbox before clicking "OK". If other project collaborators pushed out changes, you can pull their changes from GitHub with "TortoiseGit -> Pull".

Associating your private key with the git repos

To avoid having to manually start up the Pageant daemon every time you want to sync your git repos with GitHub, you can add your PuTTY key to TortoiseGit's Remote settings. Open up the context menu in the clone's folder and selecting "TortoiseGit -> Settings". In the new window, select "Git -> Remote" and select your Remote item (in my case, "origin"). Locate and select the .ppk file from earlier for the "Putty Key" field and click "OK". Now you don't have to manually start up Pageant every time you reboot your computer!

You're done! and that's it! If you check your GitHub project page, you should see your latest commits and navigate your code's different branches and tags.