Continuous Integration

Background:

"Continuous Integration (CI) is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly." (Martin Fowler, http://martinfowler.com/articles/continuousIntegration.html )

A good tutorial that explains Hudson server architecture and usage can be found here .

sipXecs CI:

sipXecs CI is using Hudson as CI server (Hudson is a popular web-based continuous integration server, written in Java, used in 17.000+ server installations and liked for its ease of use and broad extensibility via 250+ plugins).

Architecture

sipXecs use a distribute CI architecture where both master and slaves are configured as Amazon EC2 instances. Master role is to invoke builds on slave machines (configured as new instances at build time in case of nightly builds or as predefined slave containing Hudson agent and launched via execution of SSH command on master: Manage Hudson > Manage Nodes > ci-slave1.sipfoundry.org). No build is performed on master instance.

Security

sipXecs CI server is integrated with Crowd (meaning that you can use same JIRA or wiki account to login in its console). At this moment only the following groups (besides administrators with full control) are granted with access (list is open and will be modified based on community feedback): sipXecs-commiters and sipXecs-developers can start new builds and see current jobs status.

Notifications

sipXecs CI make use of email (on build success and failure, before starting a new build) and IRC (channel #sipx) notifications.

Type of builds

At the moment there are two type of builds performed by sipXecs CI server:

- nightly builds invoked per daily basis at midnight that perform clean builds: create an EC2 image, install all dependencies, clone git repository, issue make build

- sanity builds triggered by commits in sipXecs git repository : runs on a Amazon EC2 instance already configured with required dependencies and git repository, pulls latest sources and issue a make build

TBD: component related builds - as sipXconfig's precommit - could be configured as standalone build or integrated with nightly and sanity if needed

Hudson plugins used by sipXecs CI

Plugins can be configured (add, remove, disable, enable) in Hudson console under Plugin Manager (Manage Hudson > Manage Plugin) page. The list below contains Hudson plugins used by sipXecs CI. If you have a favorite plugin that you want to be integrated with sipXecs CI please start a discussion on the mailing list (check Hudson plugin list for reference)

Plugin name	Version	Description
Hudson Cobertura Plugin	1.0	Used for integrating Cobertura coverage reports in sipXecs CI email notifications
Crowd integration	1.1	Used for integrating sipXecs CI with Sipfoundry Crowd
Amazon EC2 Plugin	2.9	Used for starting new Amazon EC2 instances for running nightly builds on
Git Plugin	1.1.1-SNAPSHOT	Used for polling sipXecs repository - version installed for fixing http://issues.hudson-ci.org/browse/HUDSON-7547
Instant Messaging Plugin	1.10	Prerequisite for IRC plugin
IRC Plugin	2.7	Used for sending notifications on sipXecs IRC channel
SSH slaves plugin	0.14	Used by master for starting Amazon EC2 image as Hudson slave

General Settings (Manage Hudson > Configure System)

Setting	Value	Description
# of executors	0	no build to be executed on master
Quiet period	5	waits 5 seconds before issuing the build
Security Enabled	Crowd	integrates with Sipxfoundry Crowd Realm
IRC Notifications	enabled	sipXecs_buildbot sends notifications on #sipx channel
Amazon EC2 (used by nightly build only)
Instance Cap	3	instance will be launched for nightly build only if the number of instances in configured AMI does not exceed 3
Label	nightly	only nightly builds will make use of this feature
Number of executors	1

The Amazon EC2 plugin init script (used for system setup and executed after Hudson connects to a new Amazon EC2 instance):

yum -y install java git* gcc gcc-c++ autoconf automake libtool subversiondoxygen rpm-build
zip httpd-devel openssl-devel jpackage-utils which unzippcre-devel expat-devel unixODBC-devel
createrepo jakarta-commons-beanutilsjakarta-commons-collections jakarta-commons-net ant log4j 
junit ant-commons-loggingant-junit ant-trax ant-nodeps mod_ssl postgresql-server libXp zlib-devel 
postgresql-develruby ruby-devel net-snmp-devel net-snmp-perl libpng-devel libart_lgpl-devel 
freetypefreetype-devel oro scons rpmdevtools redhat-rpm-config alsa-lib-devel 
curl-devel gnutls-devellzo-devel gdbm-devel mysql-devel ncurses-devel python-devel
perl-devel perl-ExtUtils-Embed termcap bind
wget -P /etc/yum.repos.d http://download.sipfoundry.org/pub/sipXecs/sipxecs-rhel-snapshot.repo

yum -y clean all

yum -y install sipx-openfire cgicc-devel java-1.6.0-sun-devel ruby-dbi rubygemsw3c-libwww-devel 
cppunit* ruby-postgres xerces-c-devel jakarta-commons-net nsissipx-freeswitch sipx-jasperreports-deps

gem install -y rake
gem install -y file-tail | installs all needed dependencies

Nightly Build Settings (master-4.2-nightly job)

Setting	Value	Description
Max # of builds to keep	10	only latest 10 build logs / status are available for review
Restrict where this project can be run	nightly	same as label added for Amazon EC2 plugin
Source Code Management	Git
Build Triggers	periodically
Schedule	@midnight	runs nightly build at midnight
Build, Execute shell step	autoreconf -i make build	runs autoreconf and invokes make build
Post build actions		E-mail notifications sent on failure and success to all recipients within the configured list (if you want to receive emails with nightly builds status let us know), IRC notifications (on #sipx channel)

Sanity Build Settings (master-4.2-sanity job)

Setting	Value	Description
Max # of builds to keep	10	only latest 10 build logs / status are available for review
Restrict where this project can be run	ci-slave1.sipfoundry.org	same as label added for preconfigured slave, sanity builds can run only on this Amazon EC2 instance
Source Code Management	Git, URL of repo: git://github.com/dhubler/sipxecs Branches to build: master-4.2
Build Triggers	Poll SCM	build will be triggered as a result of a new commit
Schedule	@hourly	polls git repo for changes one time per hour
Build, Execute shell step	autoreconf -i make build	runs autoreconf and invokes make build
Post build actions		E-mail notifications sent to commiters before build, on failure and success to all recipients within the configured list (if you want to receive emails with nightly builds status let us know), IRC notifications (on #sipx channel)

Other User Actions:

The following actions are available in Hudson console (if you have proper rights):

- Build on demand: If you want to invoke a new build for a given project you can easily accomplish this (if you are in the group of authorized users) within the Hudson administration console. Click on the project name you want to build and proceed with "Build Now" link - a new entry will appear under Build History section.

- Console Output (available per specific build):

- Display Changes (per job and per specific build):

- Browse Workspace:

- Git polling log (available per specific build):