Jenkins

Jenkins
Jenkins - http://jenkins-ci.org/

Jenkins monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs:
 * Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.
 * Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Jenkins keeps those outputs and makes it easy for you to notice when something is wrong.

Installation
Get latest: wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war

Easy Start
Easy installation: Just java -jar jenkins.war, or deploy it in a servlet container. No additional install, no database. java -jar jenkins.war

Tomcat
Install tomcat6 to /opt/jenkins.

Jenkins WAR
Installation to ROOT: cd /opt/jenkins/webapps wget http://mirrors.jenkins-ci.org/war-stable/latest/jenkins.war -O ROOT.war
 * 1) latest
 * 2) wget http://mirrors.jenkins-ci.org/war/latest/jenkins.war -O ROOT.war
 * 3) stable lts

Restart tomcat and the application should deploy to ROOT.

Startup
The easiest way to execute Jenkins is through the built in Winstone servlet container. You can execute Jenkins like this:

$ java -jar jenkins.war

Of course, you probably want to send the output of Jenkins to a log file, and if you're on Unix, you probably want to use nohup:

$ nohup java -jar jenkins.war > $LOGFILE 2>&1

Command Line Parameter:


 * --httpPort=$HTTP_PORT - Runs Jenkins listener on port $HTTP_PORT using standard http protocol. The default is port 8080. To disable (because you're using https), use port -1.

Source: Starting and Accessing Jenkins - Jenkins - Jenkins Wiki - https://wiki.jenkins-ci.org/display/JENKINS/Starting+and+Accessing+Jenkins

Configuration
Jenkins configuration files are stored in ~/.jenkins rm -rf ~/.jenkins

Jenkins configuration manager: http://[SERVER:PORT]/configure

JENKINS_HOME directory
JENKINS_HOME directory - Administering Jenkins - https://wiki.jenkins-ci.org/display/JENKINS/Administering+Jenkins JENKINS_HOME +- config.xml    (jenkins root configuration) +- *.xml         (other site-wide configuration files) +- userContent   (files in this directory will be served under your http://server/userContent/) +- fingerprints  (stores fingerprint records) +- plugins       (stores plugins) +- jobs +- [JOBNAME]     (sub directory for each job) +- config.xml    (job configuration file) +- workspace     (working directory for the version control system) +- latest        (symbolic link to the last successful build) +- builds +- [BUILD_ID]    (for each build) +- build.xml     (build result summary) +- log           (log file) +- changelog.xml (change log

LDAP
Jenkins configuration manager: http://[SERVER:PORT]/configure


 * Check "Enable security" - note: looks like this has moved to "Configure Global Security"
 * Access Control - Security Realm - LDAP
 * Server: ldap.oeey.com
 * root DN: dc=oeey,dc=com
 * User search filter: uid={0}
 * Authorization
 * Anyone can do anything

Verify you can login, then change the Authorization:


 * Check "Enable security"
 * Authorization
 * Logged-in users can do anything

Or really locked down:


 * Check "Enable security"
 * Authorization
 * Matrix-based security # really locked down
 * (OR) Project-based Matrix Authorization Strategy # really locked down

NOTE: Add user/group 'authenticated' to select any authenticated user for permissions.

Plugins
Enable:
 * "SSH Slaves plugin" - https://wiki.jenkins-ci.org/display/JENKINS/SSH+Slaves+plugin (now enabled by default)

Add:
 * "Build-timeout Plugin" - This plugin allows you to automatically abort a build if it's taking too long. - https://wiki.jenkins-ci.org/display/JENKINS/Build-timeout+Plugin
 * "Discard Old Build plugin" - provides a post-build step where you can discard old build results in detailed configuration - https://wiki.jenkins-ci.org/display/JENKINS/Discard+Old+Build+plugin
 * there is a built in option that doesn't have quite as many options as this one
 * "Green Balls" - Changes Hudson to use green balls instead of blue for successful builds - https://wiki.jenkins-ci.org/display/JENKINS/Green+Balls
 * "Mercurial Plugin" - This plugin integrates the Mercurial version control system with Jenkins - https://wiki.jenkins-ci.org/display/JENKINS/Mercurial+Plugin
 * "Nested View Plugin" - View type to allow grouping job views into multiple levels instead of one big list of tabs. - https://wiki.jenkins-ci.org/display/JENKINS/Nested+View+Plugin
 * "Python Plugin" - Adds the ability to execute python scripts as build steps. - https://wiki.jenkins-ci.org/display/JENKINS/Python+Plugin
 * "Sidebar-Link Plugin" - Add links in the sidebar of the Jenkins main page, view tabs and project pages." - https://wiki.jenkins-ci.org/display/JENKINS/Sidebar-Link+Plugin
 * "Timestamper" - Adds timestamps to the Console Output. - https://wiki.jenkins-ci.org/display/JENKINS/Timestamper

Suggested Add:
 * "Email-ext plugin" - This plugin allows you to configure every aspect of email notifications. You can customize when an email is sent, who should receive it, and what the email says. - https://wiki.jenkins-ci.org/display/JENKINS/Email-ext+plugin
 * "JobConfigHistory Plugin" - Saves copies of all job and system configurations (for comparison from past builds) - https://wiki.jenkins-ci.org/display/JENKINS/JobConfigHistory+Plugin
 * "Throttle Concurrent Builds Plugin" - This plugin allows for throttling the number of concurrent builds of a project running per node or globally - https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin
 * "Text-finder Plugin" - This plugin lets you search keywords in the files you specified and use that to downgrade a "successful" build to be unstable or a failure.

Disable:
 * Ant Plugin
 * CVS Plug-in
 * Javadoc Plugin
 * Maven Integration plugin
 * Subversion Plug-in (UNLESS NEEDED) - http://wiki.jenkins-ci.org/display/JENKINS/Subversion+Plugin
 * Translation Assistance plugin

Don't disable: (or cause failure)
 * JUnit Plugin

Other useful plugins:
 * JobConfigHistory Plugin - Job history plugin for hudson.
 * Parameterized Trigger Plugin
 * JIRA Plugin - This plugin integrates Jenkins to Atlassian JIRA.
 * Config AutoRefresh Plugin - The Config AutoRefresh Plugin provides a way to configure the auto-refresh rate from the Jenkins UI.
 * Build Pipeline Plugin - Add a new view plugin to hudson
 * NodeLabel Parameter Plugin
 * Email-ext plugin - This plugin is a replacement for Jenkins's email publisher
 * OfflineOnFailure - This plugin allows you to take nodes offline immediately when a job reports FAILURE.
 * Jenkins Mercurial plugin - This plugin integrates Mercurial SCM with Hudson. It includes repository browsing support for hg serve/hgweb, Google Code, and Bitbucket. Features include guaranteed clean builds, named branch support, Forest extension support, module lists, Mercurial tool installation, and automatic caching.
 * thinBackup - Backups the most important global and job specific configuration files.
 * Backup plugin - Backup or restore your Hudson configuration files
 * Show Build Parameters plugin - This plugin shows the parameter values on the main build page
 * Jenkins Throttle Concurrent Builds Plug-in - This plugin allows for throttling the number of concurrent builds of a project running per node or globally.

Descriptions:
 * Build-timeout Plugin - This plugin allows builds to be automatically terminated after the specified amount of time has elapsed.
 * Python Plugin - Adds the ability to execute python scripts as build steps.
 * Nested View Plugin - View type to allow grouping job views into multiple levels instead of one big list of tabs.

Restart Jenkins
Jenkins can be restarted, without having to restart tomcat from the Plugin UpdateCenter:

https://jenkins-server/updateCenter/

Or even easier: https://jenkins-server/restart

Build Triggers - Build Periodically
This field follows the syntax of cron (with minor differences). Specifically, each line consists of 5 fields separated by TAB or whitespace: MINUTE HOUR DOM MONTH DOW MINUTE	Minutes within the hour (0-59) HOUR	The hour of the day (0-23) DOM	The day of the month (1-31) MONTH	The month (1-12) DOW	The day of the week (0-7) where 0 and 7 are Sunday. To specify multiple values for one field, the following operators are available. In the order of precedence,

'*' can be used to specify all valid values. 'M-N' can be used to specify a range, such as "1-5" 'M-N/X' or '*/X' can be used to specify skips of X's value through the range, such as "*/15" in the MINUTE field for "0,15,30,45" and "1-6/2" for "1,3,5" 'A,B,...,Z' can be used to specify multiple values, such as "0,30" or "1,3,5" Empty lines and lines that start with '#' will be ignored as comments.

In addition, '@yearly', '@annually', '@monthly', '@weekly', '@daily', '@midnight', and '@hourly' are supported.

Examples 5 * * * *
 * 1) every minute
 * 2) every 5 mins past the hour
 * 1) every 5 mins past the hour

Trigger Build by URL
Builds in Jenkins can be triggered periodically (on a schedule, specified in configuration), or when source changes in the project have been detected, or they can be automatically triggered by requesting the URL:

http://YOURHOST/jenkins/job/PROJECTNAME/build

Source: https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project#Buildingasoftwareproject-Configuringautomaticbuilds

Builds by e-mail (sendmail)
If you have the root account of your system and you are using sendmail, I found it the easiest to tweak /etc/aliases and add the following entry:

jenkins-foo: "|/bin/wget -o /dev/null http://YOURHOST/jenkins/job/PROJECTNAME/build"

and then run "newaliases" command to let sendmail know of the change. Whenever someone sends an e-mail to "jenkins-foo@yoursystem", this will trigger a new build. See this for more details about configuring sendmail.

Source: https://wiki.jenkins-ci.org/display/JENKINS/Building+a+software+project#Buildingasoftwareproject-Buildsbyemail%28sendmail%29

User Content
Location: /root/.jenkins/userContent

readme.txt: Files in this directory will be served under your http://server/hudson/userContent/

Fake Fail Node
fake-fail
 * 1) !/bin/bash

for i in `seq 1 10` ; do echo "FAKE MANAGER - FORCE FAILURE" done exit 1

Shell: (medium) [esx-fail] $ /bin/sh -xe /jenkins/tmp/hudson4954560349301610012.sh
 * Just change /bin/sh to point to the fake-fail script (this can startup failures however)
 * Maybe the best option, sh wrapper:

sh wrapper:
 * 1) !/bin/bash

if [ "$USER" = "fio" ] ; then /home/fio/bin/fake-fail else /bin/bash "$@" fi
 * 1) echo "Executing /bin/sh Wrapper..."

Python: (easy!) [esx-fail] $ python /jenkins/tmp/hudson8514675783666558740.py
 * Just change the path to have 'python' link to the fake-fail script.

Change Shell
Go to:
 * Configure System > Shell > Shell executable

Does not appear to be a way to change this per node.

If you start your job with #!/bin/bash Jenkins will use that instead of the default (plus -xe).

References:
 * Jenkins users - Change shell interpreter and options - http://jenkins.361315.n4.nabble.com/Change-shell-interpreter-and-options-td3877698.html

init script

 * 1) !/bin/sh


 * 1) chkconfig: 2345 90 10
 * 2) description: Jenkins Server

. /etc/init.d/functions
 * 1) Source function library.

NAME=jenkins PIDFILE=/var/run/$NAME.pid LOGFILE=/opt/jenkins/jenkins.log COMMAND=/opt/java/bin/java COMMAND_ARGS="-jar /opt/jenkins/jenkins.war --httpPort=-1 --httpsPort=443 --httpsKeyStore=/opt/jenkins/keystore --httpsKeyStorePassword=oeey"

export JENKINS_HOME=/data/jenkins

start { if [ -e $PIDFILE ] ; then echo -n "Already running!" failure return 1 fi   nohup $COMMAND $COMMAND_ARGS > $LOGFILE 2>&1 & COMMAND_PID=$! kill -0 $COMMAND_PID if [ $? -eq 0 ] ; then echo $COMMAND_PID > $PIDFILE success else failure fi }

stop { if [ -e $PIDFILE ] ; then kill $( cat $PIDFILE ) if [ $? -ne 0 ] ; then echo -n "Unable to kill!" failure return 1 fi       rm $PIDFILE success else echo -n "Not running!" failure fi }

case $1 in   start)        echo -n "Starting $NAME: "        start        echo ""    ;;    stop) echo -n "Stopping $NAME: " stop echo "" ;;   restart)        echo -n "Restarting $NAME: "        stop        sleep 1        start        echo ""    ;;    *) echo "usage: $NAME {start|stop|restart}" exit 1 ;; esac

exit 0

Modified from Source: https://wiki.jenkins-ci.org/display/JENKINS/Starting+and+Accessing+Jenkins

Reverse Proxy
Test: curl -iL http://your.reverse.proxy/jenkins/administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/test

Apache Reverse Proxy: NameVirtualHost *:80  ServerName jenkins.picmonic.net

ProxyPass        /  http://localhost:8080/ nocanon ProxyPassReverse /  http://localhost:8080/ ProxyRequests    Off AllowEncodedSlashes NoDecode

# Local reverse proxy authorization override # Most unix distribution deny proxy by default (ie /etc/apache2/mods-enabled/proxy.conf in Ubuntu)  Order deny,allow Allow from all 

ServerAdmin admin@picmonic.net ErrorLog logs/picmonic.net-error_log CustomLog logs/picmonic.net-access_log common 

References:
 * Jenkins says my reverse proxy setup is broken - Jenkins - Jenkins Wiki - https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+says+my+reverse+proxy+setup+is+broken
 * Running Jenkins behind Apache - Jenkins - Jenkins Wiki - https://wiki.jenkins-ci.org/display/JENKINS/Running+Jenkins+behind+Apache

Proxy Pass
Similar to forwarding, but takes care of redirects:

ProxyPass        /mirror/foo/ http://foo.com/ ProxyPassReverse /mirror/foo/ http://foo.com/

Suppose the local server has address http://wibble.org/; then

ProxyPass        /mirror/foo/ http://foo.com/ ProxyPassReverse /mirror/foo/ http://foo.com/

will not only cause a local request for the  to be internally converted into a proxy request to  (the functionality ProxyPass provides here). It also takes care of redirects the server foo.com sends: when http://foo.com/bar is redirected by him to http://foo.com/quux Apache adjusts this to http://wibble.org/mirror/foo/quux before forwarding the HTTP redirect response to the client.

 ServerName wiki.oeey.com

#Forwarding Traffic ProxyPass        / http://localhost:8080/ ProxyPassReverse / http://localhost:8080/

ServerAdmin website@oeey.com ErrorLog logs/wiki-error_log CustomLog logs/wiki-access_log common 

Proxy Pass to Jenkins: NameVirtualHost *:80  ServerName jenkins.oeey.com

#Forwarding Traffic #RewriteEngine on   #RewriteRule ^(/.*)  http://localhost:8080/$1   [P]

ProxyPass        /  http://localhost:8080/ nocanon ProxyPassReverse /  http://localhost:8080/ ProxyRequests    Off ProxyPreserveHost On   AllowEncodedSlashes NoDecode

# Local reverse proxy authorization override # Most unix distribution deny proxy by default (ie /etc/apache2/mods-enabled/proxy.conf in Ubuntu)  Order deny,allow Allow from all 

ServerAdmin admin@oeey.com ErrorLog logs/jenkins.oeey.com-error_log CustomLog logs/jenkins.oeey.com-access_log common 

Proxy Pass HTTPS
Proxy Pass to Jenkins: NameVirtualHost *:443  ServerName jenkins.oeey.com

# Enable SSL Support SSLEngine on   SSLCertificateFile /etc/pki/tls/certs/localhost.crt SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

#Forwarding Traffic #RewriteEngine on   #RewriteRule ^(/.*)  http://localhost:8080/$1   [P]

ProxyPass        /  http://localhost:8080/ nocanon ProxyPassReverse /  http://localhost:8080/ ProxyPassReverse /  http://jenkins.oeey.com/ ProxyRequests    Off ProxyPreserveHost On   AllowEncodedSlashes NoDecode RequestHeader set X-Forwarded-Proto "http"

# Local reverse proxy authorization override # Most unix distribution deny proxy by default (ie /etc/apache2/mods-enabled/proxy.conf in Ubuntu)  Order deny,allow Allow from all 

ServerAdmin admin@oeey.com ErrorLog logs/jenkins.oeey.com-error_log CustomLog logs/jenkins.oeey.com-access_log common 

7 Ways to Optimize Jenkins/Hudson White Paper

 * 1) Back Up and Restore - Just Do It!
 * 2) Plan for Disk Usage Growth Up Front - Prepare for Disk Usage Growth
 * 3) Use Native Packages - Use OS-specific installation packages over the default war-based installation
 * 4) Take Advantage of Distributed Builds - Do Distributed Builds with Nodes
 * 5) Use Labels - Labels are simply tags you can assign to nodes to describe their capacities
 * 6) Use a Memorable URL for Jenkins - Invest in an Easy-to-Remember URL
 * 7) Prevent Build Record Sprawl - Keep only a subset of build records around with "Discard Old Builds"

Source: 7 Ways to Optimize Jenkins/Hudson White Paper http://di388e0fcqllf.cloudfront.net/whitepapers/7WaysToOptimizeJenkins.pdf

Seven Habits of Highly Effective Jenkins Users

 * 1) HABIT 1: MAKE YOUR MASTER STABLE AND RESTORABLE
 * 2) * Use LTS Releases
 * 3) * Be conservative about upgrading plugins
 * 4) * Have an upgrade testbed
 * 5) * Back up your Jenkins configuration
 * 6) ** Example of configuration backup script: https://gist.github.com/abayer/527063a4519f205efc74
 * 7) * Avoid using the Maven job type
 * 8) HABIT 2: BREAK UP THE BLOAT
 * 9) * Multiple Masters - split up masters by team, function, access, etc.
 * 10) * Break up your jobs - modularization and reuse are good techniques
 * 11) * Tools for breaking up your jobs: parameterized trigger + conditional build step, copy artifact, promoted builds, build pipeline plugin, workflow plugin
 * 12) HABIT 3: AUTOMATE JENKINS TASKS!
 * 13) * Script console and Scriptler
 * 14) * Generate jobs programmatically (Jenkins REST API and CLI)
 * 15) HABIT 4: TEND YOUR PLUGIN GARDEN
 * 16) * Do you really need that plugin?
 * 17) * Disable plugins you don't use
 * 18) * Plugins can cause instability
 * 19) * Clean up old plugins and their data
 * 20) * My Essential plugins:
 * 21) ** Job Config History
 * 22) ** Static analysis plugins
 * 23) ** xUnit
 * 24) ** Parameterized Trigger and Conditional Build Step
 * 25) ** Tool Environment
 * 26) ** EnvInject
 * 27) ** Rebuild
 * 28) ** Build Timeout
 * 29) HABIT 5: INTEGRATE WITH OTHER TOOLS AND SERVICES
 * 30) * REST API
 * 31) * Source Control
 * 32) * Jira
 * 33) * Artifactory
 * 34) HABIT 6: MAKE YOUR SLAVES FUNGIBLE
 * 35) * "Fungibility is the property of a good or a commodity whose individual units are capable of mutual substitution"
 * 36) * A fungible slave is a slave you can replace easily with another slave
 * 37) * Make slaves identical (replication, puppet/chef/ansible, etc)
 * 38) * Make slaves reusable and flexible (general purpose)
 * 39) * Use the cloud
 * 40) HABIT 7: JOIN THE COMMUNITY
 * 41) * Write plugins, open JIRAs, fix bugs, get on mailing lists, help others

Source: Seven Habits of Highly Effective Jenkins Users (2014 edition!) - http://www.slideshare.net/andrewbayer/seven-habits-of-highly-effective-jenkins-users-2014-edition

Startup Options
[root@jenkins jenkins]# java -jar /opt/jenkins/jenkins.war --help Running from: /opt/jenkins/jenkins.war webroot: $user.home/.jenkins Jenkins Continuous Integration Engine 1.452 Usage: java -jar jenkins.war [--option=value] [--option=value]

Options: --daemon                = fork into background and run as daemon (Unix only) --config                = load configuration properties from here. Default is ./winstone.properties --prefix                = add this prefix to all URLs (eg http://localhost:8080/prefix/resource). Default is none --commonLibFolder       = folder for additional jar files. Default is ./lib

--logfile               = redirect log messages to this file --logThrowingLineNo     = show the line no that logged the message (slow). Default is false --logThrowingThread     = show the thread that logged the message. Default is false --debug                 = set the level of debug msgs (1-9). Default is 5 (INFO level)

--httpPort              = set the http listening port. -1 to disable, Default is 8080 --httpListenAddress     = set the http listening address. Default is all interfaces --httpDoHostnameLookups = enable host name lookups on incoming http connections (true/false). Default is false --httpsPort             = set the https listening port. -1 to disable, Default is disabled if neither --httpsCertificate nor --httpsKeyStore are specified, https is run with one-time self-signed certificate. --httpsListenAddress    = set the https listening address. Default is all interfaces --httpsDoHostnameLookups = enable host name lookups on incoming https connections (true/false). Default is false --httpsKeyStore         = the location of the SSL KeyStore file. --httpsKeyStorePassword = the password for the SSL KeyStore file. Default is null --httpsCertificate      = the location of the PEM-encoded SSL certificate file. (the one that starts with '-BEGIN CERTIFICATE-') must be used with --httpsPrivateKey. --httpsPrivateKey       = the location of the PEM-encoded SSL private key. (the one that starts with '-BEGIN RSA PRIVATE KEY-') --httpsKeyManagerType   = the SSL KeyManagerFactory type (eg SunX509, IbmX509). Default is SunX509 --ajp13Port             = set the ajp13 listening port. -1 to disable, Default is 8009 --ajp13ListenAddress    = set the ajp13 listening address. Default is all interfaces --controlPort           = set the shutdown/control port. -1 to disable, Default disabled

--handlerCountStartup   = set the no of worker threads to spawn at startup. Default is 5 --handlerCountMax       = set the max no of worker threads to allow. Default is 300 --handlerCountMaxIdle   = set the max no of idle worker threads to allow. Default is 50

--simulateModUniqueId   = simulate the apache mod_unique_id function. Default is false --useSavedSessions      = enables session persistence (true/false). Default is false --mimeTypes=ARG         = define additional MIME type mappings. ARG would be EXT=MIMETYPE:EXT=MIMETYPE:... (e.g., xls=application/vnd.ms-excel:wmf=application/x-msmetafile) --maxParamCount=N       = set the max number of parameters allowed in a form submission to protect against hash DoS attack (oCERT #2011-003). Default is 10000. --usage / --help        = show this message --version               = show the version and quit

Security options: --realmClassName              = Set the realm class to use for user authentication. Defaults to ArgumentsRealm class

--argumentsRealm.passwd. = Password for user. Only valid for the ArgumentsRealm realm class --argumentsRealm.roles. = Roles for user (comma separated). Only valid for the ArgumentsRealm realm class

--fileRealm.configFile        = File containing users/passwds/roles. Only valid for the FileRealm realm class

Access logging: --accessLoggerClassName       = Set the access logger class to use for user authentication. Defaults to disabled --simpleAccessLogger.format   = The log format to use. Supports combined/common/resin/custom (SimpleAccessLogger only) --simpleAccessLogger.file     = The location pattern for the log file(SimpleAccessLogger only)

Jenkins CLI
Jenkins CLI - Jenkins - Jenkins Wiki - https://wiki.jenkins.io/display/JENKINS/Jenkins+CLI

Download: https://jenkins.example.com/jnlpJars/jenkins-cli.jar

Sample (Get Node Details): java -jar jenkins-cli.jar -http -auth myuser:mypassword -s https://jenkins.oeey.com:8443 get-node my-node-01

Create Node
https://jenkins.oeey.com/cli/command/create-node

Python
See Python/Jenkins

"Python Plugin" - Adds the ability to execute python scripts as build steps. - https://wiki.jenkins-ci.org/display/JENKINS/Python+Plugin

Add node environment variable (node properties): name=PYTHONUNBUFFERED value=1

Jenkins Python API
Python API - http://python-jenkins.readthedocs.io/en/latest/api.html

pip install python-jenkins

http://python-jenkins.readthedocs.io/en/latest/

""" Jenkins Python API Utilities.""" import argparse import datetime import glob import json import multiprocessing import os import pdb import re import subprocess import sys import tempfile import time import urllib3 from copy import deepcopy from operator import itemgetter

import jenkins from jira import JIRA

urllib3.disable_warnings

if __name__ == '__main__':
 * 1) Jenkins Python API Utilities.

# Get user parameters. parser = argparse.ArgumentParser(description="Jenkins") parser.add_argument('--jenkins-user', metavar='USER', type=str, default="jenkins",                       help='The Jenkins user ID.') parser.add_argument('--jenkins-password', metavar='PASSWORD', type=str, default="password",                       help='The Jenkins password.') parser.add_argument('--jenkins-url', metavar='URL', type=str, default="https://jenkins.oeey.com:8443",                       help='The Jenkins URL.') args = parser.parse_args

# Set up jenkins. jenkins_server = jenkins.Jenkins(args.jenkins_url, username=args.jenkins_user, password=args.jenkins_password)

# Delete server nodes. nodes = jenkins_server.get_nodes for node in nodes: #print node # delete nodes with "old-" in name if "old-" in node["name"]: print "Deleting node: {}".format(node["name"]) jenkins_server.delete_node(node["name"])

Hello
import jenkins

SERVER = "https://jenkins.oeey.com:8443/" USERNMAE = "username" PASSWORD = "password"

server = jenkins.Jenkins(SERVER, username=USERNMAE, password=PASSWORD) user = server.get_whoami version = server.get_version print('Hello %s from Jenkins %s' % (user['fullName'], version))

ref:

Node Info
import jenkins

SERVER = "https://jenkins.oeey.com:8443/" USERNMAE = "username" PASSWORD = "password"

server = jenkins.Jenkins(SERVER, username=USERNMAE, password=PASSWORD)

node_info = server.get_node_info('SOME-NODE-NAME') print node_info

Node Config
import jenkins

SERVER = "https://jenkins.oeey.com:8443/" USERNMAE = "username" PASSWORD = "password"

server = jenkins.Jenkins(SERVER, username=USERNMAE, password=PASSWORD)

node_config = server.get_node_config('SOME-NODE-NAME') print node_config

Disable Node
import jenkins

SERVER = "https://jenkins.oeey.com:8443/" USERNMAE = "username" PASSWORD = "password"

server = jenkins.Jenkins(SERVER, username=USERNMAE, password=PASSWORD)

server.disable_node('SOME-NODE-NAME')
 * 1) server.enable_node('SOME-NODE-NAME')

Delete Node
import jenkins

SERVER = "https://jenkins.oeey.com:8443/" USERNMAE = "username" PASSWORD = "password" NODE = 'SOME-NODE-NAME'

server = jenkins.Jenkins(SERVER, username=USERNMAE, password=PASSWORD) server.delete_node(NODE)

Create Node
import jenkins

SERVER = "https://jenkins.oeey.com:8443/" USERNMAE = "username" PASSWORD = "password" NODE = "SOME-NEW-NODE" NODE_CREDENTIALS = 'd03e3772-783c-4749-ff93-da268e0fa9ff'
 * 1) Note: get node credentials from another existing node
 * 2)       that uses same credentials, with get_node_config

server = jenkins.Jenkins(SERVER, username=USERNMAE, password=PASSWORD)

print "Creating Node: {}".format(NODE)

params = { 'host': NODE, 'port': '22', 'credentialsId': NODE_CREDENTIALS, 'sshHostKeyVerificationStrategy': { 'stapler-class': 'hudson.plugins.sshslaves.verifiers.ManuallyTrustedKeyVerificationStrategy', 'requireInitialManualTrust': False } } server.create_node(   name=NODE,    numExecutors=1,    nodeDescription='Some description',    remoteFS='/j',    labels='some labels',    exclusive=False,    launcher=jenkins.LAUNCHER_SSH,    launcher_params=params)

server.disable_node(NODE)
 * 1) Disabled new node

Update Node
import jenkins import sys import re

jserver = jenkins.Jenkins('https://jenkins.oeey.com', username='jenkins', password='PASSWORD')

node_name = 'MYNODE'

node_config = jserver.get_node_config(node_name)

match = re.findall('<numExecutors>.*</numExecutors>', node_config)[0]

new_node_config = node_config.replace(match, '<numExecutors>6</numExecutors>')

jserver.reconfig_node(node_name, new_node_config)

<?xml version="1.0" encoding="UTF-8"?> MYNODE NODE_DESCRIPTION <remoteFS>/j</remoteFS> <numExecutors>3</numExecutors> EXCLUSIVE <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/> <launcher class="hudson.plugins.sshslaves.SSHLauncher" plugin="ssh-slaves@1.21"> MYNODE.OEEY.COM 22    <credentialsId>XX8416c5-6aac-4834-9c43-f478869f7af2</credentialsId> <maxNumRetries>0</maxNumRetries> <retryWaitTime>0</retryWaitTime> <sshHostKeyVerificationStrategy class="hudson.plugins.sshslaves.verifiers.NonVerifyingKeyVerificationStrategy"/> MY_LABELS <nodeProperties/>

Groovy Script
Run from /script https://jenkins.server.com/script

Kill long running jobs
int MAX_ALLOWED_DURATION_IN_SECONDS = 60 * 60 * 6 // 24 hours

url = "http://jenkins.oeey.com" def busyExecutors = Jenkins.instance.computers .collect { c -> c.executors.findAll { it.isBusy } }                  .flatten // reminder: transforms list(list(executor)) into list(executor)

def ok = true println "Busy Executors list" busyExecutors.each { e -> int durationInSeconds = (System.currentTimeMillis - e.executable.getTimeInMillis)/1000.0 if(durationInSeconds > MAX_ALLOWED_DURATION_IN_SECONDS ) {   println e ; nodeName = e.getOwner.nodeName println "\t $url/computer/$nodeName" durationInHours = durationInSeconds / 60 / 60 println "\t duration (hours) = $durationInHours" println "" //e.doStop ok = false; } } println "Done" return ok

Doc:
 * http://javadoc.jenkins-ci.org/hudson/model/Computer.html
 * http://javadoc.jenkins-ci.org/hudson/model/Executor.html

Ref: Find builds currently running that has been executing for more than N seconds - Jenkins - Jenkins Wiki - https://wiki.jenkins.io/display/JENKINS/Find+builds+currently+running+that+has+been+executing+for+more+than+N+seconds

Shorten workspace name
A job typically runs at: C:\jenkins\workspace\[JOB]\[LABEL_TITLE]\[LABEL]

Windows has a 260 character path limit. To help with this, make the following changes:
 * 1) Remote root directory to "c:\j" instead of "c:\jenkins"  (node settings)
 * 2) Change "workspace" to "w"  (server setting that affects slaves -Dhudson.model.Slave.workspaceRoot=ws)
 * 3) Use shorter job names like "t1"  (job's "project name")
 * 4) Use shorter label expression title name, like "l"  (job's label expression name defaults to "label_exp")
 * 5) Use shorter label expression name, like "win"

The Jenkins jobs will now be running at these shortened paths: C:\j\w\[JOB]\l\[LABEL]

Example: C:\j\w\T1\l\WIN

Instead of something like: C:\jenkins\workspace\MYTEST\labels\WIN-NODES

-

Change "workspace" to "ws"

-Dhudson.model.Slave.workspaceRoot=ws or -Dhudson.model.Slave.workspaceRoot=w

Added to /usr/share/tomcat7/bin/catalina.sh under CATALINA_OPTS. This applies to all slaves.

-

Alternative:

Windows 10 has already added the feature for paths > 260 chars to the insider preview.

You can go to Advanced Options -> Windows Update and enable Insider builds. You then have to wait for Windows to download the latest insider build. I believe the path length feature was added to the official Windows Anniversary update. Maybe the lab team can add the anniversary update to provisioning?

Windows Web Start
Run IE with Elevated Permissions

Launch agent from browser

Java slave agent -> Fil -> Install as a service

Stop service and set "Log on as" -> "This account" -> administrator
 * Running some jobs as Local System account causes issuse

Delete Jenkins Service
sc stop jenkinsslave-c__j sc delete jenkinsslave-c__j

Don't Exit Shell on Failure
set +e git commit -m "Bla." set -e

-

set +e
 * 1) Disable exit on non 0


 * 1) Do something. If something fails with exit!=0 the script continues anyway

set -e
 * 1) Enable exit on non 0

Ref:
 * don't fail jenkins build if execute shell fails - Stack Overflow - https://stackoverflow.com/questions/14392349/dont-fail-jenkins-build-if-execute-shell-fails
 * How to Ignore Failures in a Shell Step? – CloudBees Support - https://support.cloudbees.com/hc/en-us/articles/218517228-How-to-Ignore-Failures-in-a-Shell-Step-

keywords: bash shell execute failure

UTF-8 decode URLs
Warning:
 * "Your container doesn’t use UTF-8 to decode URLs. If you use non-ASCII characters as a job name etc, this will cause problems. See Containers and Tomcat i18n for more details."

Solution:
 * Tomcat - Jenkins - Jenkins Wiki - i18n - https://wiki.jenkins-ci.org/display/JENKINS/Tomcat#Tomcat-i18n

Some versions of Tomcat (such as 5.0.28) uses iso-8859-1 to decode URLs, which is in a clear violation of the relevant RFCs. To fix this problem, add the following URIEncoding attribute to the connector definition in $TOMCAT_HOME/conf/server.xml.

<Connector port="8080" URIEncoding="UTF-8"/>

Unsecured Jenkins
Warning:
 * "Unsecured Jenkins allows anyone on the network to launch processes on your behalf. Consider at least enabling authentication to discourage misuse. "

Reverse Proxy is Broken
See

Jenkins says my reverse proxy setup is broken - Jenkins - Jenkins Wiki - https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+says+my+reverse+proxy+setup+is+broken

java.nio.charset.UnsupportedCharsetException: UTF-8-server
SEVERE: Servlet.service for servlet Stapler threw exception java.nio.charset.UnsupportedCharsetException: UTF-8-server

Had to remove this line from the Tomcat6 setenv.sh... -Dfile.encoding=UTF-8