Kojid Builder

Koji Daemon - Builder

Kojid is the build daemon that runs on each of the build machines. Its primary responsibility is polling for incoming build requests and handling them accordingly. Koji also has support for tasks other than building. Creating install images is one example. kojid is responsible for handling these tasks as well. kojid uses mock for building. It also creates a fresh buildroot for every build. kojid is written in Python and communicates with koji-hub via XML-RPC.

Configuration Files:

* /etc/kojid/kojid.conf - Koji Daemon Configuration
* /etc/sysconfig/kojid - Koji Daemon Switches

Install kojid:

root@localhost$ yum install koji-builder

Required Configuration

Put this entry in your /etc/hosts file:

192.168.122.1 hongkong

/etc/kojid/kojid.conf: This needs to point at your koji-hub.

; The URL for the xmlrpc server
server=http://hongkong/kojihub

; the username has to be the same as what you used with add-host
; in this example follow as below
user = arm-001-001 ;Matches your VM name

; The URL for the packages tree
pkgurl=http://hongkong/yum/arm_built/base/12/arm/

This item may be changed, but may not be the same as KojiDir on the kojihub.conf file (although it can be something under KojiDir, just not the same as KojiDir)

; The directory root for temporary storage
workdir=/tmp/koji

SSL Certificates For Authentication

/etc/kojid/kojid.conf: If using SSL, these settings need to be valid.

;client certificate
; This should reference the builder certificate we created above, for
; kojibuilder1.example.com
cert = /etc/kojid/arm-001-008.pem

;certificate of the CA that issued the client certificate
ca = /etc/kojid/koji_ca_cert.crt

;certificate of the CA that issued the HTTP server certificate
serverca = /etc/kojid/koji_ca_cert.crt

Start Kojid

Before starting Kojid, make sure that the entry for your builder is in the database.

After, start Kojid:

root@localhost$ /sbin/service kojid start

Check /var/log/kojid.log to verify that kojid has started successfully.

Building RPMs Part 4

Building with Koji

We use Koji to build a package on different architectures such as i386, x86_64, ppc32, and ppc64. This will allow us to check if the rpm package we created will work on different machines with different architectures. When building for the 'Fedora Project' they will provide you with a certificate so you could use their servers. Using Koji can also be a good way to monitor errors and to get stats of how long it takes to build.

I'm not going to lie, I haven't done much with Koji. I've only used it as a client and haven't created my own Koji farm yet. But I'll explain how to get a certificate from Fedora and how to submit your package for building.

First thing is to create an account with the Fedora Project. click here to start creating your account. Once your account is done click here to create your certificate.

Now save it to your build user's home directory. Let's install Koji as root before we do anything else.

yum install fedora-packager

Now run this as your build user.

fedora-packager-setup

Now when I downloaded the certificate it got a file called dogencert which I changed to .fedora.cert

mv dogencert ~/.fedora.cert

Now we can start to build.

koji build dist-f12 --scratch 'rpmbuild/SRPMS/packagename.src.rpm'

And now we just play the waiting game to see if all is well.

If I missed anything please comment below and I will gladly add it to my 4 piece HowTo.

Building RPMs Part 3

Now for part 3 of 'Building RPMs' we will be looking at how to determine what dependencies the package needs. In most cases, you would have installed software on your computer that may already have downloaded the libraries required to install your present application. For this reason it is best to use some sort of bare bone environment to install the software under that will than prompt you what libraries need to be installed. To setup a PC with a clean installation can be very time consuming. So instead we use a program called mock that chroots an environment that we could use to test what dependencies are required.

Lets install and configure mock. The best thing to do is create a user to do the mock builds because you don't want people to have access to your files while they test with mock. Add this also gives the user permissions to use mock using group permissions

adduser -m -G mock build

if your the only user on this system than you can just add your username to the mock group

usermod -a -G mock username

Once that's all done, it's time to test with mock. Let's assume you are building on a 64bit architecture and you're running Fedora 12 on that machine. If so this would be the command you would run:

#mock rebuild -r fedora-12-x86_64 --rebuild rpmbuild/SRPM/packagename*.src.rpm
#cat /var/lib/mock/fedora-12-x86_64/result/build.log

First command builds the source rpm package and the second will allow you to see the output of the build so you can determine what dependencies are needed.

When you find the required libraries than you have to find out the names for them to install them with 'yum' which than you would include them to the line in the spec file that says 'BuildRequires: " and add the library package name there. Now run mock again until you get no errors. Let's move to 'Koji'.

TO BE CONTINUED... (Build testing with Koji)

Building RPMs Part 2

So, lets continue with the rpm creation process. (part1)

Part 1 covers how to download and check the files involved with most packages.

NOW the fun begins!!!

I'm going to explain how I created a package rpm for Irssi.


First I downloaded the tarball from http://www.irssi.org/files/irssi-0.8.14.tar.gz
which I than put it in the rpmbuild/SOURCE/ and created a new blank spec file using

rpmbuild-newspec irssi

First thing we need to do is fill in the fields on the top like Name, Version, etc... they are pretty straight forward and require very little google'ing to find the answers.

We need to understand how the file would install using just the source. Would you use a make and make install? Or ./configure, make, make install? This will affect the spec file depending on the procedure required to make the rpm package.

Here is my spec file:

Name: irssi
Version: 0.8.14
Release: 1%{?dist}
Summary: Command Line IRC Client
Group: Applications/Communications
License: GPLv2+
URL: http://www.irssi.org/
Source0: http://www.irssi.org/files/irssi-0.8.14.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildRequires: glib2-devel
BuildRequires: ncurses-devel
%description
Irssi is a terminal based IRC client for UNIX systems.

%prep
%setup -q

%build
%configure
make %{?_smp_mflags}

%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT
rm -rf $RPM_BUILD_ROOT/%{_docdir}/%{name}

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root,-)
%doc docs/*.txt docs/*.html AUTHORS COPYING NEWS README TODO
%config(noreplace) %{_sysconfdir}/%{name}.conf
%{_bindir}/%{name}
%{_datadir}/%{name}
%{_includedir}/%{name}/
%{_mandir}/man1/%{name}.1*

%changelog
* Wed Feb 3 2010 - Dave - 0.8.14-1
- Initial build..

I'm not going to explain everything in a spec file because I'm still learning. Hopefully from looking at my spec file you can determine what everything does. My Irssi spec file is really basic so it's good to learn from.

Now we try testing the spec file by using

rpmbuild -ba ~/rpmbuild/SPECS/irssi.spec

Issuing this command will try to build using the spec file above. Which had no problems and the exit code was 0. And to even test the spec file further we can try using rpmlint.

rpmlint ~/rpmbuild/SPECS/irssi.spec

If you have no problems than we move on to 'mock' to test what the package requires.


TO BE CONTINUED...(Using Mock to test RPM requirements)

Building RPMs

Creating a RPM in Fedora 12.

RPMs are the binary packages that Red Hat, Fedora, Suse and other distros use to install software. In most cases you can find a rpm packages somewhere on the web but for those rare occasions where you can only find the tar.gz source code than you can simply compile it using ./configure, make, make install. However, that isn't always the best way! It makes it harder to keep a rpm package list of all software installed if some apps are installed using 'make install' or what if you had a bunch of server racks that need to be running the same software and need the same updates? Wouldn't it be easier to use a rpm repository than to manually install on each computer?

So in a rpm package there are two files that are essential.
1) The actual software that needs installed or data that needs to be placed on the machine.
2) The SPEC file which holds the meta data of where everything goes.

I'm learning the in and outs of how to create/maintain rpm packages but I'll share with the web what I've learned so far.

So lets begin with the procedure required to creating a rpm package.

Needed packages

• rpm-build
• rpmdevtools
• rpmlint

yum install rpm-build rpmdevtools rpmlint

After the packages are downloaded and installed we can start creating/editing rpm packages.
First we create a directory to do all the work in.

rpmdev-setuptree

In that directory there are subdirectories such as SPECS, SOURCE and BUILD.
If your interested in editing a rpm than the easiest way would be to download a src.rpm.

yumdownloader --source packagename

This downloads a *.src.rpm which you can disassemble the rpm and see how they do it.

rpm -i packagename*.src.rpm

This installs the files in the packagename.src.rpm into ~/rpmbuild which you can find the tarball and the spec file for most packages.

TO BE CONTINUED...

Compile software with make -j

In my previous blog post I explained how to install software on GNU/Linux using Fedora's yum installer and how to compile software.

My professor was telling us how when we compile software it will compile each line of the make file one at a time. This can be a bit of a hassle when installing software like Apache or other large applications. So I did a small benchmark testing to see how much of a difference it makes when you tell make that your cpu can handle more than one job at a time. This was tested on a Q6600 which means its a quad-core.

Compiling irssi-0.8.14

Make(without -j):

$ time make
$ real 0m11.364s
$ user 0m6.175s
$ sys 0m4.332s

Eleven seconds to compile the irc client. Not bad but let us see how it compares to make with the multi-job flag.

Make -j 5

$ time make -j5
$ real 0m3.819s
$ user 0m5.882s
$ sys 0m4.990s

Wow, took four seconds to compile. That makes a big difference.

When you have the horsepower, you may as well use it because it will save you time.

Install from Source

Welcome! This is my first useful post of the year.

In most cases people who use Linux use some sort of application manager to install software, such as yum, yast, apt-get, etc... These application managers make installing software reduculously easily. The only issue is that you can only install the software version that is available in the repositories provided by the your Linux distro or by user-defined repositories which normally means that it isn't the latest stable version of that software. In this case we would either see if the software is packaged in some sort of binary package (rpm or deb in most cases). This just makes installing software and managing software a bit easier.

If your familiar with using Linux before these application managers came around you would remember that everything was manually compiled and installed which means that that software should work optimized for your computer. However, it also makes upgrading and managing software a bit more of a hassle.

Now let me show you how you would install software using yum on fedora 12 and how to install using source code.


Irssi - IRC CLI Client

Irssi is an irc client which has a very small memory footprint, allows users to create scripts to create additional features and functions.
This is how you would install it using yum:

yum install irssi

That is it! Now you can run the app by typing irssi in the terminal. howto use irssi

Now lets do it from source. irssi 0.8.14 is the latest version when I wrote this.

wget http://www.irssi.org/files/irssi-0.8.14.tar.gz
tar xzvf irssi-*.tar.gz
cd irssi-*
./configure
make
make install

*Make sure your root when you use make install.

You shouldn't run into any problems since the dependencies normally are pre-installed on most distros.

Wasn't hard to do either of the installs but with yum its a bit easier to remove or to upgrade irssi in the future.

Sometimes you get software that you can only install using the source since it isn't packaged or available in any of the public repositories. If that happens to be the case then you should follow the instructions that the application provides on how to install. An example would be NLED.
installing NLED is super easy and the website provides step by step instructions of the install procedure.