Mer/Build/UsingGitorious

Revision as of 16:37, 12 July 2009

Packaging

Initial Setup

The initial Mer DVCS packaging process looks like this:

An upstream package is unpacked and the debian/ directory is removed. It is then commited onto the upstream branch.

pristine-tar is used to ensure that the upstream tarball can be recreated perfectly.

All features or bug fixes are broken out to additional branches; 1 branch per feature or fix. The main difference is that features are not likely to go upstream whilst fixes are. This is very similar to quilt.

master is the main code branch and there should only be merge commits on this branch; the only non-merge commits should be in ./ChangeLog

Mer should be packaging-based and the only non-merge commits should be in debian/

Local Updates

A local update simply adds a commit to one or more feature branches, merges to master, tags, merges to Mer, tags and releases.

Upstream Updates

An upstream update loads the new upstream into the upstream branch and replaces any code there.

Then it is merged into each feature branch in turn. This ensures the feature branch is updated ready for the next upstream release (which wouldn't happen if the feature branch was merged into upstream).

Since master contains all the features; any attempt to merge upstream into master will result in the same conflicts arising en-masse. Instead a local branch is created from upstream HEAD and all features/patches are staged into this branch before being merged and tagged in master.

Finally master is merged into Mer and any packaging modifications are made.

Maintenance Releases

[ Note this is still theoretical. ]

Maintenance releases are made by branching master and patching or cherry picking commits. There is no attempt to maintain or integrate feature branches.

Once a tag is made in master, the Mer branch is branched, merged and tagged.

Overall

Developing

Development of a feature is done by cloning master; then branching locally and developing against master.

Once development is complete a new feature branch is created and the development branch is then cherry-picked onto the new feature branch.

Moving a Package to Gitorious

Initial setup

The following scripts need:

 PKG=<package name>
 GPKG=<gitorious-safe package name> (ie [a-z0-9_-]
 UPVER=<upstream version>
 TARBALL=<tarball name>
 TAG=<new version>

Get the upstream source unpacked

 mkdir gitify
 cd gitify
 tar xf ../$TARBALL
 mv * ../$PKG
 cd ..
 rm -rf gitify
 cd $PKG

Cleanse the install

 mv debian ..

Create a git repo with the new stuff

 git init
 git add .
 git commit -am"${PKG}_$UPVER"

This is really the upstream branch

 git branch -m master upstream

For keeping an eye on things you may want gitk running. Use File->reload frequently (F5 doesn't work for me)

 gitk --all &

Save state

 pristine-tar commit ../$TARBALL

Prepare structure create master branch from upstream

 git checkout -b master

create Mer branch from master

 git checkout -b Mer

Apply debianisation

 mv ../debian .
 git add debian/
 git commit -am "initial debianize from upstream"

Feature/patch branches

Now apply features from the .diff.gz this has to be done manually

Identify a libtool feature

 git checkout upstream
 git checkout -b mer/libtool

apply hunks

 git commit -am"libtool fixes"

Identify an obs-fix feature

 git checkout upstream
 git checkout -b mer/obs-fix

apply hunks

 git commit -am"obs pthread fixes"

Now any debian/ hunks

 git checkout Mer

apply hunks

 git commit -am"Added Mer debianisation"

Now pull it all together

 git checkout master

look for all the mer/* and patch/* branches and merge them

 git branch -l
 git merge mer/libtool
 git merge mer/obs-fix

now use git log to create a top level ChangeLog entry

 git add ChangeLog
 git commit -m"$TAG"

Mark an 'upstream' release

 git tag $TAG

Now make a distro release

 git checkout Mer
 git merge master

now use git log to create a debian/changelog entry

 git add debian/changelog
 git commit -m"Mer_$TAG"
 git tag Mer_$TAG

Verify

Remove all obs code:

 cd ..
 rm $PKG*

regenerate it:

 cd $PKG
 pristine-tar checkout ../$TARBALL
 debuild -S -i.git

Build or examine to verify

Push

Now push to gitorious

 git symbolic-ref HEAD refs/heads/Mer

Logon to http://gitorious.org/mer and create gitorious project

 git remote add origin git@gitorious.org:mer/$GPKG.git
 git push --mirror origin

on gitorious goto 'Edit Repository' and set default to Mer

Updating a feature branch

python-hildon

Setup some variables

PKG=python-hildon
GPKG=$PKG
FEATURE=codegen-in-gobject
TAG=0.9.0-1maemo4mer2

First go and get the source from gitorious or update your copy

git clone git@gitorious.org:mer/$GPKG.git

git fetch

Get the branch you want to work on

git checkout mer/$FEATURE

make changes and commit

git add $FILES
git commit -m"Fix to build on debian"

Merge into Mer branch via local and master

git checkout upstream
git checkout  -b local

Merge all features and patches

git merge mer/$FEATURE

Get the master branch

git checkout --track origin/master

git checkout master

Then merge into Mer

git merge local

Make a comment in the toplevel Changelog

git add ChangeLog
git commit -m"$TAG"
git tag $TAG
git checkout Mer

Make a comment in the debian changelog

git add debian/changeLog
git commit -m"Mer_$TAG"
git tag Mer_$TAG

Then

git push

Handling new upstream

hildon-input-method

get new upstream

 wget http://repository.maemo.org/pool/fremantle/free/h/hildon-input-method/hildon-input-method_2.1.16-1+0m5.tar.gz

Make the tarball an "orig"

 mv hildon-input-method_2.1.16-1+0m5.tar.gz hildon-input-method_2.1.16-1+0m5.orig.tar.gz
 TARBALL=hildon-input-method_2.1.16-1+0m5.orig.tar.gz
 PKG=hildon-input-method
 GPKG=$PKG
 UPVER=2.1.16-1+0m5

Prepare to merge

 git clone git@gitorious.org:mer/$GPKG.git
 cd $GPKG
 git checkout --track origin/upstream
 git rm -r *

replace the upstream code

 mkdir _tmp
 cd _tmp
 tar xf ../../$TARBALL
 mv */* ../
 cd ..
 rm -rf _tmp
 git add .

Now we have a clean upstream release

 git commit -am"${PKG}_$UPVER"
 git tag $UPVER
 pristine-tar commit ../$TARBALL

Now to merge it into the branches for each mer/* branch

 git checkout --track origin/mer/buttonwidth
 git merge upstream

 git checkout --track origin/mer/him-arabic
 git merge upstream

This fails:

 Auto-merging hildon-input-method-ui3.schemas
 CONFLICT (content): Merge conflict in hildon-input-method-ui3.schemas
 Automatic merge failed; fix conflicts and then commit the result.

Looking in hildon-input-method-ui3.schemas we have:

 ...
     <owner>hildon-input-method</owner>
     <type>string</type>
<<<<<<< HEAD:hildon-input-method-ui3.schemas
     <default>him-arabic</default>
=======
     <default>hildon_keyboard_assistant</default>
>>>>>>> upstream:hildon-input-method-ui3.schemas
     <locale name="C">
       <short>Current input method</short>
...

fix this to:

 ...
     <owner>hildon-input-method</owner>
     <type>string</type>
     <default>him-arabic</default>
     <locale name="C">
       <short>Current input method</short>
...

Now a git status shows

 $ git status
 hildon-input-method-ui3.schemas: needs merge
 # On branch mer/him-arabic
 # Changes to be committed:
 #   (use "git reset HEAD <file>..." to unstage)
 #
 #       modified:   AUTHORS
 #       modified:   ChangeLog
 #       modified:   configure.ac
 #       new file:   debian/changelog
 #       new file:   debian/compat
 ...
 #       modified:   src/hildon-im-widget-loader.c
 #
 # Changed but not updated:
 #   (use "git add <file>..." to update what will be committed)
 #   (use "git checkout -- <file>..." to discard changes in working directory)
 #
 #       unmerged:   hildon-input-method-ui3.schemas
 #

So, as suggested:

 git add hildon-input-method-ui3.schemas

and then

 git commit

This opens an editor session with some normal merge text and the option to add more

 Merge branch 'upstream' into mer/him-arabic

 Conflicts:

hildon-input-method-ui3.schemas

 #
 # It looks like you may be committing a MERGE.
 # If this is not correct, please remove the file
 #	.git/MERGE_HEAD
 # and try again.
 #

 Merge branch 'upstream' into mer/him-arabic
 default keyboard made him-arabic

Carry on:

 git checkout --track origin/mer/keyboard-available
 git merge upstream

More conflicts:

 Auto-merging hildon-input-method-ui3.schemas
 Auto-merging src/hildon-im-ui.c
 CONFLICT (content): Merge conflict in src/hildon-im-ui.c
 Auto-merging src/hildon-im-ui.h
 CONFLICT (content): Merge conflict in src/hildon-im-ui.h
 Automatic merge failed; fix conflicts and then commit the result.

This time it is helpful to examine the diff in the last commit. Looking at remotes/origin/mer/keyboard-available "add feature for if a physical keyboard is available" in gitk shows the diff. We can see that

 +  self->priv->keyboard_available =
 +    gconf_client_get_bool(self->client, HILDON_IM_GCONF_KEYBOARD_AVAILABLE, NULL);

was added to hildon-im-ui.c and

 +#define HILDON_IM_GCONF_KEYBOARD_AVAILABLE         HILDON_IM_GCONF_DIR       "/keyboard_available"

to hildon-im-ui.h

The diff algorithm is confused because lines in the vicinity have changed. So examine these files, resolve the conflicts and add them to git.

 git add src/hildon-im-ui.h src/hildon-im-ui.c
 git commit

Finally:

 git checkout --track origin/mer/werror

Once again we see that the old branch simply wanted to remove -Werror so edit and

 git add configure.ac
 git commit

Now we have resolved each of the branches we can merge into master. Because there were conflicts we need to

 git checkout upstream
 git checkout  -b local
 git merge mer/buttonwidth
 git merge mer/him-arabic
 git merge mer/keyboard-available
 git merge mer/werror

Once the conflicting branches are consolidated, they can be merged:

 git checkout --track origin/master
 git merge local

And tagged for release:

 git add ChangeLog
 git tag 2.1.16-1+0m5-mer1

And Mer can be released too:

 git checkout Mer
 git merge master
 git add debian/changelog debian/control
 git commit
 git tag Mer_2.1.16-1+0m5-mer1

Now push

 git push --all
 git push --tags

When it's all done it looks like this:

glib

This shows by example how glib upstream was merged.

Clone the gitorious repository (I made a clone on gitorious first) at git@gitorious.org:~lbt/mer/glib2_0-mer.git

http://gitorious.org/~lbt/mer/glib2_0-mer

Get new upstream

 wget https://launchpad.net/ubuntu/jaunty/+source/glib2.0/2.20.1-0ubuntu2/+files/glib2.0_2.20.1.orig.tar.gz
 TARBALL=glib2.0_2.20.1.orig.tar.gz
 mkdir ../patches
 cd ../patches/
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/25-gatomic.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/30-gfileutils.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/35-gmessages.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/40-gscanner.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/45-gunicode.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/50-gthread.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/60_wait-longer-for-threads-to-die.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/70_use-monotonic-clock-for-timeouts.patch
 wget https://stage.maemo.org/svn/maemo/projects/haf/branches/glib/glib-2-20/debian/patches/series

Prepare to merge

 git clone git@gitorious.org:~lbt/mer/glib2_0-mer.git
 cd glib2_0-mer/
 git checkout --track origin/upstream
 git rm -r *

replace the upstream code

 mkdir _tmp
 cd _tmp
 tar xf ../../$TARBALL
 mv */* ../
 cd ..
 rm -rf _tmp
 git add .

Now we have a clean upstream release

 git commit -am"glib2.0_2.20.1"
 git tag 2.20.1
 pristine-tar commit ../glib2.0_2.20.1.orig.tar.gz

Apply all the maemo patches to their own branches

 git checkout upstream;git checkout -b mer/25-gatomic
 patch -p1 < ../patches/25-gatomic.patch
 git commit -am"applied gatomic"

 git checkout upstream;git checkout -b mer/30-gfileutils
 patch -p1 < ../patches/30-gfileutils.patch
 git commit -am"applied gfileutils"

 git checkout upstream;git checkout -b mer/35-gmessages
 patch -p1 < ../patches/35-gmessages.patch
 git commit -am"applied gmessages"

 git checkout upstream ;git checkout -b mer/40-gscanner
 patch -p1 < ../patches/40-gscanner.patch
 git commit -am"applied gscanner"

 git checkout upstream ;git checkout -b mer/45-gunicode
 patch -p1 < ../patches/45-gunicode.patch
 git commit -am"applied gunicode"

 git checkout upstream ;git checkout -b mer/50-gthread
 patch -p1 < ../patches/50-gthread.patch
 git commit -am"applied gthread"

 git checkout upstream ;git checkout -b mer/60-wait-longer-for-threads-to-die
 patch -p1 < ../patches/60_wait-longer-for-threads-to-die.patch
 git commit -am"applied wait-longer-for-threads-to-die"
 
 git checkout upstream ;git checkout -b mer/70_use-monotonic-clock-for-timeouts
 patch -p1 < ../patches/70_use-monotonic-clock-for-timeouts.patch
 git commit -am"applied use-monotonic-clock-for-timeouts"

Now to create a Mer glib from a master

 git checkout --track origin/master

Upgrade entire glib

 git merge upstream

Now we apply in series order.. feel free to do this in different orders

 git merge mer/60-wait-longer-for-threads-to-die
 git merge mer/25-gatomic
 git merge mer/30-gfileutils
 git merge mer/35-gmessages
 git merge mer/40-gscanner
 git merge mer/45-gunicode
 git merge mer/50-gthread
 git merge mer/70_use-monotonic-clock-for-timeouts

Previous Mer patches...

 git merge origin/mer/libtool
 git merge origin/mer/obs-fix

(up to date!)

Now we have a new release

 editor ChangeLog
 git add ChangeLog
 git commit -m"2.20.1-mer1"
 git tag 2.20.1-mer1

And a new Mer release

 git checkout --track origin/Mer
 git merge master
 editor debian/changelog
 git add debian/changelog
 git commit -m"Mer_2.20.1-mer1"
 git tag Mer_2.20.1-mer1

Now push

 git push --all
 git push --tags

The resultant merge looks like this :

Read upwards to progress in time. Note how on the right is the 'upstream' branch tagged at 2.20.1 then you see the Mer patches being applied to master which is then merged into Mer. (There is slight complexity here since the upstream changed to the *real* upstream).

Git based upstream

Working with a git based upstream is similar but we need to track the main origin branch as upstream. We'll do two things to make this sane:

rename origin to Upstream (differentiate from the branch)
to stay in keeping with the process above we'll track the Upstream/master as upstream

usually that will be:

 git remote rename origin Upstream
 git branch --track  upstream Upstream/master

Then create a local master (it may be necessary to remove the debian/ directory)

 git checkout -b master
 git rm -r debian/
 git commit -am"Cleaned up debian/"

It may however, be useful as a base for packaging:

 git checkout upstream
 mv debian/ ..
 git checkout master
 git checkout -b Mer master
 mv ../debian .
 <hack on debian/>
 git add debian
 git commit -am "initial debianize based on upstream"

Push this to the gitorious origin:

 git remote add origin git@gitorious.org:mer/$GPKG.git
 git push --mirror origin

For updates:

 git fetch Upstream

SVN based upstream

Working with an svn based upstream is similar again but it's easier to track the svn upstream.

git svn clone https://svn.tinymail.org/svn/tinymail/trunk
mv trunk $PKG
git branch -m master upstream

Then create a local master (it may be necessary to remove the debian/ directory)

 git checkout -b master
 git rm -r debian/
 git commit -am"Cleaned up debian/"

It may however, be useful as a base for packaging:

 git checkout upstream
 mv debian/ ..
 git checkout master
 git checkout -b Mer master
 mv ../debian .
 <hack on debian/>
 git add debian
 git commit -am "initial debianize based on upstream"

Push this to the gitorious origin:

 git remote add origin git@gitorious.org:mer/$GPKG.git
 git push --mirror origin

For updates:

 git fetch Upstream

Working from Gitorious

This should be written to say "Clone this repository on gitorious" and then publish back there and then submit a pull request to Mer.

For core work:

Essentially

 git clone git@gitorious.org:mer/$GPKG.git
 git checkout --track origin/Mer

determine which feature or branch needs work. Then:

 git checkout --track origin/mer/<feature>

hack...

Ready to test? You need to make a local test branch based on Mer

 git checkout Mer
 git checkout -b local
 git merge mer/<feature>

Build and test...

If this works OK then clean up your mer/<feature> branch... probably doing a --squash and push the branch to gitorious.

Retrieved from "http://bugs.maemo.org/Mer/Build/UsingGitorious"

Category: Mer

Personal tools

Navigation

Views

Mer/Build/UsingGitorious

Revision as of 16:37, 12 July 2009

Contents

Packaging

Initial Setup

Local Updates

Upstream Updates

Maintenance Releases

Overall

Developing

Moving a Package to Gitorious

Initial setup

Feature/patch branches

Verify

Push

Updating a feature branch

python-hildon

Handling new upstream

hildon-input-method

glib

Git based upstream

SVN based upstream

Working from Gitorious