作者：张华 发表于：2021-08-18
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明
问题
在ussuri (16.3.0)里有一个bug, 但在steain(15.3.4)里没有，需要二分，先通过’git tag’找到其中的16.0.0, 16.1.0, 16.3.0三个tag
15.3.4
16.0.0 https://launchpad.net/ubuntu/+source/neutron/2:16.0.0-0ubuntu2
16.0.0.0b1
16.0.0.0rc1
16.0.0.0rc2
16.1.0 https://launchpad.net/ubuntu/+source/neutron/2:16.1.0-0ubuntu2
16.2.0 https://launchpad.net/ubuntu/+source/neutron/2:16.2.0-0ubuntu2
16.3.0
前一篇关于debian的文章是： https://blog.csdn.net/quqi99/article/details/50511173
一些感受:
- 在bionic中运行git-buildpackage的pristine-tar checkout这步会报一个XD3_INVALID_INPUT错误，在focal上无此问题
- 在focal上运行git-buildpackage + git-pbuiler(cowpbuilder)时会报一个dbus相关的错误。运行git-buildpackage + sbuild或git-buildpackage + debuild都无此问题。直接运行pdebuild也无此问题(git-pbuilder用的是cowpbuilder是cow格式) .git-pbuilder创建出来的镜像里没有dbus包和环境变量（DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus），暂不清楚是否和这相关. 后来，使用git-pubilder时又提示输入什么密码。看来git-pbuilder确实有问题不能用.
- git-buildpackage从git源码编译时都要用到tarball，但tarball有通过prinstine-tar命令人工准备和它从neutron git源码中自动tarball两种方式
- 使用git-buildpackage + sbuild即可
- 源码包看debian/changelog就能确实是适合在bionic上还是bionic-steain上还是focal上编译，这样能避免包依赖问题
准备测试机器
juju deploy ubuntu debbuild --series=bionic --config hostname=debbuild --constraints "mem=8G cores=2 root-disk=80G"
# juju add-machine --series bionic
sudo cp /etc/apt/sources.list /etc/apt/sources.list_bak
cat <<EOF | sudo tee /etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu bionic-proposed main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu bionic-security main restricted universe multiverse
deb-src http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu bionic-proposed main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu bionic-backports main restricted universe multiverse
EOF
sudo apt clean && sudo apt update
使用pbuilder
会报一些包依赖的错（openstack-pkg-tools (>= 85ubuntu3~) ），见最后，后来证明和pbuilder无关，是该dsc文件就不是针对bionic的所以一堆问题。
sudo apt install pbuilder debootstrap devscripts -y
sudo apt install gnupg ubuntu-dev-tools bzr-builddeb apt-file debhelper dh-systemd openstack-pkg-tools -y
sudo apt install build-essential quilt dh-autoreconf fakeroot dpkg-dev python-sphinx dh-make -y
#sudo apt install apt-cacher-ng -y
#echo 'Acquire::http::Proxy "http://127.0.0.1:3142";' | sudo tee /etc/apt/apt.conf.d/01acng
# sometimes base.tgz can't be generated when using pbuilderrc
sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/bionic-train-base.tgz
# need to use '--basetgz' when using no pbuilderrc
sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
#chroot to image to ajust what you want
sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgzecho 'precedence ::ffff:0:0/96 100' >> /etc/gai.confapt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -yadd-apt-repository cloud-archive:train#deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main#apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EAsed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.listapt update
# then exit chroot, press ctrl-d to save# but pdebuild needs pbuilderrc
cat <<EOF | sudo tee /etc/pbuilderrc
MIRRORSITE=http://archive.ubuntu.com/ubuntu/
DEBFULLNAME='Zhang Hua'
DEBEMAIL='xxx@xxx.com'
DISTRIBUTION='bionic'
EXTRAPACKAGES=''
#EXTRAPACKAGES+=' vim sudo bash-completion wget git build-essential apt-transport-https ca-certificates ubuntu-cloud-keyring'
# Cloud Archive
#OS_RELEASE='train'
if [ -n "\$OS_RELEASE" ]; thenBASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-\$OS_RELEASE-base.tgz"
elseBASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-base.tgz"
fi
EOF
mkdir -p /bak/work/debian/neutron/16.1.0 && cd /bak/work/debian/neutron/16.1.0
dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.1.0-0ubuntu2/neutron_16.1.0-0ubuntu2.dsc
dpkg-source -x neutron_16.1.0-0ubuntu2.dsc
cd /bak/work/debian/neutron/16.1.0/neutron-16.1.0
# install dependency, use mk-build-deps instead of 'apt build-dep neutron'
add-apt-repository cloud-archive:train #also add uca
sudo apt install devscripts equivs -y
sudo mk-build-deps --install debian/control
# dpkg-source: error: aborting due to unexpected upstream changes
rm -rf debian/source/fomat
sudo pdebuild --debug
#sudo pdebuild --debbuildopts -sa #when hitting 'Unable to find xxx.orig.tar.gz in upload or distribution'
#DEB_BUILD_OPTIONS=nocheck debuild -us -uc
报这个错：
ubuntu@debbuild:/bak/work/debian/neutron/16.1.0/neutron-16.1.0$ sudo pdebuild
W: /home/ubuntu/.pbuilderrc does not exist
dpkg-checkbuilddeps: error: Unmet build dependencies: debhelper-compat (= 12) openstack-pkg-tools (>= 85ubuntu3~) python3-pbr (>= 4.0.0)
W: Unmet build-dependency in source
dpkg-source: info: using options from neutron-16.1.0/debian/source/options: --extend-diff-ignore=^[^/]*[.]egg-info/
pyversions: missing X(S)-Python-Version in control file, fall back to debian/pyversions
pyversions: missing debian/pyversions file, fall back to supported versions
py3versions: no X-Python3-Version in control file, using supported versions
debian/rules:42: warning: overriding recipe for target 'override_dh_installinit'
/usr/share/openstack-pkg-tools/pkgos.make:43: warning: ignoring old recipe for target 'override_dh_installinit'
dh clean --with python3 --buildsystem=pybuild
dh: Compatibility levels before 5 are no longer supported (level 1 requested)
debian/rules:17: recipe for target 'clean' failed
make: *** [clean] Error 25
原因是运行pbuilder的机器上也得添加train uca并安装(pbuilder镜像里也记得添加train uca哦)，也别记了将pbuilderrc中的RELEASE放开来构建正确的BASETGZ变量：
apt install debhelper-compat openstack-pkg-tools python3-pbr -y
但是接着会报下列错，这是debian包的问题，因为我们编译的neutron-16.1.0是位于train和ussuri之间的版本，并不是严格的train (train中的debain/control中的确实是python3-neutron-lib (>= 1.29.1),)
pbuilder-satisfydepends-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installedDepends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installedDepends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installedpython3-cryptography : Depends: libssl1.1 (>= 1.1.1) but 1.1.0g-2ubuntu4 is installed
后来又这样：
sudo mv /etc/pbuilderrc /tmp/
sudo pbuilder --create --distribution focal --architecture amd64 --debootstrapopts --variant=buildd
sudo mv /tmp/pbuilderrc /etc/
sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/focal-base.tgz
sudo pbuilder --login --basetgz /var/cache/pbuilder/focal-base.tgz --save-after-login
#/etc/pbuilderrc will point out 'BASETGZ='
gbp buildpackage --git-pbuilder --git-dist=focal --git-arch=amd64 --git-ignore-branch --git-ignore-new -uc -us -j8
结果又提示输出密码：
[sudo] password for pbuilder:
看来gbp与pbuilder结合这一块确实有问题
使用sbuild
最后遇到错误”Not removing build depends: cloned chroot in use"
#using sbuild - https://wiki.ubuntu.com/SecurityTeam/BuildEnvironment
sudo apt install sbuild debhelper ubuntu-dev-tools piuparts moreutils schroot gnupg-agent -y
sudo adduser $USER sbuild
mkdir -p $HOME/ubuntu/scratch
cat <<EOF | sudo tee -a /etc/schroot/sbuild/fstab
/home/$USER/ubuntu/scratch /scratch none rw,bind 0 0
EOF
cat <<EOF | tee ~/.sbuildrc
# Name to use as override in .changes files for the Maintainer: field
# (mandatory, no default!).
\$maintainer_name='Zhang Hua <xxx.zhang@xxx.com>';
# Default distribution to build.
\$distribution = "bionic";
# Build arch-all by default.
\$build_arch_all = 0;# When to purge the build directory afterwards; possible values are "never",
# "successful", and "always". "always" is the default. It can be helpful
# to preserve failing builds for debugging purposes. Switch these comments
# if you want to preserve even successful builds, and then use
# "schroot -e --all-sessions" to clean them up manually.
\$purge_build_directory = 'successful';
\$purge_session = 'successful';
\$purge_build_deps = 'successful';
# $purge_build_directory = 'never';
# $purge_session = 'never';
# $purge_build_deps = 'never';# Directory for writing build logs to
\$log_dir=\$ENV{HOME}."/ubuntu/logs";# don't remove this, Perl needs it:
1;
EOF
mkdir -p $HOME/ubuntu/{build,logs}
cat <<EOF | tee ~/.mk-sbuild.rc
SCHROOT_CONF_SUFFIX="source-root-users=root,sbuild,admin
source-root-groups=root,sbuild,admin
preserve-environment=true"
SKIP_UPDATES="1"
SKIP_PROPOSED="1"
EOF
#mk-sbuild bionic --arch=amd64 --skip-updates --debootstrap-mirror=http://<mirror>/ubuntu --distro=ubuntu
# sg sbuild #do not need this line if you reenter ssh
mk-sbuild bionic --arch=amd64 --skip-updates# install uca
schroot -l
schroot -c source:bionic-amd64 -u rootecho 'precedence ::ffff:0:0/96 100' >> /etc/gai.confapt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -yadd-apt-repository cloud-archive:train#deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main#apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EAapt updatechmod 1777 /tmp && apt update
exitTo CHANGE the golden image: sudo schroot -c source:bionic-amd64 -u rootTo ENTER an image snapshot: schroot -c bionic-amd64To BUILD within a snapshot: sbuild -A -d bionic-amd64 PACKAGE*.dscTo BUILD for : sbuild -A -d bionic-amd64 --host PACKAGE*.dscmkdir -p /bak/work/debian/neutron/16.1.0 && cd /bak/work/debian/neutron/16.1.0
dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.1.0-0ubuntu2/neutron_16.1.0-0ubuntu2.dsc
dpkg-source -x neutron_16.1.0-0ubuntu2.dsc
sbuild-update -udc bionic-amd64 #it equals: apt update && apt upgrade
cd /bak/work/debian/neutron/16.1.0/neutron-16.1.0
sbuild --verbose -d bionic-amd64 ../neutron_*.dsc #use .. to point out the parent directory
#dsc: amd64 not in arch list or does not match any arch wildcards: all -- skipping
# ../neutron_*.dsc is using arch=all so need to use --arch-all for sbuild
sbuild --verbose -d bionic-amd64 --arch-all ../neutron_*.dsc
# Not removing build depends: cloned chroot in use
# E: Unable to correct problems, you have held broken packages.
镜像里添加了UCA后，报的错也和pbuilder是一样的，当然，这种错误和publider和sbuild本身都无关,所以sbuild也运行正确。
sbuild-build-depends-neutron-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installedDepends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installedDepends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed
注意，要改为使用focal的话，或者修改默认值（sed -i -e “s/bionic/focal/g” ~/.sbuildrc），或者使用（‘–git-builder=sbuild -As -d focal-amd64’）
gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d focal-amd64' --git-dist=focal --git-arch=amd64 -j8
使用debuild
先使用sbuild快速弄一个bionic环境出来：
sudo cp /etc/schroot/chroot.d/sbuild-bionic-amd64 /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
sudo cp -r /var/lib/schroot/chroots/bionic-amd64 /var/lib/schroot/chroots/bionic-amd64-tmp
sudo sed -i -e "s/bionic-amd64/bionic-amd64-tmp/g" /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
#sudo rm /etc/schroot/chroot.d/sbuild-bionic-amd64-tmp
#sudo rm -rf /var/lib/schroot/chroots/bionic-amd64-tmp
schroot -l
schroot -c source:bionic-amd64-tmp -u root
再试debuild
apt install devscripts curl ca-certificates bash-completion -y
dget https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/neutron/2:16.0.0-0ubuntu2/neutron_16.0.0-0ubuntu2.dsc
dpkg-source -x neutron_16.0.0-0ubuntu2.dsc
cd neutron-16.0.0/
# install dependency, use mk-build-deps instead of 'apt build-dep neutron'
apt install devscripts equivs -y
mk-build-deps --install debian/control
apt install debhelper-compat openstack-pkg-tools python3-pbr -y
# install some dependency according to the output of 'debuild -i -us -uc -S'
apt install debhelper-compat dh-python openstack-pkg-tools python3-all python3-pbr -y
#build unsigned source package
debuild -i -us -uc -S
debuild的分解动作如下：
- 清理源代码树(debian/rules clean)
- 构建源代码包(dpkg-source -b)
- 构建程序(debian/rules build)
- 构建二进制包(fakeroot debian/rules binary)
- 使用 gpg 签署 .dsc 文件 使用
- dpkg-genchanges 和 gpg 创建并签署上传用的.changes 文件
报这个错：
dpkg-buildpackage: info: binary and diff upload (original source NOT included)
那就加-sa参数。
debuild -i -us -uc -S -sa
上面测16.0.1因为源码有问题不典型，让我们改测train(15.3.4)来只证明debuild可以即可(注意：下列参数-S也改成了-b):
apt-get source neutron
mk-build-deps --install debian/control
debuild -i -us -uc -b
通过自己从头构建熟悉git-buildpackage原理
https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html
git-buildpackage的原理是：
- 准备git工程. 注意：这里初始添加了README.md之后会报：will cause dpkg-source: error: aborting due to unexpected upstream changes，但初始不加，在git branch时看不到master与upstream分支，还是先添加，呆会再删除
mkdir neutron-git-buildpackage && cd neutron-git-buildpackage
echo "# neutron-git-buildpackage" >> README.md
git init
git add README.md
git commit -m "first commit"
#git remote add origin git@github.com:zhhuabj/neutron-git-buildpackage.git
#git push origin master
- 会将neutron source code (not debian/ directory)提交到upstream branch，故先创建它：
git checkout -b upstream
- 回到master分支(此时好像不能创建新分支，是都测试好之后再创建分支）并准备neutron source code, 'gbp import-dsc’会自动将neutron source code下载下来，并提交到upstream branch
git checkout master
# gbp import-orig /tmp/tarball/new-upstream.tar.gz -u 15.3.4
gbp import-dsc https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
- 也在master分支准备debianized repository, 这时master中同时有upstream中的neutron源码和debian目录（Imported the debian/ directory into master as well as upstream’s files）
git checkout master
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
git add *
git commit -m 'init master'
- 此时状态是：
ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git branch
* masterupstream
ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ ls
AUTHORS LICENSE api-ref debian neutron releasenotes setup.py
CONTRIBUTING.rst PKG-INFO babel.cfg devstack neutron.egg-info requirements.txt test-requirements.txt
ChangeLog README.rst bin doc playbooks roles tools
HACKING.rst TESTING.rst bindep.txt etc rally-jobs setup.cfg tox.ini
ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git checkout upstream
Switched to branch 'upstream'
ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ ls
AUTHORS HACKING.rst README.rst babel.cfg devstack neutron rally-jobs roles test-requirements.txt
CONTRIBUTING.rst LICENSE TESTING.rst bin doc neutron.egg-info releasenotes setup.cfg tools
ChangeLog PKG-INFO api-ref bindep.txt etc playbooks requirements.txt setup.py tox.ini
ubuntu@debbuild:/bak/work/neutron-git-buildpackage$ git tag
debian/2%15.3.4-0ubuntu1_cloud1
upstream/15.3.4
- pristine-tar分支中准备tarball
git checkout -b pristine-tar
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4.orig.tar.gz
chmod +x neutron_15.3.4.orig.tar.gz
git add neutron_15.3.4.orig.tar.gz
git commit -m 'neutron_15.3.4.orig.tar.gz'
# pristine-tar: committed neutron_15.3.4.orig.tar.gz.delta to branch pristine-tar
pristine-tar commit ./neutron_15.3.4.orig.tar.gz 15.3.4
pristine-tar list |grep 15
#Remember to tell git to push both the pristine-tar branch, and your tag:
#git push --all --tags
#once your local tarball has lost - http://manpages.ubuntu.com/manpages/trusty/man1/pristine-tar.1.html
#pristine-tar checkout ../hello-1.0.tar.gz
或者直接从上游git中获取
# generate tarball from upstream
cd /bak/openstack/neutron
git checkout master
git checkout -b 15.3.4 15.3.4
git archive --output=../neutron_15.3.4.tar.gz --format=tar HEAD# commit tarball into this git repo
pristine-tar commit /bak/openstack/neutron_15.3.4.orig.tar.gz 15.3.4
- 之前就可以回master分支，安装依赖并运行gbp buildpackage了，它会调用’dpkg-buildpackage -rfakeroot -us -uc -ui -i -I’。都测试完之后，就可以将master分支打tag了
git checkout master
# also need to enable UCA
sudo mk-build-deps --install debian/control #install dependency
gbp buildpackage --git-ignore-branch --git-ignore-new
总结，这个重点，即要安装UCA，也要安装依赖，git有改还得添加–git-ignore-new来忽略改动。
但是它会报这个错：#neutron_15.3.4.orig.tar.gz does not match stored hash，用下列workaround就OK啦：
#neutron_15.3.4.orig.tar.gz does not match stored hash
pristine-tar checkout ../neutron_15.3.4.orig.tar.gz
gbp buildpackage --git-tarball-dir=/bak/work --git-ignore-branch --git-ignore-new
解决运行gbp import-dsc过程中遇到的问题
https://wiki.debian.org/PackagingWithGit
得将neutron source code 导入到debianized repository中去，
1, 先看通过"gbp import-dsc"从dsc中导入neutron source code
可以直接从neutron的上游下载source code:
https://releases.openstack.org/train/#train-neutron
https://tarballs.openstack.org/neutron/neutron-15.3.4.tar.gz
也可以用’gbp import-dsc’从dsc中导入neutron source code
sudo apt install git-buildpackage -y
mkdir -p /tmp/deb
https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
sudo chown -R $USER ~/.gnupg/
# bakup gpg keys
gpg --output mygpgkey_pub.gpg --armor --export $GPGKEY
gpg --output mygpgkey_sec.gpg --armor --export-secret-key $GPGKEY
# import gpg keys from other machine
sudo apt-get install rng-tools gnupg-agent -y && sudo rngd -r /dev/urandom && killall -q gpg-agent && eval $(gpg-agent --daemon)
gpg --import mygpgkey_pub.gpg
gpg --allow-secret-key-import --import mygpgkey_sec.gpg
gpg --list-public
gpg --list-sig
export GPGKEY=C7100A4CD0D8F3AE44212746E6A84FFFCF31A67F
gpg --send-keys --keyserver keyserver.ubuntu.com $GPGKEY # import dsc. the dsc provider signs dsc, and dsc user verifies dsc
gpg --verify ./neutron_15.3.4-0ubuntu1~cloud1.dsc
gpg --search-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
# you can ignore 'There is no indication that the signature belongs to the owner'
# as it simply means you have not established a web of trust with other GPG users
gpg --verify ./neutron_15.3.4-0ubuntu1~cloud1.dsc
rm -rf ./neutron_15.3.4-0ubuntu1~cloud1.dsc
git config --global user.email "xxx.zhang@xxx.com"
git config --global user.name "Zhang Hua"
＃ Then export the key to your local trustedkeys to make it trusted:
gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
# but it still says: gpg: Can't check signature: No public key
gbp import-dsc https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
尽管已经导入key, 但上面使用’gbp import-dsc’仍然报“Can’t check signature: No public key"。是因为还要将dsc provider的公钥 export the key to your local trustedkeys to make it trusted
＃ Then export the key to your local trustedkeys to make it trusted:
gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
如果还是不行，可以使用下列dget命令作为workaround，dget即使没有证书也会下载文件：
sudo apt-get install debian-keyring -y
dget --download-only -q https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.dsc
dpkg-source -x neutron_15.3.4-0ubuntu1~cloud1.dsc
gpg --keyserver keyserver.ubuntu.com --recv-keys 3516D1A5BF0077E71414E19715B9B3EE0DCDF806
＃ Then export the key to your local trustedkeys to make it trusted:
gpg --no-default-keyring -a --export 3516D1A5BF0077E71414E19715B9B3EE0DCDF806 | gpg --no-default-keyring --keyring ~/.gnupg/trustedkeys.gpg --import -
使用ubuntu-openstack-dev debian源码构建
上面熟悉了从零构建git-buildpackage工程，实际上ubuntu-openstack-dev中已经有了这个工程了。
- master分支应该同时有debian目录和neutron源码
- upstream分支只有neutron源码，没有debain目录
- pristine-tar分支只有tarball，并且使用pristine-tar list命令能看到，也要能运行pristine-tar checkout …/xxx.tar.gz
- 编译时是在master分支运行
- 暂不清楚如果编译15.3.4分支，因为15.3.4下没有debain目录，有debian目录的是这个：
ubuntu@debbuild:/bak/work/debsource/neutron$ git tag |grep debian |grep 15.3.4
debian/2%15.3.4-0ubuntu1_cloud0
debian/2%15.3.4-0ubuntu1_cloud1
git-buildpackage命令运行在neutron代码分支之后，会先使用’pristine-tar checkout …/xxx.orig.tar.gz’先tarball弄到neutron代码分支（tarball存在pristine-tar分支中），然后再build.
# https://wiki.ubuntu.com/OpenStack/CorePackages
sudo apt install git-buildpackage -y
sudo mkdir -p /bak/work/debsource && sudo chown -R $USER /bak && cd /bak/work/debsource
git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
cd neutron
git checkout pristine-tar
git checkout upstream
git checkout master# 仅编译15.3.4, 这个tag (debian/2%15.3.4-0ubuntu1_cloud0)有debian目录，应该用它
git checkout -b debian/2%15.3.4-0ubuntu1_cloud0 debian/2%15.3.4-0ubuntu1_cloud0
git checkout debian/2%15.3.4-0ubuntu1_cloud0
gbp buildpackage --git-ignore-branch --git-ignore-new -us -uc
但是上面命令报下面这个错：
xdelta3: target window checksum mismatch: XD3_INVALID_INPUT
所以本想用下列workaround (–git-tarball-dir=/bak/openstack/tarball)来避开上面问题，但是又报不匹配的错
# generate tarball from upstream
cd /bak/openstack/neutron
git checkout master
git checkout -b 15.3.4 15.3.4
git archive --output=/bak/openstack/tarball/neutron_15.3.4.orig.tar.gz --format=tar HEADgbp buildpackage --git-tarball-dir=/bak/openstack/tarball --git-ignore-branch --git-ignore-new -us -uc
看来还是得想办法解决XD3_INVALID_INPUT的问题。得想办法运行下列命令成功：
pristine-tar checkout ../neutron_15.3.4.orig.tar.gz
但这个命令在focal上是能运行成功的，可能是bionic的pristine-tar版本有问题吧，可以在focal导入tarball后再拷到bionic中，再使用’–git-tarball-dir='来workaroud.
或者我们可以直接在focal中使用pbuilder + git-buildpackage 来规避这个问题。
使用pbuilder + ubuntu-openstack-dev debian源码构建
第一步，创建pbuilder环境。这里的pbuilder实际上是cowpbuilder, 镜像格式是cow格式。
sudo apt-get install git-buildpackage cowbuilder debian-archive-keyring -y
#it will delete /var/cache/pbuilder/aptcache
#sudo pbuilder clean#sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
rm -rf sudo rm -rf /var/cache/pbuilder/base-bionic-amd64.cow
sudo mv /etc/pbuilderrc /tmp/
DIST=bionic ARCH=amd64 git-pbuilder create
sudo mv /tmp/pbuilderrc /etc/#sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
DIST=bionic ARCH=amd64 git-pbuilder update
#sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
DIST=bionic ARCH=amd64 git-pbuilder login --save-after-loginecho 'precedence ::ffff:0:0/96 100' >> /etc/gai.confapt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -yadd-apt-repository cloud-archive:train#deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main#apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EAsed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.listapt updateexit
第二步，使用pubilder (在gbp buuildpackage中添加–git-pbuilder --git-dist=bionic --git-arch=amd64）编译
sudo apt install devscripts equivs -y
sudo mkdir -p /bak/work/debsource && sudo chown -R $USER /bak && cd /bak/work/debsource
git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
cd neutron
git checkout pristine-tar
git checkout upstream
git checkout master
git checkout -b debian/2%15.3.4-0ubuntu1_cloud0 debian/2%15.3.4-0ubuntu1_cloud0
git checkout debian/2%15.3.4-0ubuntu1_cloud0#sudo mk-build-deps --install debian/control
#gbp buildpackage --git-ignore-branch --git-ignore-new -us -uc
gbp buildpackage --git-pbuilder --git-dist=bionic --git-arch=amd64 --git-ignore-branch --git-ignore-new -uc -us -j16
#debuild clean
在neutron源码中自动导入tarball构建
下面直接从neutron源码中使用gbp来编译:
https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html
1, 首先，neutron source code必须从orgin更名为upstream避免和debian repository混淆
git clone --no-checkout -o upstream https://github.com/openstack/neutron
2, 将15.3.4的源码切换到debian/15.3.4分支，并且将debian目录加入到debian分支
cd neutron
git checkout -b debian/15.3.4 15.3.4
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
git add debian/
git commit -m 'add debian'
#gbp:error: Couldn't commit to 'pristine-tar' with upstream '15.3.4': pristine-tar: more than one ref matches "15.3.4":
3, 因为debian/15.3.4分支有debian目录，应该是在这个分支下运行编译命令，这们这里没有指定tarball包因为’–git-pristine-tar --git-pristine-tar-commit '会自动从源码中准备tarball
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 -us -uc
若更新了源文件的话, ‘gbp dch’会在changelog中添加一记录（15.3.4-0ubuntu1cloud1ubuntu11.gbp5d92a7），’–git-ignore-new’用于告诉 'gbp buildpackage’工作目录不干净。
git fetch upstream
git merge 15.3.4
# gbp:info: Changelog 2:15.3.4-0ubuntu1~cloud1ubuntu1~1.gbp5d92a7 (snapshot #1) prepared up to 5d92a7b
gbp dch --debian-branch=debian/15.3.4 --snapshot --auto debian/
gbp buildpackage --git-ignore-new --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' -us -uc
测试没问题之后，修改changelog和build一个新版本
gbp dch --release --auto --debian-branch=debian/15.3.4
git commit -m"Release 1.1-1" debian/changelog
gbp buildpackage --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 -us -uc
再总结 - 通过自己从头构建熟悉git-buildpackage原理
这个是对”通过自己从头构建熟悉git-buildpackage原理“这一节的总结，这里还是要自己准备tarball麻烦，最好参考”通过neutron git来使用gbp“这一节由git来自动处理tarball. 还最好考虑git-buildpackage怎么与pbuilder结合(git-pbuilder)。
1, 准备git工程, 注意：这里初始添加了README.md之后会报：will cause dpkg-source: error: aborting due to unexpected upstream changes，但初始不加，在git branch时看不到master与upstream分支，还是先添加，呆会再删除 (注意：最后一次试就又遇到这个问题，不需要删除）.
mkdir neutron-git-buildpackage2 && cd neutron-git-buildpackage2
echo "# neutron-git-buildpackage" >> README.md
git init
git add README.md
git commit -m "first commit"
#git remote add origin git@github.com:zhhuabj/neutron-git-buildpackage.git
#git push origin master
2, neutron源码工程下准备 tarball
mkdir -p /bak/openstack/tarball/ && cd /bak/openstack
git clone https://github.com/openstack/neutron
cd neutron
git checkout master
git checkout -b 15.3.4 15.3.4
mkdir -p /tmp/tarball
git archive --output=/tmp/tarball/neutron_15.3.4.orig.tar.gz --format=tar.gz HEAD
3, 创建upstream分支
git checkout -b upstream
4, 回到master分支（这时不能使用15.3.4，但master测试没问题后再加tag为15.3.4)导入tarball
git checkout master
gbp import-orig /tmp/tarball/neutron_15.3.4.orig.tar.gz -u 15.3.4
只有使用’gbp import-orig’导入tarball包才会提交到upstream分支，并merge到master分支（也就是upstream与master分支都有）, 此时状态为：
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git branch
* masterupstream
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
CONTRIBUTING.rst LICENSE TESTING.rst babel.cfg bindep.txt doc neutron rally-jobs requirements.txt setup.cfg test-requirements.txt tox.ini
HACKING.rst README.rst api-ref bin devstack etc playbooks releasenotes roles setup.py tools
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git checkout upstream
Switched to branch 'upstream'
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
CONTRIBUTING.rst LICENSE TESTING.rst babel.cfg bindep.txt doc neutron rally-jobs requirements.txt setup.cfg test-requirements.txt tox.ini
HACKING.rst README.rst api-ref bin devstack etc playbooks releasenotes roles setup.py tools
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git tag
upstream/15.3.4
5, master分支中添加debian目录(此时只能用master，不能创建tag, tag可在master中测试好了再打)
git checkout master
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
git add debian
git commit -m 'add debain'
6, pristine-tar commit tarball, 总之，通过’pristine-tar list’ 得能看到neutron_15.3.4.orig.tar.gz 15.3.4, 这样编译系统在编译的时候才能通过’pristine-tar checkout …/neutron_15.3.4.orig.tar.gz’获取它。
pristine-tar commit /tmp/tarball/neutron_15.3.4.orig.tar.gz 15.3.4
pristine-tar list
pristine-tar commit会自动添加pristine-tar分支：
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ git checkout pristine-tar
Switched to branch 'pristine-tar'
ubuntu@debbuild:/bak/work/neutron-git-buildpackage2$ ls
neutron_15.3.4.orig.tar.gz.delta neutron_15.3.4.orig.tar.gz.id
７，回到master分支编译. 它是使用debuild来编译的，应该也可以和pbuilder等集成。
git checkout master
gbp buildpackage --git-ignore-branch --git-ignore-new
上面会遇到这个错误”pristine-tar: /bak/work/build-area/neutron_15.3.4.orig.tar.gz does not match stored hash“，下面使用pristine-tar checkout时也会报这个错，直接用–git-tarall-dir来指定之前使用git archive创建的tarball来workaround就好了（-us -uc意为不签名)：
#pristine-tar checkout /tmp/tarball/neutron_15.3.4.orig.tar.gz
#gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -S -us -uc -sa
gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -us -uc
8，编译后的包在/bak/work/build-area目录中
9, 都没问题之后，打tag归档，今后可以在该tag下运行编译命令了：
git checkout master
git tag '15.3.4'
git checkout -b 15.3.4 15.3.4git checkout 15.3.4
gbp buildpackage --git-tarball-dir=/tmp/tarball --git-ignore-branch --git-ignore-new -us -uc
再总结 - 在neutron源码中自动导入tarball构建
这是针对"在neutron源码中自动导入tarball构建”一节更快速的总结.
sudo apt install git-buildpackage pristine-tar -y
# git clone --no-checkout -o upstream https://github.com/openstack/neutron
git clone https://github.com/openstack/neutron
cd neutron
# create upstream branch
git checkout master && git checkout -b upstream
# create pristine-tar branch, but we don't need to prepare tarball because '--git-pristine-tar --git-pristine-tar-commit' will help us create it automatically
git checkout master && git checkout -b pristine-tar
# create debain/15.3.4 branch with upstream neutron=15.3.4
git checkout master && git checkout -b debian/15.3.4 15.3.4
# add debian package into debian/15.3.4
git checkout debian/15.3.4
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
git add debian
git commit -m 'add debian for 15.3.4'
# run 'gbp buildpackage' command under the directory which has debian
# --git-pristine-tar and --git-pristine-tar-commit will create tarball automatically
git checkout debian/15.3.4
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
报下列错，那是因为要检出15.3.4分支(git checkout master && git checkout -b 15.3.4 15.3.4)：
ubuntu@debbuild:/bak/openstack/neutron$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
gbp:error: Couldn't commit to 'pristine-tar' with upstream '15.3.4': pristine-tar: more than one ref matches "15.3.4":
480f4b7479bc0b19752edde46fe90660a94cc654 refs/heads/debian/15.3.4
729f6aefe103a0ba10b71888f84b6247d694bc04 refs/tags/15.3.4
附录 - 如何检查包依赖问题
上面种种方法都有这么一个问题（openstack-pkg-tools (>= 85ubuntu3~)），所以我们有必要检查这个源码包究竟有没有问题。
原来，错误原因是必须添加：sudo add-apt-repository cloud-archive:stein
sudo add-apt-repository cloud-archive:stein
sudo sed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.list
sudo apt update
sudo apt-cache policy neutron-l3-agent
sudo apt-get source neutron=2:15.3.4-0ubuntu1~cloud1
cd neutron-15.3.4/
$ grep -r 'openstack-pkg-tools' debian/control openstack-pkg-tools (>= 85ubuntu3~),
$ sudo apt-cache policy openstack-pkg-tools |grep CandidateCandidate: 99ubuntu1~cloud0
附录 - 其他
1, gbp buildpackage之后无问题，可以加tag，会自动将neutron source code和debian source code加tag: debian/2%15.3.4-0ubuntu1_cloud1
gbp buildpackage --git-tag-only --git-sign-tags --git-ignore-new
#git checkout -b debian/2%15.3.4-0ubuntu1_cloud1 debian/2%15.3.4-0ubuntu1_cloud1
2, 然后可以推入远程repo
git push origin --all --follow-tags
最佳实践
- 使用pbuilder避免在物理机上安装依赖
- 直接使用neutron源码，并在neutron源码自动产生tarball的方式
第一步，创建pbuilder环境
#sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
rm -rf sudo rm -rf /var/cache/pbuilder/base-bionic-amd64.cow
sudo mv /etc/pbuilderrc /tmp/
DIST=bionic ARCH=amd64 git-pbuilder create
sudo mv /tmp/pbuilderrc /etc/#sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-train-base.tgz
DIST=bionic ARCH=amd64 git-pbuilder update
#sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-train-base.tgz
DIST=bionic ARCH=amd64 git-pbuilder login --save-after-loginecho 'precedence ::ffff:0:0/96 100' >> /etc/gai.confapt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -y# Failed to connect to socket /run/user/1000/bus: No such file or directoryapt install dbus -y && /etc/init.d/dbus startadd-apt-repository cloud-archive:train#deb http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/train main#apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EAsed -i -e "s/# deb-src/deb-src/" /etc/apt/sources.list.d/cloudarchive-train.listapt updateexit
第二步，准备neutron源码环境，还要创建upstream与prinstine-tar两个分支，以及要编译的15.3.4分支，以及存放15.3.4源码及debian目录的debian/15.3.4分支
sudo apt install git-buildpackage pristine-tar -y
git clone https://github.com/openstack/neutron
cd neutron
git checkout master && git checkout -b upstream
git checkout master && git checkout -b pristine-tar
git checkout master && git checkout -b debian/15.3.4 15.3.4
git checkout master && git checkout -b 15.3.4 15.3.4git checkout debian/15.3.4
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
git add debian
git commit -m 'add debian for 15.3.4'
第三步，使用了git-builder来编译。并且通过“–git-pristine-tar --git-pristine-tar-commit ”来配置自动导出tarball
git checkout debian/15.3.4
# dont's use git-builder
#gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc# use git-builder
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc --git-pbuilder --git-dist=bionic --git-arch=amd64 -j8
但是会报下列dbus的错：
dbus.exceptions.DBusException: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /run/user/1000/bus: No such file or directory
debian/rules:16: recipe for target 'override_dh_install' failed
试了下面的还是不行。
DIST=bionic ARCH=amd64 git-pbuilder login --save-after-login
apt install dbus
/etc/init.d/dbus start
exit
git-pbuilder创建出来的镜像里没有dbus包和环境变量（DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus），暂不清楚是否和这相关。
如果不使用git-builder，直接运行’gpb buildpackage’ + debuild模式，it works. 注意：下列这个测试没用pbuilder只能在bionic上进行。
# NOTE: must run on bionic
sudo mk-build-deps --install debian/control
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc -j8
将pbuilder换成sbuild也成功了（换sbuild时要去掉-us -uc, 见：https://www.pseudorandom.co.uk/2008/sbuild-dm/ , 另外，使用的是–git-builder=‘sbuild -A’， -A代表all-arch，构建源码包时用: --git-builder=‘sbuild -As’
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new --git-builder='sbuild -A' --git-dist=bionic --git-arch=amd64 -j8
在neutron debian source code(https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron)中运行pbuilder也出现了dbus的错误，现在确认是这个dbus错误只是在pbuilder中才出现。
示例：ubuntu debian源码上编译16.1.0
neutron debian source 上运行：
git checkout -b debian/2%16.1.0-0ubuntu2 debian/2%16.1.0-0ubuntu2
gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As' --git-dist=bionic --git-arch=amd64 -j8
但会报错：
sbuild-build-depends-main-dummy : Depends: python3-neutron-lib (>= 2.2.0) but 1.29.1-0ubuntu1~cloud0 is to be installedDepends: python3-ovsdbapp (>= 1.0.0) but 0.17.0-0ubuntu1~cloud0 is to be installedDepends: python3-pyroute2 (>= 0.5.7) but 0.5.6-0ubuntu1~cloud0 is to be installed
那是因为镜像中使用的是train UCA(python3-neutron-lib=1.29.1)，train对应的是15.3.4，并不是16.1.0，所以有问题。vimdiff一下：
git checkout debian/2%16.1.0-0ubuntu2
mkdir -p /tmp/16.1.0 && cp -r debian /tmp/16.1.0/
git checkout debian/2%15.3.4-0ubuntu1_cloud0
mkdir -p /tmp/15.3.4 && cp -r debian /tmp/15.3.4/
vimdiff /tmp/15.3.4/debian/control /tmp/16.1.0/debian/control15.3.4 -> python3-neutron-lib (>= 1.29.1),
16.1.0 -> python3-neutron-lib (>= 2.2.0),
python3-neutron-lib 2.2.0来自focal前的一个版本也就是19.10
apt install devscripts liburi-perl wget -y
root@node1:/# rmadison python3-neutron-libpython3-neutron-lib | 0.0.2-2 | xenial/universe | allpython3-neutron-lib | 1.13.0-0ubuntu1 | bionic/universe | allpython3-neutron-lib | 2.3.0-0ubuntu1 | focal | allpython3-neutron-lib | 2.10.1-0ubuntu1 | hirsute | allpython3-neutron-lib | 2.12.0-0ubuntu1 | impish | all$ cmadison python3-neutron-libpython3-neutron-lib | 0.0.2-2~cloud0 | mitaka | trusty-updates | allpython3-neutron-lib | 0.0.2-2~cloud0 | mitaka-proposed | trusty-proposed | allpython3-neutron-lib | 1.13.0-0ubuntu1~cloud0 | queens | xenial-updates | allpython3-neutron-lib | 1.13.0-0ubuntu1~cloud0 | queens-proposed | xenial-proposed | allpython3-neutron-lib | 1.25.0-0ubuntu1~cloud0 | stein | bionic-updates | allpython3-neutron-lib | 1.25.0-0ubuntu1~cloud0 | stein-proposed | bionic-proposed | allpython3-neutron-lib | 1.29.1-0ubuntu1~cloud0 | train | bionic-updates | allpython3-neutron-lib | 1.29.1-0ubuntu1~cloud0 | train-proposed | bionic-proposed | allpython3-neutron-lib | 2.3.0-0ubuntu1~cloud0 | ussuri | bionic-updates | allpython3-neutron-lib | 2.3.0-0ubuntu1~cloud0 | ussuri-proposed | bionic-proposed | allpython3-neutron-lib | 2.6.1-0ubuntu1~cloud0 | victoria | focal-updates | allpython3-neutron-lib | 2.6.1-0ubuntu1~cloud0 | victoria-proposed | focal-proposed | allpython3-neutron-lib | 2.10.1-0ubuntu1~cloud0 | wallaby | focal-updates | allpython3-neutron-lib | 2.10.1-0ubuntu1~cloud0 | wallaby-proposed | focal-proposed | allpython3-neutron-lib | 2.12.0-0ubuntu1~cloud0 | xena | focal-updates | allpython3-neutron-lib | 2.12.0-0ubuntu1~cloud0 | xena-proposed | focal-proposed | all
所以上面的16.1.0的源码显然是针对focal来做的（这里也有显示：https://launchpad.net/ubuntu/+source/neutron/2%3A16.1.0-0ubuntu2/+publishinghistory），所以：
- 目前bionic-stein的版本是15.3.4, focal-ussuri的版本是16.4.0, 现在想要测的是16.1.0
- 如果在bionic里测16.1.0的话，得修改debian/control与requirment.txt中的发生问题的pip版本号，但这也不一定行，因为neutron代码也变了既然requirement.txt说依赖某个包的高版本可能就真是需要高版本
- 那样就只能在focal里测16.1.0了。将focal现有的16.4.0改成16.1.0来测，sbuild的镜像什么的都得由bionic改成16.1.0
# create focal sbuid image
mk-sbuild focal --arch=amd64
schroot -l
schroot -c source:focal-amd64 -u root# clone neutron debian source code
cd /bak/work/debsource
git clone https://git.launchpad.net/~ubuntu-openstack-dev/ubuntu/+source/neutron
cd neutron
git checkout pristine-tar
git checkout upstream
git checkout master# build with focal image
git checkout -b debian/2%16.1.0-0ubuntu2 debian/2%16.1.0-0ubuntu2
git checkout debian/2%16.1.0-0ubuntu2# 注意：下面使用了--git-dist=focal --git-arch=amd64来指定用focal但实际用的还是bionic，
# 原来git-dist只是针对pbuilder，这样使用了~/.sbuildrc的默认项：sed -i -e "s/bionic/focal/g" ~/.sbuildrc
#gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As' --git-dist=focal --git-arch=amd64 -j8#所以或者使用（sed -i -e "s/bionic/focal/g" ~/.sbuildrc），或者使用（'--git-builder=sbuild -As -d focal-amd64'）
gbp buildpackage --git-debian-branch=debian/2%16.1.0-0ubuntu2 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d focal-amd64' --git-dist=focal --git-arch=amd64 -j8
示例：neutron upsteam源码上编译15.3.4
# prepare bionic-train sbuild image
sudo cp /etc/schroot/chroot.d/sbuild-bionic-amd64 /etc/schroot/chroot.d/sbuild-bionic-train-amd64
sudo cp -r /var/lib/schroot/chroots/bionic-amd64 /var/lib/schroot/chroots/bionic-train-amd64
sudo sed -i -e "s/bionic-amd64/bionic-train-amd64/g" /etc/schroot/chroot.d/sbuild-bionic-train-amd64
schroot -l
schroot -c source:bionic-train-amd64 -u rootapt install devscripts curl ca-certificates bash-completion software-properties-common vim -y# Couldn't create temporary file /tmp/apt.conf.5gKxoB for passing config to apt-keychmod 1777 /tmp# /usr/bin/apt-key: cannot create /dev/null: Permission denied# https://askubuntu.com/questions/764719/fix-permissions-after-upgrade-to-16-04chmod go+rw /dev/nulladd-apt-repository cloud-archive:traingit clone https://github.com/openstack/neutron
cd neutron
#upstream与pristine-tar不是必须的吧
#git checkout master && git checkout -b upstream
#git checkout master && git checkout -b pristine-tar
git checkout master && git checkout -b debian/15.3.4 15.3.4
git checkout master && git checkout -b 15.3.4 15.3.4git checkout debian/15.3.4
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
git add debian
git commit -m 'add debian for 15.3.4'# 已经在15.3.4分支下，--git-debian-branch=debian/15.3.4也不是必须的吧
git checkout debian/15.3.4
#gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new -us -uc
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.4 --git-ignore-branch --git-ignore-new --git-builder='sbuild -As -d bionic-train-amd64' -j8
示例 - 在15.3.4与16.0.0之间二分准备15.3.5的测试包
gbp buildpackage要想不自己准备tarball包的话，只能根据tag来做，不能根据commit来做，所以需自己准备如15.3.5与debian/15.3.5 (不自己准备tag的话似乎tarball包不更新使用15.3.4).
当然，想自己准备tarball的话，就是另外一种方式。这里讲的是不自己准备tarball包的模式。
e562ecec2669c39b3628f7744011467c7394088e (在15.3.4基础上提交debian目录的commit id)
15.3.4 - 0f18d406a4134cf769fdf6f2a004c98cbe409215
16.0.0 - c06cb95b6f46574e74244fd738873290fd1773cb# 下载代码
# first fork https://github.com/openstack/neutron
git clone https://github.com/zhhuabj/neutron.git
git config user.name
git config user.email
git remote add upstream https://github.com/openstack/neutron
git remote set-url --push upstream no_push
# update
git fetch upstream
git checkout master
git merge upstream/master
git push origin master
#git rebase upstream/master
#git push origin master -f# http://www.360doc.com/content/18/0208/09/19862658_728539054.shtml
# git二分，先找到15.3.4与16.0.0之间的中间版本1c2e10f859
cd neutron
git checkout master
git bisect start
git bisect good 15.3.4
$ git bisect bad 16.0.0
Bisecting: a merge base must be tested
[1c2e10f8595d2286bd9bec513bc5a346a84a6f7c] Merge "Remove get_external_network_id for router"
$ git status
HEAD detached at 1c2e10f859# 在上面1c2e10f859的基础上创建几个分支
git checkout -b 15.3.5
git checkout -b debian/15.3.5
# 往debian/15.3.5中添加15.3.4的debian目录
cp -r ../debian .
git add debian/ && git commit -m 'add debian'# 修改debian/changelog
dch -i
neutron (2:15.3.5-0ubuntu1~cloud0) bionic-train; urgency=medium
git commit -m "releasing package 2:15.3.5-0ubuntu1~cloud0" debian/changelog# 视情况清空debian/patches/避免一些错误, 并记得用debcommit提交
cp -r debian/patches/ /tmp/
rm -rf debian/patches/* && touch debian/patches/series
git add -u && debcommit -m 'delete series'
# 然后git log会看到：commit f1ed620b2abd4a04013ed9549fcafff5a290ff5b (HEAD -> debian/15.3.5)# 编译
git checkout 15.3.5 && git tag --no-sign 15.3.5 -m "15.3.5"
git checkout debian/15.3.5 && git tag --no-sign debian/15.3.5 -m "debian/15.3.5"
git checkout debian/15.3.5
#git push --all
#git push --tags# 测试之后可以使用'git bisect bad'或'git bisect good'来标记, 使用’git bisect replay‘可以看到曾经测试过的内容
git bisect log > bisectlog
git bisect bad | git bisect good# 下一个二分可以先回到15.3.4基准，然后再git merge下一个二分点
git reset --hard e562ecec2669c39b3628f7744011467c7394088e
git merge <下一个二分点># 二分完成之后
git bisect reset
一个错误：
# 错误 - https://paste.ubuntu.com/p/DtfMXg3Yq2/
# The 'weakrefmethod>=1.0.2' distribution was not found and is required by the application
1, 15.3.4与1c2e10f859源码中有对weakrefmethod的定义
# 15.3.4
$ grep -r 'weakrefmethod' requirements.txt
weakrefmethod>=1.0.2;python_version=='2.7' # PSF
# git checkout -b 1c2e10f859 1c2e10f859
$ grep -r 'weakrefmethod' requirements.txt
weakrefmethod>=1.0.2;python_version=='2.7' # PSF2, 在tarball中也有啊
hua@node1:/bak/work/tmp/build-area/neutron-15.3.5$ grep -r 'weakrefmethod' requirements.txt
weakrefmethod>=1.0.2;python_version=='2.7' # PSF3, 难道是要在/bak/work/tmp/neutron/debian/control中加吗？git add -u && debcommit -m 'add python-weakrefmethod'hua@node1:/bak/work/tmp/neutron$ git diff
diff --git a/debian/control b/debian/control
index f25fd6f9ea..3f65330fd2 100644
--- a/debian/control
+++ b/debian/control
@@ -13,6 +13,7 @@ Build-Depends:python3-all,python3-pbr (>= 4.0.0),python3-setuptools,
+ python-weakrefmethod,Build-Depends-Indep:crudini,python3-alembic (>= 0.8.10),
通过debuild调试上速问题的测试环境快速搭建：
sudo cp /etc/schroot/chroot.d/sbuild-bionic-train-amd64 /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
sudo cp -r /var/lib/schroot/chroots/bionic-train-amd64 /var/lib/schroot/chroots/bionic-train-amd64-tmp
sudo sed -i -e "s/bionic-train-amd64/bionic-train-amd64-tmp/g" /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
#sudo rm /etc/schroot/chroot.d/sbuild-bionic-train-amd64-tmp
#sudo rm -rf /var/lib/schroot/chroots/bionic-train-amd64-tmp
schroot -l
schroot -c source:bionic-train-amd64-tmp -u root
chmod 1777 /tmp
apt install git vim wget devscripts equivs -y
git clone https://github.com/zhhuabj/neutron.git && cd neutron
git bisect start
git bisect good 15.3.4
git bisect bad 16.0.0
git checkout -b 15.3.5
git checkout -b debian/15.3.5
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/train-staging/+sourcefiles/neutron/2:15.3.4-0ubuntu1~cloud1/neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
tar -xf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
rm -rf neutron_15.3.4-0ubuntu1~cloud1.debian.tar.xz
# mkdir /home/hua
#git config --global user.email "you@example.com"
#git config --global user.name "Your Name"
git add debian/ && git commit -m 'add debian'
dch -i
#neutron (2:15.3.5-0ubuntu1~cloud0) bionic-train; urgency=medium
git commit -m "releasing package 2:15.3.5-0ubuntu1~cloud0" debian/changelogcp -r debian/patches/ /tmp/
rm -rf debian/patches/* && touch debian/patches/series
git add -u && debcommit -m 'delete series'git checkout 15.3.5 && git tag --no-sign 15.3.5 -m "15.3.5"
git checkout debian/15.3.5 && git tag --no-sign debian/15.3.5 -m "debian/15.3.5"git checkout 15.3.5
git archive --format=tar HEAD |gzip > ../neutron_15.3.5.orig.tar.gz
git checkout debian/15.3.5
mk-build-deps --install debian/control
rm -rf ../neutron_15.3.5-0ubuntu1~cloud0* && rm -rf .stestr
debuild -i -us -uc
有很多种方式有可以很轻易重现上面问题：
一种是直接运行’stestr run --subunit’，不清楚要不要运行’tox -e cover’（运行这句也hang在那）
stestr取代了testrepository - https://storyboard.openstack.org/#!/story/1666232
# https://github.com/mtreinish/stestr/
apt install python-tox python3-stestr python3-mock -y
#tox -e cover
stestr run --subunit
二是直接运行debian/rules build：
debian/rules clean && debian/rules build
三是运行’pkgos-dh_auto_test --no-py2’：
ls build/lib.linux-x86_64-2.7/neutron/
cat /usr/share/openstack-pkg-tools/pkgos.make
pkgos-dh_auto_test --no-py2vim debian/rules
ifeq (,$(findstring nocheck, $(DEB_BUILD_OPTIONS)))
override_dh_auto_test:pkgos-dh_auto_test --no-py2
endif
所以最后通过‘DEB_BUILD_OPTIONS=nocheck’（不再运行unit test)来bypass了上述错误
echo 'DEBOOTSTRAP_PROXY=http://t440p:3142/' >> ~/.mk-sbuild.rc
gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s' --git-debian-branch=debian/15.3.5 --git-ignore-branch --git-ignore-new --git-builder='DEB_BUILD_OPTIONS=nocheck sbuild -As -d bionic-train-amd64' -j8
20220505 - one sru example
sudo apt install pbuilder debootstrap devscripts -y
sudo apt install gnupg ubuntu-dev-tools bzr-builddeb apt-file debhelper openstack-pkg-tools -y
sudo apt install build-essential quilt dh-autoreconf fakeroot dpkg-dev python3-sphinx dh-make -y
sudo apt install python2 dh-python python-all python-setuptools -y
sudo apt install apt-cacher-ng -y
echo 'Acquire::http::Proxy "http://127.0.0.1:3142";' | sudo tee /etc/apt/apt.conf.d/01acng
# sometimes base.tgz can't be generated when using /etc/pbuilderrc
#sudo mv /etc/pbuilderrc /etc/pbuilderrc_bak
sudo pbuilder --create --distribution bionic --architecture amd64 --debootstrapopts --variant=buildd
sudo cp /var/cache/pbuilder/base.tgz /var/cache/pbuilder/bionic-base.tgz
# need to use '--basetgz' when using no pbuilderrc
sudo pbuilder update --basetgz /var/cache/pbuilder/bionic-base.tgz
#chroot to image to ajust what you want
sudo pbuilder login --save-after-login --basetgz /var/cache/pbuilder/bionic-base.tgzecho 'precedence ::ffff:0:0/96 100' >> /etc/gai.confapt install software-properties-common dirmngr vim ca-certificates ubuntu-cloud-keyring apt-transport-https -yvim /etc/apt/sources.list.d/cloudarchive-xena.listdeb http://ubuntu-cloud.archive.canonical.com/ubuntu boinic-updates/queens maindeb-src http://ubuntu-cloud.archive.canonical.com/ubuntu boinic-updates/queens main#apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 5EDB1B62EC4926EAsed -i -e "s/#deb-src/deb-src/" /etc/apt/sources.listapt update
# then press ctrl-d to save and exit chroot# but pdebuild needs pbuilderrc
cat <<EOF | sudo tee /etc/pbuilderrc
MIRRORSITE=http://archive.ubuntu.com/ubuntu/
DEBFULLNAME='Zhang Hua'
DEBEMAIL='xxx@xxx.com'
DISTRIBUTION='focal'
EXTRAPACKAGES=''
EXTRAPACKAGES+=' vim sudo bash-completion wget git build-essential apt-transport-https ca-certificates ubuntu-cloud-keyring'
# Cloud Archive
OS_RELEASE='xena'
if [ -n "\$OS_RELEASE" ]; thenBASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-\$OS_RELEASE-base.tgz"
elseBASETGZ="/var/cache/pbuilder/\$DISTRIBUTION-base.tgz"
fi
EOF
sudo pbuilder update --basetgz /var/cache/pbuilder/focal-xena-base.tgzcmadison neutron
rmadison neutron
mkdir -p /bak/work/sru/neutron/19.1.0 && cd /bak/work/sru/neutron/19.1.0
#UCA - https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/xena-staging
#UA - https://launchpad.net/ubuntu/+source/neutron
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/xena-staging/+sourcefiles/neutron/2:19.1.0-0ubuntu2~cloud0/neutron_19.1.0-0ubuntu2~cloud0.debian.tar.xz
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/xena-staging/+sourcefiles/neutron/2:19.1.0-0ubuntu2~cloud0/neutron_19.1.0-0ubuntu2~cloud0.dsc
wget https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/xena-staging/+sourcefiles/neutron/2:19.1.0-0ubuntu2~cloud0/neutron_19.1.0.orig.tar.gz
dpkg-source -x neutron_19.1.0-0ubuntu2~cloud0.dsc
cd neutron-19.1.0/
sudo apt install devscripts equivs -y
sudo pdebuild --debug
#sudo pdebuild --debbuildopts -sa #when hitting 'Unable to find xxx.orig.tar.gz in upload or distribution'
#DEB_BUILD_OPTIONS=nocheck debuild -us -uc
#sudo mk-build-deps --install debian/control
#dpkg-source: error: aborting due to unexpected upstream changes
#rm -rf debian/source/fomatcd /bak/openstack/neutron
#tox -epy38 neutron.tests.unit.agent.linux.test_dhcp.TestDeviceManager.test_setup_reserved_with_isolated_metadata_enable
git checkout 19.1.0
git cherry-pick -s -x 8a890ed29c681847d930bdf5403926c2e1450f9d
git format-patch -1 7932b9099b941e601de3245d0733547f85b7b047export DEBEMAIL="xxx@xx.com"
export DEBFULLNAME="Zhang Hua"
export QUILT_NO_DIFF_TIMESTAMPS=1
export QUILT_REFRESH_ARGS=-p ab --no-timestamps --no-index
export QUILT_PATCHES=debian/patches
export QUILT_NO_DIFF_INDEX=1
cp /bak/openstack/neutron/0001-Fix-list-of-DNS-extensions-supported-by-OVN.patch debian/patches/
echo '0001-Fix-list-of-DNS-extensions-supported-by-OVN.patch' >> debian/patches/series
quilt push -a
quilt refresh
dch -i
#dch -e
# for UCA, it's focal not focal-xena, Unable to find distroseries: focal-xena
neutron (2:19.1.0-0ubuntu2~cloud1) focal; urgency=medium* d/0001-Fix-list-of-DNS-extensions-supported-by-OVN.patch fix list of dns extensions supported by OVN (LP: #1947127)#sudo pdebuild
#sudo pdebuild --debbuildopts -sa # sometimes need to include source code for uca
# do not run unit test to avoid override_dh_auto_test error
sudo DEB_BUILD_OPTIONS="nocheck" pdebuild --debug
cd ..
debdiff neutron_19.1.0-0ubuntu2~cloud0.dsc neutron_19.1.0-0ubuntu2~cloud1.dsc > focal-xena.debdiff
debsign neutron_19.1.0-0ubuntu2~cloud0_source.changes -k$GPGKEY
debsign neutron_19.1.0-0ubuntu2~cloud1_source.changes -k$GPGKEY
#create test ppa with PPA dependency xena-staging - https://launchpad.net/~zhhuabj/+archive/ubuntu/focal-xena-testing
# use '--unchecked' to avoid: Error checking signature from xxx: SignatureVerifyError: 0
dput --unchecked -f ppa:zhhuabj/focal-xena-testing neutron_19.1.0-0ubuntu2~cloud1_source.changes#SRU - https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template , and remember to add '[SRU] prefix for the title
#for UA SRU: subscribe ubuntu-sponsors team, add sts-sru-needed and sts-sponsor tag
#for UCA SRU: add sts sts-sru-needed
#We add 'Ubuntu Cloud Archive' project ourselves for upstream stable/xena fix, but we don't have permission to add 'Xena' project
20220627更新
运行’sudo pdebuild’时报下列错误：
dh_clean: error: Compatibility levels before 7 are no longer supported (level 5 requested)
本来下面命令就能解决：
echo '10' > debian/compat
但是我画蛇添足了，运行了这个命令：sudo mk-build-deps --install debian/control ， 导致在源码目录添加了一个文件，这样报：binary file contents changed
Reference
[1] https://www.debian.org/doc/manuals/maint-guide/build.zh-cn.html
[2] https://wiki.softwareheritage.org/wiki/Debian_packaging