Merge branch 'RPi-Distro-master' into yunohost-buster
This commit is contained in:
Alexandre Aubin
commit
d6993788ca
|
|
@ -7,8 +7,8 @@ RUN apt-get -y update && \
|
|||
apt-get -y install --no-install-recommends \
|
||||
git vim parted \
|
||||
quilt coreutils qemu-user-static debootstrap zerofree zip dosfstools \
|
||||
bsdtar libcap2-bin rsync grep udev xz-utils curl xxd file kmod bc\
|
||||
binfmt-support ca-certificates \
|
||||
libarchive-tools libcap2-bin rsync grep udev xz-utils curl xxd file kmod bc\
|
||||
binfmt-support ca-certificates qemu-utils kpartx \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY . /pi-gen/
|
||||
|
|
|
|||
97
README.md
97
README.md
|
|
@ -14,7 +14,8 @@ To install the required dependencies for `pi-gen` you should run:
|
|||
|
||||
```bash
|
||||
apt-get install coreutils quilt parted qemu-user-static debootstrap zerofree zip \
|
||||
dosfstools bsdtar libcap2-bin grep rsync xz-utils file git curl bc
|
||||
dosfstools libarchive-tools libcap2-bin grep rsync xz-utils file git curl bc \
|
||||
qemu-utils kpartx
|
||||
```
|
||||
|
||||
The file `depends` contains a list of tools needed. The format of this
|
||||
|
|
@ -36,7 +37,30 @@ The following environment variables are supported:
|
|||
but you should use something else for a customized version. Export files
|
||||
in stages may add suffixes to `IMG_NAME`.
|
||||
|
||||
* `RELEASE` (Default: buster)
|
||||
* `USE_QCOW2`(Default: `1` )
|
||||
|
||||
Instead of using traditional way of building the rootfs of every stage in
|
||||
single subdirectories and copying over the previous one to the next one,
|
||||
qcow2 based virtual disks with backing images are used in every stage.
|
||||
This speeds up the build process and reduces overall space consumption
|
||||
significantly.
|
||||
|
||||
<u>Additional optional parameters regarding qcow2 build:</u>
|
||||
|
||||
* `BASE_QCOW2_SIZE` (Default: 12G)
|
||||
|
||||
Size of the virtual qcow2 disk.
|
||||
Note: it will not actually use that much of space at once but defines the
|
||||
maximum size of the virtual disk. If you change the build process by adding
|
||||
a lot of bigger packages or additional build stages, it can be necessary to
|
||||
increase the value because the virtual disk can run out of space like a normal
|
||||
hard drive would.
|
||||
|
||||
**CAUTION:** Although the qcow2 build mechanism will run fine inside Docker, it can happen
|
||||
that the network block device is not disconnected correctly after the Docker process has
|
||||
ended abnormally. In that case see [Disconnect an image if something went wrong](#Disconnect-an-image-if-something-went-wrong)
|
||||
|
||||
* `RELEASE` (Default: buster)
|
||||
|
||||
The release version to build images against. Valid values are jessie, stretch
|
||||
buster, bullseye, and testing.
|
||||
|
|
@ -240,6 +264,10 @@ fix is to ensure `binfmt-support` is installed on the host machine before
|
|||
starting the `./build-docker.sh` script (or using your own docker build
|
||||
solution).
|
||||
|
||||
### Passing arguments to Docker
|
||||
|
||||
When the docker image is run various required command line arguments are provided. For example the system mounts the `/dev` directory to the `/dev` directory within the docker container. If other arguments are required they may be specified in the PIGEN_DOCKER_OPTS environment variable. For example setting `PIGEN_DOCKER_OPTS="--add-host foo:192.168.0.23"` will add '192.168.0.23 foo' to the `/etc/hosts` file in the container. The `--name`
|
||||
and `--privileged` options are already set by the script and should not be redefined.
|
||||
|
||||
## Stage Anatomy
|
||||
|
||||
|
|
@ -332,6 +360,71 @@ follows:
|
|||
* Once you're happy with the image you can remove the SKIP_IMAGES files and
|
||||
export your image to test
|
||||
|
||||
# Regarding Qcow2 image building
|
||||
|
||||
### Get infos about the image in use
|
||||
|
||||
If you issue the two commands shown in the example below in a second command shell while a build
|
||||
is running you can find out, which network block device is currently being used and which qcow2 image
|
||||
is bound to it.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
root@build-machine:~/$ lsblk | grep nbd
|
||||
nbd1 43:32 0 10G 0 disk
|
||||
├─nbd1p1 43:33 0 10G 0 part
|
||||
└─nbd1p1 253:0 0 10G 0 part
|
||||
|
||||
root@build-machine:~/$ ps xa | grep qemu-nbd
|
||||
2392 pts/6 S+ 0:00 grep --color=auto qemu-nbd
|
||||
31294 ? Ssl 0:12 qemu-nbd --discard=unmap -c /dev/nbd1 image-stage4.qcow2
|
||||
```
|
||||
|
||||
Here you can see, that the qcow2 image `image-stage4.qcow2` is currently connected to `/dev/nbd1` with
|
||||
the associated partition map `/dev/mapper/nbd1p1`. Don't worry that `lsblk` shows two entries. It is totally fine, because the device map is accessible via `/dev/mapper/nbd1p1` and also via `/dev/dm-0`. This is all part of the device mapper functionality of the kernel. See `dmsetup` for further information.
|
||||
|
||||
### Mount a qcow2 image
|
||||
|
||||
If you want to examine the content of a a single stage, you can simply mount the qcow2 image found in the `WORK_DIR` directory with the tool `./imagetool.sh`.
|
||||
|
||||
See `./imagetool.sh -h` for further details on how to use it.
|
||||
|
||||
### Disconnect an image if something went wrong
|
||||
|
||||
It can happen, that your build stops in case of an error. Normally `./build.sh` should handle image disconnection appropriately, but in rare cases, especially during a Docker build, this may not work as expected. If that happens, starting a new build will fail and you may have to disconnect the image and/or device yourself.
|
||||
|
||||
A typical message indicating that there are some orphaned device mapper entries is this:
|
||||
|
||||
```
|
||||
Failed to set NBD socket
|
||||
Disconnect client, due to: Unexpected end-of-file before all bytes were read
|
||||
```
|
||||
|
||||
If that happens go through the following steps:
|
||||
|
||||
1. First, check if the image is somehow mounted to a directory entry and umount it as you would any other block device, like i.e. a hard disk or USB stick.
|
||||
|
||||
2. Second, to disconnect an image from `qemu-nbd`, the QEMU Disk Network Block Device Server, issue the following command (be sure to change the device name to the one actually used):
|
||||
|
||||
```bash
|
||||
sudo qemu-nbd -d /dev/nbd1
|
||||
```
|
||||
|
||||
Note: if you use Docker build, normally no active `qemu-nbd` process exists anymore as it will be terminated when the Docker container stops.
|
||||
|
||||
3. To disconnect a device partition map from the network block device, execute:
|
||||
|
||||
```bash
|
||||
sudo kpartx -d /dev/nbd1
|
||||
or
|
||||
sudo ./imagetool.sh --cleanup
|
||||
```
|
||||
|
||||
Note: The `imagetool.sh` command will cleanup any /dev/nbdX that is not connected to a running `qemu-nbd` daemon. Be careful if you use network block devices for other tasks utilizing NBDs on your build machine as well.
|
||||
|
||||
Now you should be able to start a new build without running into troubles again. Most of the time, especially when using Docker build, you will only need no. 3 to get everything up and running again.
|
||||
|
||||
# Troubleshooting
|
||||
|
||||
## `64 Bit Systems`
|
||||
|
|
|
|||
|
|
@ -1,4 +1,5 @@
|
|||
#!/bin/bash -eu
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
|
||||
|
||||
BUILD_OPTS="$*"
|
||||
|
|
@ -47,6 +48,7 @@ fi
|
|||
CONTAINER_NAME=${CONTAINER_NAME:-pigen_work}
|
||||
CONTINUE=${CONTINUE:-0}
|
||||
PRESERVE_CONTAINER=${PRESERVE_CONTAINER:-0}
|
||||
PIGEN_DOCKER_OPTS=${PIGEN_DOCKER_OPTS:-""}
|
||||
|
||||
if [ -z "${IMG_NAME}" ]; then
|
||||
echo "IMG_NAME not set in 'config'" 1>&2
|
||||
|
|
@ -87,6 +89,10 @@ ${DOCKER} build --build-arg BASE_IMAGE=${BASE_IMAGE} -t pi-gen "${DIR}"
|
|||
if [ "${CONTAINER_EXISTS}" != "" ]; then
|
||||
trap 'echo "got CTRL+C... please wait 5s" && ${DOCKER} stop -t 5 ${CONTAINER_NAME}_cont' SIGINT SIGTERM
|
||||
time ${DOCKER} run --rm --privileged \
|
||||
--cap-add=ALL \
|
||||
-v /dev:/dev \
|
||||
-v /lib/modules:/lib/modules \
|
||||
${PIGEN_DOCKER_OPTS} \
|
||||
--volume "${CONFIG_FILE}":/config:ro \
|
||||
-e "GIT_HASH=${GIT_HASH}" \
|
||||
--volumes-from="${CONTAINER_NAME}" --name "${CONTAINER_NAME}_cont" \
|
||||
|
|
@ -98,6 +104,10 @@ if [ "${CONTAINER_EXISTS}" != "" ]; then
|
|||
else
|
||||
trap 'echo "got CTRL+C... please wait 5s" && ${DOCKER} stop -t 5 ${CONTAINER_NAME}' SIGINT SIGTERM
|
||||
time ${DOCKER} run --name "${CONTAINER_NAME}" --privileged \
|
||||
--cap-add=ALL \
|
||||
-v /dev:/dev \
|
||||
-v /lib/modules:/lib/modules \
|
||||
${PIGEN_DOCKER_OPTS} \
|
||||
--volume "${CONFIG_FILE}":/config:ro \
|
||||
-e "GIT_HASH=${GIT_HASH}" \
|
||||
pi-gen \
|
||||
|
|
@ -106,6 +116,7 @@ else
|
|||
rsync -av work/*/build.log deploy/" &
|
||||
wait "$!"
|
||||
fi
|
||||
|
||||
echo "copying results from deploy/"
|
||||
${DOCKER} cp "${CONTAINER_NAME}":/pi-gen/deploy .
|
||||
ls -lah deploy
|
||||
|
|
|
|||
149
build.sh
149
build.sh
|
|
@ -1,4 +1,5 @@
|
|||
#!/bin/bash -e
|
||||
|
||||
# shellcheck disable=SC2119
|
||||
run_sub_stage()
|
||||
{
|
||||
|
|
@ -13,7 +14,7 @@ $(cat "${i}-debconf")
|
|||
SELEOF
|
||||
EOF
|
||||
|
||||
log "End ${SUB_STAGE_DIR}/${i}-debconf"
|
||||
log "End ${SUB_STAGE_DIR}/${i}-debconf"
|
||||
fi
|
||||
if [ -f "${i}-packages-nr" ]; then
|
||||
log "Begin ${SUB_STAGE_DIR}/${i}-packages-nr"
|
||||
|
|
@ -22,6 +23,11 @@ EOF
|
|||
on_chroot << EOF
|
||||
apt-get -o APT::Acquire::Retries=3 install --no-install-recommends -y $PACKAGES
|
||||
EOF
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
on_chroot << EOF
|
||||
apt-get clean
|
||||
EOF
|
||||
fi
|
||||
fi
|
||||
log "End ${SUB_STAGE_DIR}/${i}-packages-nr"
|
||||
fi
|
||||
|
|
@ -32,6 +38,11 @@ EOF
|
|||
on_chroot << EOF
|
||||
apt-get -o APT::Acquire::Retries=3 install -y $PACKAGES
|
||||
EOF
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
on_chroot << EOF
|
||||
apt-get clean
|
||||
EOF
|
||||
fi
|
||||
fi
|
||||
log "End ${SUB_STAGE_DIR}/${i}-packages"
|
||||
fi
|
||||
|
|
@ -82,17 +93,30 @@ EOF
|
|||
run_stage(){
|
||||
log "Begin ${STAGE_DIR}"
|
||||
STAGE="$(basename "${STAGE_DIR}")"
|
||||
|
||||
pushd "${STAGE_DIR}" > /dev/null
|
||||
unmount "${WORK_DIR}/${STAGE}"
|
||||
|
||||
STAGE_WORK_DIR="${WORK_DIR}/${STAGE}"
|
||||
ROOTFS_DIR="${STAGE_WORK_DIR}"/rootfs
|
||||
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
if [ ! -f SKIP ]; then
|
||||
load_qimage
|
||||
fi
|
||||
else
|
||||
# make sure we are not umounting during export-image stage
|
||||
if [ "${USE_QCOW2}" = "0" ] && [ "${NO_PRERUN_QCOW2}" = "0" ]; then
|
||||
unmount "${WORK_DIR}/${STAGE}"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ ! -f SKIP_IMAGES ]; then
|
||||
if [ -f "${STAGE_DIR}/EXPORT_IMAGE" ]; then
|
||||
EXPORT_DIRS="${EXPORT_DIRS} ${STAGE_DIR}"
|
||||
fi
|
||||
fi
|
||||
if [ ! -f SKIP ]; then
|
||||
if [ "${CLEAN}" = "1" ]; then
|
||||
if [ "${CLEAN}" = "1" ] && [ "${USE_QCOW2}" = "0" ] ; then
|
||||
if [ -d "${ROOTFS_DIR}" ]; then
|
||||
rm -rf "${ROOTFS_DIR}"
|
||||
fi
|
||||
|
|
@ -103,13 +127,21 @@ run_stage(){
|
|||
log "End ${STAGE_DIR}/prerun.sh"
|
||||
fi
|
||||
for SUB_STAGE_DIR in "${STAGE_DIR}"/*; do
|
||||
if [ -d "${SUB_STAGE_DIR}" ] &&
|
||||
[ ! -f "${SUB_STAGE_DIR}/SKIP" ]; then
|
||||
if [ -d "${SUB_STAGE_DIR}" ] && [ ! -f "${SUB_STAGE_DIR}/SKIP" ]; then
|
||||
run_sub_stage
|
||||
fi
|
||||
done
|
||||
fi
|
||||
unmount "${WORK_DIR}/${STAGE}"
|
||||
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
unload_qimage
|
||||
else
|
||||
# make sure we are not umounting during export-image stage
|
||||
if [ "${USE_QCOW2}" = "0" ] && [ "${NO_PRERUN_QCOW2}" = "0" ]; then
|
||||
unmount "${WORK_DIR}/${STAGE}"
|
||||
fi
|
||||
fi
|
||||
|
||||
PREV_STAGE="${STAGE}"
|
||||
PREV_STAGE_DIR="${STAGE_DIR}"
|
||||
PREV_ROOTFS_DIR="${ROOTFS_DIR}"
|
||||
|
|
@ -143,6 +175,15 @@ do
|
|||
esac
|
||||
done
|
||||
|
||||
term() {
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
log "Unloading image"
|
||||
unload_qimage
|
||||
fi
|
||||
}
|
||||
|
||||
trap term EXIT INT TERM
|
||||
|
||||
export PI_GEN=${PI_GEN:-pi-gen}
|
||||
export PI_GEN_REPO=${PI_GEN_REPO:-https://github.com/RPi-Distro/pi-gen}
|
||||
|
||||
|
|
@ -157,7 +198,7 @@ export IMG_FILENAME="${IMG_FILENAME:-"${IMG_DATE}-${IMG_NAME}"}"
|
|||
export ZIP_FILENAME="${ZIP_FILENAME:-"image_${IMG_DATE}-${IMG_NAME}"}"
|
||||
|
||||
export SCRIPT_DIR="${BASE_DIR}/scripts"
|
||||
export WORK_DIR="${WORK_DIR:-"${BASE_DIR}/work/${IMG_DATE}-${IMG_NAME}"}"
|
||||
export WORK_DIR="${WORK_DIR:-"${BASE_DIR}/work/${IMG_NAME}"}"
|
||||
export DEPLOY_DIR=${DEPLOY_DIR:-"${BASE_DIR}/deploy"}
|
||||
export DEPLOY_ZIP="${DEPLOY_ZIP:-1}"
|
||||
export LOG_FILE="${WORK_DIR}/build.log"
|
||||
|
|
@ -209,6 +250,18 @@ source "${SCRIPT_DIR}/common"
|
|||
# shellcheck source=scripts/dependencies_check
|
||||
source "${SCRIPT_DIR}/dependencies_check"
|
||||
|
||||
export NO_PRERUN_QCOW2="${NO_PRERUN_QCOW2:-1}"
|
||||
export USE_QCOW2="${USE_QCOW2:-1}"
|
||||
export BASE_QCOW2_SIZE=${BASE_QCOW2_SIZE:-12G}
|
||||
source "${SCRIPT_DIR}/qcow2_handling"
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
NO_PRERUN_QCOW2=1
|
||||
else
|
||||
NO_PRERUN_QCOW2=0
|
||||
fi
|
||||
|
||||
export NO_PRERUN_QCOW2="${NO_PRERUN_QCOW2:-1}"
|
||||
|
||||
dependencies_check "${BASE_DIR}/depends"
|
||||
|
||||
if [[ -n "${APT_PROXY}" ]] && ! curl --silent "${APT_PROXY}" >/dev/null ; then
|
||||
|
|
@ -242,22 +295,98 @@ for EXPORT_DIR in ${EXPORT_DIRS}; do
|
|||
# shellcheck source=/dev/null
|
||||
source "${EXPORT_DIR}/EXPORT_IMAGE"
|
||||
EXPORT_ROOTFS_DIR=${WORK_DIR}/$(basename "${EXPORT_DIR}")/rootfs
|
||||
run_stage
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
USE_QCOW2=0
|
||||
EXPORT_NAME="${IMG_FILENAME}${IMG_SUFFIX}"
|
||||
echo "------------------------------------------------------------------------"
|
||||
echo "Running export stage for ${EXPORT_NAME}"
|
||||
rm -f "${WORK_DIR}/export-image/${EXPORT_NAME}.img" || true
|
||||
rm -f "${WORK_DIR}/export-image/${EXPORT_NAME}.qcow2" || true
|
||||
rm -f "${WORK_DIR}/${EXPORT_NAME}.img" || true
|
||||
rm -f "${WORK_DIR}/${EXPORT_NAME}.qcow2" || true
|
||||
EXPORT_STAGE=$(basename "${EXPORT_DIR}")
|
||||
for s in $STAGE_LIST; do
|
||||
TMP_LIST=${TMP_LIST:+$TMP_LIST }$(basename "${s}")
|
||||
done
|
||||
FIRST_STAGE=${TMP_LIST%% *}
|
||||
FIRST_IMAGE="image-${FIRST_STAGE}.qcow2"
|
||||
|
||||
pushd "${WORK_DIR}" > /dev/null
|
||||
echo "Creating new base "${EXPORT_NAME}.qcow2" from ${FIRST_IMAGE}"
|
||||
cp "./${FIRST_IMAGE}" "${EXPORT_NAME}.qcow2"
|
||||
|
||||
ARR=($TMP_LIST)
|
||||
# rebase stage images to new export base
|
||||
for CURR_STAGE in "${ARR[@]}"; do
|
||||
if [ "${CURR_STAGE}" = "${FIRST_STAGE}" ]; then
|
||||
PREV_IMG="${EXPORT_NAME}"
|
||||
continue
|
||||
fi
|
||||
echo "Rebasing image-${CURR_STAGE}.qcow2 onto ${PREV_IMG}.qcow2"
|
||||
qemu-img rebase -f qcow2 -u -b ${PREV_IMG}.qcow2 image-${CURR_STAGE}.qcow2
|
||||
if [ "${CURR_STAGE}" = "${EXPORT_STAGE}" ]; then
|
||||
break
|
||||
fi
|
||||
PREV_IMG="image-${CURR_STAGE}"
|
||||
done
|
||||
|
||||
# commit current export stage into base export image
|
||||
echo "Committing image-${EXPORT_STAGE}.qcow2 to ${EXPORT_NAME}.qcow2"
|
||||
qemu-img commit -f qcow2 -p -b "${EXPORT_NAME}.qcow2" image-${EXPORT_STAGE}.qcow2
|
||||
|
||||
# rebase stage images back to original first stage for easy re-run
|
||||
for CURR_STAGE in "${ARR[@]}"; do
|
||||
if [ "${CURR_STAGE}" = "${FIRST_STAGE}" ]; then
|
||||
PREV_IMG="image-${CURR_STAGE}"
|
||||
continue
|
||||
fi
|
||||
echo "Rebasing back image-${CURR_STAGE}.qcow2 onto ${PREV_IMG}.qcow2"
|
||||
qemu-img rebase -f qcow2 -u -b ${PREV_IMG}.qcow2 image-${CURR_STAGE}.qcow2
|
||||
if [ "${CURR_STAGE}" = "${EXPORT_STAGE}" ]; then
|
||||
break
|
||||
fi
|
||||
PREV_IMG="image-${CURR_STAGE}"
|
||||
done
|
||||
popd > /dev/null
|
||||
|
||||
mkdir -p "${WORK_DIR}/export-image/rootfs"
|
||||
mv "${WORK_DIR}/${EXPORT_NAME}.qcow2" "${WORK_DIR}/export-image/"
|
||||
echo "Mounting image ${WORK_DIR}/export-image/${EXPORT_NAME}.qcow2 to rootfs ${WORK_DIR}/export-image/rootfs"
|
||||
mount_qimage "${WORK_DIR}/export-image/${EXPORT_NAME}.qcow2" "${WORK_DIR}/export-image/rootfs"
|
||||
|
||||
CLEAN=0
|
||||
run_stage
|
||||
CLEAN=1
|
||||
USE_QCOW2=1
|
||||
|
||||
else
|
||||
run_stage
|
||||
fi
|
||||
if [ "${USE_QEMU}" != "1" ]; then
|
||||
if [ -e "${EXPORT_DIR}/EXPORT_NOOBS" ]; then
|
||||
# shellcheck source=/dev/null
|
||||
source "${EXPORT_DIR}/EXPORT_NOOBS"
|
||||
STAGE_DIR="${BASE_DIR}/export-noobs"
|
||||
run_stage
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
USE_QCOW2=0
|
||||
run_stage
|
||||
USE_QCOW2=1
|
||||
else
|
||||
run_stage
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
if [ -x ${BASE_DIR}/postrun.sh ]; then
|
||||
if [ -x postrun.sh ]; then
|
||||
log "Begin postrun.sh"
|
||||
cd "${BASE_DIR}"
|
||||
./postrun.sh
|
||||
log "End postrun.sh"
|
||||
fi
|
||||
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
unload_qimage
|
||||
fi
|
||||
|
||||
log "End ${BASE_DIR}"
|
||||
|
|
|
|||
4
depends
4
depends
|
|
@ -7,7 +7,7 @@ zerofree
|
|||
zip
|
||||
mkdosfs:dosfstools
|
||||
capsh:libcap2-bin
|
||||
bsdtar
|
||||
bsdtar:libarchive-tools
|
||||
grep
|
||||
rsync
|
||||
xz:xz-utils
|
||||
|
|
@ -17,3 +17,5 @@ file
|
|||
git
|
||||
lsmod:kmod
|
||||
bc
|
||||
qemu-nbd:qemu-utils
|
||||
kpartx
|
||||
|
|
|
|||
|
|
@ -1,13 +1,18 @@
|
|||
#!/bin/bash -e
|
||||
|
||||
IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
if [ "${NO_PRERUN_QCOW2}" = "0" ]; then
|
||||
|
||||
IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
|
||||
IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
|
||||
BOOT_PARTUUID="${IMGID}-01"
|
||||
ROOT_PARTUUID="${IMGID}-02"
|
||||
IMGID="$(dd if="${IMG_FILE}" skip=440 bs=1 count=4 2>/dev/null | xxd -e | cut -f 2 -d' ')"
|
||||
|
||||
sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
|
||||
sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
|
||||
BOOT_PARTUUID="${IMGID}-01"
|
||||
ROOT_PARTUUID="${IMGID}-02"
|
||||
|
||||
sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
|
||||
sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/etc/fstab"
|
||||
|
||||
sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/cmdline.txt"
|
||||
|
||||
fi
|
||||
|
||||
sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${ROOTFS_DIR}/boot/cmdline.txt"
|
||||
|
|
|
|||
|
|
@ -76,25 +76,30 @@ cp "$ROOTFS_DIR/etc/rpi-issue" "$INFO_FILE"
|
|||
dpkg -l --root "$ROOTFS_DIR"
|
||||
} >> "$INFO_FILE"
|
||||
|
||||
ROOT_DEV="$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')"
|
||||
|
||||
unmount "${ROOTFS_DIR}"
|
||||
zerofree "${ROOT_DEV}"
|
||||
|
||||
unmount_image "${IMG_FILE}"
|
||||
|
||||
mkdir -p "${DEPLOY_DIR}"
|
||||
|
||||
rm -f "${DEPLOY_DIR}/${ZIP_FILENAME}${IMG_SUFFIX}.zip"
|
||||
rm -f "${DEPLOY_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
|
||||
mv "$INFO_FILE" "$DEPLOY_DIR/"
|
||||
|
||||
if [ "${USE_QCOW2}" = "0" ] && [ "${NO_PRERUN_QCOW2}" = "0" ]; then
|
||||
ROOT_DEV="$(mount | grep "${ROOTFS_DIR} " | cut -f1 -d' ')"
|
||||
|
||||
unmount "${ROOTFS_DIR}"
|
||||
zerofree "${ROOT_DEV}"
|
||||
|
||||
unmount_image "${IMG_FILE}"
|
||||
else
|
||||
unload_qimage
|
||||
make_bootable_image "${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.qcow2" "$IMG_FILE"
|
||||
fi
|
||||
|
||||
if [ "${DEPLOY_ZIP}" == "1" ]; then
|
||||
pushd "${STAGE_WORK_DIR}" > /dev/null
|
||||
zip "${DEPLOY_DIR}/${ZIP_FILENAME}${IMG_SUFFIX}.zip" \
|
||||
"$(basename "${IMG_FILE}")"
|
||||
popd > /dev/null
|
||||
else
|
||||
cp "$IMG_FILE" "$DEPLOY_DIR"
|
||||
mv "$IMG_FILE" "$DEPLOY_DIR/"
|
||||
fi
|
||||
|
||||
cp "$INFO_FILE" "$DEPLOY_DIR"
|
||||
|
|
|
|||
|
|
@ -1,85 +1,87 @@
|
|||
#!/bin/bash -e
|
||||
|
||||
IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
if [ "${NO_PRERUN_QCOW2}" = "0" ]; then
|
||||
IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
|
||||
unmount_image "${IMG_FILE}"
|
||||
unmount_image "${IMG_FILE}"
|
||||
|
||||
rm -f "${IMG_FILE}"
|
||||
rm -f "${IMG_FILE}"
|
||||
|
||||
rm -rf "${ROOTFS_DIR}"
|
||||
mkdir -p "${ROOTFS_DIR}"
|
||||
rm -rf "${ROOTFS_DIR}"
|
||||
mkdir -p "${ROOTFS_DIR}"
|
||||
|
||||
BOOT_SIZE="$((256 * 1024 * 1024))"
|
||||
ROOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --exclude boot --block-size=1 | cut -f 1)
|
||||
BOOT_SIZE="$((256 * 1024 * 1024))"
|
||||
ROOT_SIZE=$(du --apparent-size -s "${EXPORT_ROOTFS_DIR}" --exclude var/cache/apt/archives --exclude boot --block-size=1 | cut -f 1)
|
||||
|
||||
# All partition sizes and starts will be aligned to this size
|
||||
ALIGN="$((4 * 1024 * 1024))"
|
||||
# Add this much space to the calculated file size. This allows for
|
||||
# some overhead (since actual space usage is usually rounded up to the
|
||||
# filesystem block size) and gives some free space on the resulting
|
||||
# image.
|
||||
ROOT_MARGIN="$(echo "($ROOT_SIZE * 0.2 + 200 * 1024 * 1024) / 1" | bc)"
|
||||
# All partition sizes and starts will be aligned to this size
|
||||
ALIGN="$((4 * 1024 * 1024))"
|
||||
# Add this much space to the calculated file size. This allows for
|
||||
# some overhead (since actual space usage is usually rounded up to the
|
||||
# filesystem block size) and gives some free space on the resulting
|
||||
# image.
|
||||
ROOT_MARGIN="$(echo "($ROOT_SIZE * 0.2 + 200 * 1024 * 1024) / 1" | bc)"
|
||||
|
||||
BOOT_PART_START=$((ALIGN))
|
||||
BOOT_PART_SIZE=$(((BOOT_SIZE + ALIGN - 1) / ALIGN * ALIGN))
|
||||
ROOT_PART_START=$((BOOT_PART_START + BOOT_PART_SIZE))
|
||||
ROOT_PART_SIZE=$(((ROOT_SIZE + ROOT_MARGIN + ALIGN - 1) / ALIGN * ALIGN))
|
||||
IMG_SIZE=$((BOOT_PART_START + BOOT_PART_SIZE + ROOT_PART_SIZE))
|
||||
BOOT_PART_START=$((ALIGN))
|
||||
BOOT_PART_SIZE=$(((BOOT_SIZE + ALIGN - 1) / ALIGN * ALIGN))
|
||||
ROOT_PART_START=$((BOOT_PART_START + BOOT_PART_SIZE))
|
||||
ROOT_PART_SIZE=$(((ROOT_SIZE + ROOT_MARGIN + ALIGN - 1) / ALIGN * ALIGN))
|
||||
IMG_SIZE=$((BOOT_PART_START + BOOT_PART_SIZE + ROOT_PART_SIZE))
|
||||
|
||||
truncate -s "${IMG_SIZE}" "${IMG_FILE}"
|
||||
truncate -s "${IMG_SIZE}" "${IMG_FILE}"
|
||||
|
||||
parted --script "${IMG_FILE}" mklabel msdos
|
||||
parted --script "${IMG_FILE}" unit B mkpart primary fat32 "${BOOT_PART_START}" "$((BOOT_PART_START + BOOT_PART_SIZE - 1))"
|
||||
parted --script "${IMG_FILE}" unit B mkpart primary ext4 "${ROOT_PART_START}" "$((ROOT_PART_START + ROOT_PART_SIZE - 1))"
|
||||
parted --script "${IMG_FILE}" mklabel msdos
|
||||
parted --script "${IMG_FILE}" unit B mkpart primary fat32 "${BOOT_PART_START}" "$((BOOT_PART_START + BOOT_PART_SIZE - 1))"
|
||||
parted --script "${IMG_FILE}" unit B mkpart primary ext4 "${ROOT_PART_START}" "$((ROOT_PART_START + ROOT_PART_SIZE - 1))"
|
||||
|
||||
PARTED_OUT=$(parted -sm "${IMG_FILE}" unit b print)
|
||||
BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 2 | tr -d B)
|
||||
BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 4 | tr -d B)
|
||||
PARTED_OUT=$(parted -sm "${IMG_FILE}" unit b print)
|
||||
BOOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 2 | tr -d B)
|
||||
BOOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^1:' | cut -d':' -f 4 | tr -d B)
|
||||
|
||||
ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 2 | tr -d B)
|
||||
ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 4 | tr -d B)
|
||||
ROOT_OFFSET=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 2 | tr -d B)
|
||||
ROOT_LENGTH=$(echo "$PARTED_OUT" | grep -e '^2:' | cut -d':' -f 4 | tr -d B)
|
||||
|
||||
echo "Mounting BOOT_DEV..."
|
||||
cnt=0
|
||||
until BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}"); do
|
||||
if [ $cnt -lt 5 ]; then
|
||||
cnt=$((cnt + 1))
|
||||
echo "Error in losetup for BOOT_DEV. Retrying..."
|
||||
sleep 5
|
||||
else
|
||||
echo "ERROR: losetup for BOOT_DEV failed; exiting"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
echo "Mounting BOOT_DEV..."
|
||||
cnt=0
|
||||
until BOOT_DEV=$(losetup --show -f -o "${BOOT_OFFSET}" --sizelimit "${BOOT_LENGTH}" "${IMG_FILE}"); do
|
||||
if [ $cnt -lt 5 ]; then
|
||||
cnt=$((cnt + 1))
|
||||
echo "Error in losetup for BOOT_DEV. Retrying..."
|
||||
sleep 5
|
||||
else
|
||||
echo "ERROR: losetup for BOOT_DEV failed; exiting"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Mounting ROOT_DEV..."
|
||||
cnt=0
|
||||
until ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}"); do
|
||||
if [ $cnt -lt 5 ]; then
|
||||
cnt=$((cnt + 1))
|
||||
echo "Error in losetup for ROOT_DEV. Retrying..."
|
||||
sleep 5
|
||||
else
|
||||
echo "ERROR: losetup for ROOT_DEV failed; exiting"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
echo "Mounting ROOT_DEV..."
|
||||
cnt=0
|
||||
until ROOT_DEV=$(losetup --show -f -o "${ROOT_OFFSET}" --sizelimit "${ROOT_LENGTH}" "${IMG_FILE}"); do
|
||||
if [ $cnt -lt 5 ]; then
|
||||
cnt=$((cnt + 1))
|
||||
echo "Error in losetup for ROOT_DEV. Retrying..."
|
||||
sleep 5
|
||||
else
|
||||
echo "ERROR: losetup for ROOT_DEV failed; exiting"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
|
||||
echo "/: offset $ROOT_OFFSET, length $ROOT_LENGTH"
|
||||
echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
|
||||
echo "/: offset $ROOT_OFFSET, length $ROOT_LENGTH"
|
||||
|
||||
ROOT_FEATURES="^huge_file"
|
||||
for FEATURE in metadata_csum 64bit; do
|
||||
ROOT_FEATURES="^huge_file"
|
||||
for FEATURE in metadata_csum 64bit; do
|
||||
if grep -q "$FEATURE" /etc/mke2fs.conf; then
|
||||
ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
|
||||
ROOT_FEATURES="^$FEATURE,$ROOT_FEATURES"
|
||||
fi
|
||||
done
|
||||
mkdosfs -n boot -F 32 -v "$BOOT_DEV" > /dev/null
|
||||
mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
|
||||
done
|
||||
mkdosfs -n boot -F 32 -v "$BOOT_DEV" > /dev/null
|
||||
mkfs.ext4 -L rootfs -O "$ROOT_FEATURES" "$ROOT_DEV" > /dev/null
|
||||
|
||||
mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
|
||||
mkdir -p "${ROOTFS_DIR}/boot"
|
||||
mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot" -t vfat
|
||||
mount -v "$ROOT_DEV" "${ROOTFS_DIR}" -t ext4
|
||||
mkdir -p "${ROOTFS_DIR}/boot"
|
||||
mount -v "$BOOT_DEV" "${ROOTFS_DIR}/boot" -t vfat
|
||||
|
||||
rsync -aHAXx --exclude /var/cache/apt/archives --exclude /boot "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"
|
||||
rsync -rtx "${EXPORT_ROOTFS_DIR}/boot/" "${ROOTFS_DIR}/boot/"
|
||||
rsync -aHAXx --exclude /var/cache/apt/archives --exclude /boot "${EXPORT_ROOTFS_DIR}/" "${ROOTFS_DIR}/"
|
||||
rsync -rtx "${EXPORT_ROOTFS_DIR}/boot/" "${ROOTFS_DIR}/boot/"
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
#!/bin/bash -e
|
||||
|
||||
NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
|
||||
NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_NAME}${IMG_SUFFIX}"
|
||||
|
||||
install -v -m 744 files/partition_setup.sh "${NOOBS_DIR}/"
|
||||
install -v files/partitions.json "${NOOBS_DIR}/"
|
||||
|
|
@ -20,7 +20,7 @@ BOOT_SIZE="$(( BOOT_SIZE / 1024 / 1024 + 1))"
|
|||
ROOT_SIZE="$(( ROOT_SIZE / 1024 / 1024 + 1))"
|
||||
|
||||
BOOT_NOM="256"
|
||||
ROOT_NOM="$(( ROOT_SIZE + 400 ))"
|
||||
ROOT_NOM="$(echo "$ROOT_SIZE" | awk '{printf "%.0f", (($1 + 400) * 1.2) + 0.5 }')"
|
||||
|
||||
mv "${NOOBS_DIR}/OS.png" "${NOOBS_DIR}/${NOOBS_NAME// /_}.png"
|
||||
|
||||
|
|
@ -37,7 +37,12 @@ sed "${NOOBS_DIR}/os.json" -i -e "s|UNRELEASED|${IMG_DATE}|"
|
|||
sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_NAME|${NOOBS_NAME}|"
|
||||
sed "${NOOBS_DIR}/os.json" -i -e "s|NOOBS_DESCRIPTION|${NOOBS_DESCRIPTION}|"
|
||||
sed "${NOOBS_DIR}/os.json" -i -e "s|RELEASE|${RELEASE}|"
|
||||
sed "${NOOBS_DIR}/os.json" -i -e "s|KERNEL|$(cat "${STAGE_WORK_DIR}/kernel_version")|"
|
||||
|
||||
sed "${NOOBS_DIR}/release_notes.txt" -i -e "s|UNRELEASED|${IMG_DATE}|"
|
||||
|
||||
cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
mv "${NOOBS_DIR}" "${DEPLOY_DIR}/"
|
||||
else
|
||||
cp -a "${NOOBS_DIR}" "${DEPLOY_DIR}/"
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"description": "NOOBS_DESCRIPTION",
|
||||
"feature_level": 35120124,
|
||||
"kernel": "4.19",
|
||||
"kernel": "KERNEL",
|
||||
"name": "NOOBS_NAME",
|
||||
"password": "raspberry",
|
||||
"release_date": "UNRELEASED",
|
||||
|
|
|
|||
|
|
@ -1,4 +1,57 @@
|
|||
UNRELEASED:
|
||||
* Chromium upgraded to version 88.0.4324.187
|
||||
* NuScratch upgraded to version 20210507
|
||||
* Node-RED upgraded to version 1.3.4
|
||||
* pigpio upgraded to version 1.79
|
||||
* Thonny upgraded to version 3.3.6
|
||||
* Icelandic and Italian translations updated for several packages
|
||||
* piclone: Remove hiding of application in other desktops
|
||||
* agnostics: Remove hiding of app in other desktops
|
||||
* rp-bookshelf:
|
||||
- Remove hiding of app in other desktops
|
||||
- GTK+3 version
|
||||
* lxplug-bluetooth:
|
||||
- Fix some memory leaks
|
||||
- Add authorisation dialog required by some BT-LE pairings
|
||||
* alsa-utils: Add custom init files for bcm2835 on Raspberry Pi to set volume correctly
|
||||
* rp-prefapps: Remove hiding of app in other desktops
|
||||
* OpenSSH and OpenSSL speed improvements
|
||||
* Install gpiozero in lite images
|
||||
* Raspberry Pi firmware 518ee7c871aaa9aaa88116953d57e73787ee6e43
|
||||
* Linux kernel 5.10.17
|
||||
2021-03-04:
|
||||
* Thonny upgraded to version 3.3.5
|
||||
* SD Card Copier made compatible with NVMe devices; now built against GTK+3 toolkit
|
||||
* Composite video options removed from Raspberry Pi 4 in Raspberry Pi Configuration
|
||||
* Boot order options in raspi-config adjusted for more flexibility
|
||||
* Recommended Software now built against GTK+3 toolkit
|
||||
* Fix for crash in volume plugin when using keyboard could push value out of range
|
||||
* Fix for focus changing between windows in file manager when using keyboard to navigate directory view
|
||||
* Fix for Raspberry Pi 400 keyboard country not being read correctly in startup wizard
|
||||
* Armenian and Japanese translations added to several packages
|
||||
* Automatically load aes-neon-bs on ARM64 to speed up OpenSSL
|
||||
* Raspberry Pi firmware fcf8d2f7639ad8d0330db9c8db9b71bd33eaaa28
|
||||
* Linux kernel 5.10.17
|
||||
2021-01-11:
|
||||
* Chromium version 86.0.4240.197 included
|
||||
* Screen reader support enabled in Chromium
|
||||
* Adobe have end-of-lifed Flash Player, so it has been removed
|
||||
* Scratch 2 required Flash, so it has been removed
|
||||
* Added Epson printer drivers
|
||||
* Added timeout to hide messages from USB device monitor after 5 seconds
|
||||
* Bug fix - PulseAudio output was in mono
|
||||
* Bug fix - brief audio interruptions at start of playback in VLC
|
||||
* Bug fix - old ALSA output settings being used instead of PulseAudio settings by some applications
|
||||
* Bug fix - crash in PulseAudio volume controller when used on multichannel devices
|
||||
* Bug fix - battery monitor failing to load on x86 platforms
|
||||
* Bug fix - setting of password in startup wizard failed if language was changed
|
||||
* Bug fix - Chromium video playback lockup on small number of devices
|
||||
* Bug fix - Chromium Google Maps 3D view artefacts
|
||||
* Slovak, Italian and Norwegian translations updated
|
||||
* Added Epson printer drivers
|
||||
* Raspberry Pi firmware 70f1581eec2c036b7e9309f1af41c651fb125447
|
||||
* Linux kernel 5.4.83
|
||||
2020-12-02:
|
||||
* PulseAudio now included and running by default
|
||||
* Bluealsa Bluetooth interface removed - Bluetooth audio is now handled by PulseAudio
|
||||
* LXPanel volume control plugin replaced with PulseAudio version
|
||||
|
|
|
|||
|
|
@ -1,11 +1,15 @@
|
|||
#!/bin/bash -e
|
||||
|
||||
IMG_FILE="${STAGE_WORK_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_DATE}-${IMG_NAME}${IMG_SUFFIX}"
|
||||
unmount_image "${IMG_FILE}"
|
||||
|
||||
NOOBS_DIR="${STAGE_WORK_DIR}/${IMG_NAME}${IMG_SUFFIX}"
|
||||
mkdir -p "${STAGE_WORK_DIR}"
|
||||
cp "${WORK_DIR}/export-image/${IMG_FILENAME}${IMG_SUFFIX}.img" "${STAGE_WORK_DIR}/"
|
||||
|
||||
if [ "${DEPLOY_ZIP}" == "1" ]; then
|
||||
IMG_FILE="${WORK_DIR}/export-image/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
else
|
||||
IMG_FILE="${DEPLOY_DIR}/${IMG_FILENAME}${IMG_SUFFIX}.img"
|
||||
fi
|
||||
|
||||
unmount_image "${IMG_FILE}"
|
||||
|
||||
rm -rf "${NOOBS_DIR}"
|
||||
|
||||
|
|
@ -53,8 +57,15 @@ mount "$BOOT_DEV" "${STAGE_WORK_DIR}/rootfs/boot"
|
|||
|
||||
ln -sv "/lib/systemd/system/apply_noobs_os_config.service" "$ROOTFS_DIR/etc/systemd/system/multi-user.target.wants/apply_noobs_os_config.service"
|
||||
|
||||
KERNEL_VER="$(zgrep -oPm 1 "Linux version \K(.*)$" "${STAGE_WORK_DIR}/rootfs/usr/share/doc/raspberrypi-kernel/changelog.Debian.gz" | cut -f-2 -d.)"
|
||||
echo "$KERNEL_VER" > "${STAGE_WORK_DIR}/kernel_version"
|
||||
|
||||
bsdtar --numeric-owner --format gnutar -C "${STAGE_WORK_DIR}/rootfs/boot" -cpf - . | xz -T0 > "${NOOBS_DIR}/boot.tar.xz"
|
||||
umount "${STAGE_WORK_DIR}/rootfs/boot"
|
||||
bsdtar --numeric-owner --format gnutar -C "${STAGE_WORK_DIR}/rootfs" --one-file-system -cpf - . | xz -T0 > "${NOOBS_DIR}/root.tar.xz"
|
||||
|
||||
if [ "${USE_QCOW2}" = "1" ]; then
|
||||
rm "$ROOTFS_DIR/etc/systemd/system/multi-user.target.wants/apply_noobs_os_config.service"
|
||||
fi
|
||||
|
||||
unmount_image "${IMG_FILE}"
|
||||
|
|
|
|||
114
imagetool.sh
Executable file
114
imagetool.sh
Executable file
|
|
@ -0,0 +1,114 @@
|
|||
#!/bin/bash
|
||||
|
||||
if [ "$(id -u)" != "0" ]; then
|
||||
echo "Please run as root" 1>&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
progname=$(basename $0)
|
||||
|
||||
function usage()
|
||||
{
|
||||
cat << HEREDOC
|
||||
|
||||
Usage:
|
||||
Mount Image : $progname [--mount] [--image-name <path to qcow2 image>] [--mount-point <mount point>]
|
||||
Umount Image: $progname [--umount] [--mount-point <mount point>]
|
||||
Cleanup NBD : $progname [--cleanup]
|
||||
|
||||
arguments:
|
||||
-h, --help show this help message and exit
|
||||
-c, --cleanup cleanup orphaned device mappings
|
||||
-m, --mount mount image
|
||||
-u, --umount umount image
|
||||
-i, --image-name path to qcow2 image
|
||||
-p, --mount-point mount point for image
|
||||
|
||||
This tool will use /dev/nbd1 as default for mounting an image. If you want to use another device, execute like this:
|
||||
NBD_DEV=/dev/nbd2 ./$progname --mount --image <your image> --mount-point <your path>
|
||||
|
||||
HEREDOC
|
||||
}
|
||||
|
||||
MOUNT=0
|
||||
UMOUNT=0
|
||||
IMAGE=""
|
||||
MOUNTPOINT=""
|
||||
|
||||
nbd_cleanup() {
|
||||
DEVS="$(lsblk | grep nbd | grep disk | cut -d" " -f1)"
|
||||
if [ ! -z "${DEVS}" ]; then
|
||||
for d in $DEVS; do
|
||||
if [ ! -z "${d}" ]; then
|
||||
QDEV="$(ps xa | grep $d | grep -v grep)"
|
||||
if [ -z "${QDEV}" ]; then
|
||||
kpartx -d /dev/$d && echo "Unconnected device map removed: /dev/$d"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
}
|
||||
|
||||
# As long as there is at least one more argument, keep looping
|
||||
while [[ $# -gt 0 ]]; do
|
||||
key="$1"
|
||||
case "$key" in
|
||||
-h|--help)
|
||||
usage
|
||||
exit
|
||||
;;
|
||||
-c|--cleanup)
|
||||
nbd_cleanup
|
||||
;;
|
||||
-m|--mount)
|
||||
MOUNT=1
|
||||
;;
|
||||
-u|--umount)
|
||||
UMOUNT=1
|
||||
;;
|
||||
-i|--image-name)
|
||||
shift
|
||||
IMAGE="$1"
|
||||
;;
|
||||
-p|--mount-point)
|
||||
shift
|
||||
MOUNTPOINT="$1"
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option '$key'"
|
||||
usage
|
||||
exit
|
||||
;;
|
||||
esac
|
||||
# Shift after checking all the cases to get the next option
|
||||
shift
|
||||
done
|
||||
|
||||
if [ "${MOUNT}" = "1" ] && [ "${UMOUNT}" = "1" ]; then
|
||||
usage
|
||||
echo "Concurrent mount options not possible."
|
||||
exit
|
||||
fi
|
||||
|
||||
if [ "${MOUNT}" = "1" ] && ([ -z "${IMAGE}" ] || [ -z "${MOUNTPOINT}" ]); then
|
||||
usage
|
||||
echo "Can not mount image. Image path and/or mount point missing."
|
||||
exit
|
||||
fi
|
||||
|
||||
if [ "${UMOUNT}" = "1" ] && [ -z "${MOUNTPOINT}" ]; then
|
||||
usage
|
||||
echo "Can not umount. Mount point parameter missing."
|
||||
exit
|
||||
fi
|
||||
|
||||
export NBD_DEV="${NBD_DEV:-/dev/nbd1}"
|
||||
export MAP_BOOT_DEV=/dev/mapper/nbd1p1
|
||||
export MAP_ROOT_DEV=/dev/mapper/nbd1p2
|
||||
source scripts/qcow2_handling
|
||||
|
||||
if [ "${MOUNT}" = "1" ]; then
|
||||
mount_qimage "${IMAGE}" "${MOUNTPOINT}"
|
||||
elif [ "${UMOUNT}" = "1" ]; then
|
||||
umount_qimage "${MOUNTPOINT}"
|
||||
fi
|
||||
|
|
@ -21,8 +21,10 @@ bootstrap(){
|
|||
|
||||
setarch linux32 capsh --drop=cap_setfcap -- -c "'${BOOTSTRAP_CMD}' $BOOTSTRAP_STR" || true
|
||||
|
||||
if [ -d "$2/debootstrap" ]; then
|
||||
rmdir "$2/debootstrap"
|
||||
if [ -d "$2/debootstrap" ] && ! rmdir "$2/debootstrap"; then
|
||||
cp "$2/debootstrap/debootstrap.log" "${STAGE_WORK_DIR}"
|
||||
log "bootstrap failed: please check ${STAGE_WORK_DIR}/debootstrap.log"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
export -f bootstrap
|
||||
|
|
|
|||
|
|
@ -28,11 +28,26 @@ dependencies_check()
|
|||
false
|
||||
fi
|
||||
|
||||
# If we're building on a native arm platform, we don't need to check for
|
||||
# binfmt_misc or require it to be loaded.
|
||||
|
||||
if ! grep -q "/proc/sys/fs/binfmt_misc" /proc/mounts; then
|
||||
echo "Module binfmt_misc not loaded in host"
|
||||
echo "Please run:"
|
||||
echo " sudo modprobe binfmt_misc"
|
||||
exit 1
|
||||
binfmt_misc_required=1
|
||||
|
||||
case $(uname -m) in
|
||||
aarch64)
|
||||
binfmt_misc_required=0
|
||||
;;
|
||||
arm*)
|
||||
binfmt_misc_required=0
|
||||
;;
|
||||
esac
|
||||
|
||||
if [[ "${binfmt_misc_required}" == "1" ]]; then
|
||||
if ! grep -q "/proc/sys/fs/binfmt_misc" /proc/mounts; then
|
||||
echo "Module binfmt_misc not loaded in host"
|
||||
echo "Please run:"
|
||||
echo " sudo modprobe binfmt_misc"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
|
|
|||
256
scripts/qcow2_handling
Normal file
256
scripts/qcow2_handling
Normal file
|
|
@ -0,0 +1,256 @@
|
|||
#!/bin/bash
|
||||
|
||||
# QCOW2 Routines
|
||||
|
||||
export CURRENT_IMAGE
|
||||
export CURRENT_MOUNTPOINT
|
||||
|
||||
export NBD_DEV
|
||||
export MAP_BOOT_DEV
|
||||
export MAP_ROOT_DEV
|
||||
|
||||
# set in build.sh
|
||||
# should be fairly enough for the beginning
|
||||
# overwrite here by uncommenting following lines
|
||||
# BASE_QCOW2_SIZE=12G
|
||||
|
||||
# find and initialize free block device nodes
|
||||
init_nbd() {
|
||||
modprobe nbd max_part=16
|
||||
if [ -z "${NBD_DEV}" ]; then
|
||||
for x in /sys/class/block/nbd* ; do
|
||||
S=`cat $x/size`
|
||||
if [ "$S" == "0" ] ; then
|
||||
NBD_DEV=/dev/$(basename $x)
|
||||
MAP_BOOT_DEV=/dev/mapper/$(basename $x)p1
|
||||
MAP_ROOT_DEV=/dev/mapper/$(basename $x)p2
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
}
|
||||
export -f init_nbd
|
||||
|
||||
# connect image to block device
|
||||
connect_blkdev() {
|
||||
init_nbd
|
||||
qemu-nbd --discard=unmap -c $NBD_DEV "$1"
|
||||
sync
|
||||
kpartx -as $NBD_DEV
|
||||
sync
|
||||
CURRENT_IMAGE="$1"
|
||||
}
|
||||
export -f connect_blkdev
|
||||
|
||||
# disconnect image from block device
|
||||
disconnect_blkdev() {
|
||||
kpartx -d $NBD_DEV
|
||||
qemu-nbd -d $NBD_DEV
|
||||
NBD_DEV=
|
||||
MAP_BOOT_DEV=
|
||||
MAP_ROOT_DEV=
|
||||
CURRENT_IMAGE=
|
||||
}
|
||||
export -f disconnect_blkdev
|
||||
|
||||
# mount qcow2 image: mount_image <image file> <mountpoint>
|
||||
mount_qimage() {
|
||||
connect_blkdev "$1"
|
||||
mount -v -t ext4 $MAP_ROOT_DEV "$2"
|
||||
mkdir -p "${ROOTFS_DIR}/boot"
|
||||
mount -v -t vfat $MAP_BOOT_DEV "$2/boot"
|
||||
CURRENT_MOUNTPOINT="$2"
|
||||
}
|
||||
export -f mount_qimage
|
||||
|
||||
# umount qcow2 image: umount_image <current mountpoint>
|
||||
umount_qimage() {
|
||||
sync
|
||||
#umount "$1/boot"
|
||||
while mount | grep -q "$1"; do
|
||||
local LOCS
|
||||
LOCS=$(mount | grep "$1" | cut -f 3 -d ' ' | sort -r)
|
||||
for loc in $LOCS; do
|
||||
echo "$loc"
|
||||
while mountpoint -q "$loc" && ! umount "$loc"; do
|
||||
sleep 0.1
|
||||
done
|
||||
done
|
||||
done
|
||||
CURRENT_MOUNTPOINT=
|
||||
disconnect_blkdev
|
||||
}
|
||||
export -f umount_qimage
|
||||
|
||||
# create base image / backing image / mount image
|
||||
load_qimage() {
|
||||
if [ -z "${CURRENT_MOUNTPOINT}" ]; then
|
||||
if [ ! -d "${ROOTFS_DIR}" ]; then
|
||||
mkdir -p "${ROOTFS_DIR}";
|
||||
fi
|
||||
|
||||
if [ "${CLEAN}" = "1" ] && [ -f "${WORK_DIR}/image-${STAGE}.qcow2" ]; then
|
||||
rm -f "${WORK_DIR}/image-${STAGE}.qcow2";
|
||||
fi
|
||||
|
||||
if [ ! -f "${WORK_DIR}/image-${STAGE}.qcow2" ]; then
|
||||
pushd ${WORK_DIR} > /dev/null
|
||||
init_nbd
|
||||
if [ -z "${PREV_STAGE}" ]; then
|
||||
echo "Creating base image: image-${STAGE}.qcow2"
|
||||
# -o preallocation=falloc
|
||||
qemu-img create -f qcow2 image-${STAGE}.qcow2 $BASE_QCOW2_SIZE
|
||||
sync
|
||||
qemu-nbd --discard=unmap -c $NBD_DEV image-${STAGE}.qcow2
|
||||
sync
|
||||
sfdisk $NBD_DEV << EOF
|
||||
4MiB,250MiB,c,*
|
||||
254MiB,,83;
|
||||
EOF
|
||||
sync
|
||||
kpartx -as $NBD_DEV
|
||||
mkdosfs -n boot -F 32 -v $MAP_BOOT_DEV
|
||||
mkfs.ext4 -L rootfs -O "^huge_file,^metadata_csum,^64bit" $MAP_ROOT_DEV
|
||||
sync
|
||||
else
|
||||
if [ ! -f "${WORK_DIR}/image-${PREV_STAGE}.qcow2" ]; then
|
||||
exit 1;
|
||||
fi
|
||||
echo "Creating backing image: image-${STAGE}.qcow2 <- ${WORK_DIR}/image-${PREV_STAGE}.qcow2"
|
||||
qemu-img create -f qcow2 \
|
||||
-o backing_file=${WORK_DIR}/image-${PREV_STAGE}.qcow2 \
|
||||
${WORK_DIR}/image-${STAGE}.qcow2
|
||||
sync
|
||||
qemu-nbd --discard=unmap -c $NBD_DEV image-${STAGE}.qcow2
|
||||
sync
|
||||
kpartx -as $NBD_DEV
|
||||
fi
|
||||
|
||||
mount -v -t ext4 $MAP_ROOT_DEV "${ROOTFS_DIR}"
|
||||
mkdir -p "${ROOTFS_DIR}/boot"
|
||||
mount -v -t vfat $MAP_BOOT_DEV "${ROOTFS_DIR}/boot"
|
||||
CURRENT_IMAGE=${WORK_DIR}/image-${STAGE}.qcow2
|
||||
CURRENT_MOUNTPOINT=${ROOTFS_DIR}
|
||||
popd > /dev/null
|
||||
else
|
||||
mount_qimage "${WORK_DIR}/image-${STAGE}.qcow2" "${ROOTFS_DIR}"
|
||||
fi
|
||||
echo "Current image in use: ${CURRENT_IMAGE} (MP: ${CURRENT_MOUNTPOINT})"
|
||||
fi
|
||||
}
|
||||
export -f load_qimage
|
||||
|
||||
# umount current image and refresh mount point env var
|
||||
unload_qimage() {
|
||||
if [ ! -z "${CURRENT_MOUNTPOINT}" ]; then
|
||||
fstrim -v "${CURRENT_MOUNTPOINT}" || true
|
||||
umount_qimage "${CURRENT_MOUNTPOINT}"
|
||||
fi
|
||||
}
|
||||
export -f unload_qimage
|
||||
|
||||
# based on: https://github.com/SirLagz/RaspberryPi-ImgAutoSizer
|
||||
# helper function for make_bootable_image, do not call directly
|
||||
function resize_qcow2() {
|
||||
if [ -z "$CALL_FROM_MBI" ]; then
|
||||
echo "resize_qcow2: cannot be called directly, use make_bootable_image instead"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# ROOT_MARGIN=$((800*1024*1024))
|
||||
ROOT_MARGIN=$((1*1024*1024))
|
||||
PARTED_OUT=`parted -s -m "$NBD_DEV" unit B print`
|
||||
PART_NO=`echo "$PARTED_OUT" | grep ext4 | awk -F: ' { print $1 } '`
|
||||
PART_START=`echo "$PARTED_OUT" | grep ext4 | awk -F: ' { print substr($2,1,length($2)-1) } '`
|
||||
|
||||
e2fsck -y -f $MAP_ROOT_DEV || true
|
||||
|
||||
DATA_SIZE=`resize2fs -P $MAP_ROOT_DEV | awk -F': ' ' { print $2 } '`
|
||||
BLOCK_SIZE=$(dumpe2fs -h $MAP_ROOT_DEV | grep 'Block size' | awk -F': ' ' { print $2 }')
|
||||
BLOCK_SIZE=${BLOCK_SIZE// /}
|
||||
|
||||
let DATA_SIZE=$DATA_SIZE+$ROOT_MARGIN/$BLOCK_SIZE
|
||||
resize2fs -p $MAP_ROOT_DEV $DATA_SIZE
|
||||
sleep 1
|
||||
|
||||
let PART_NEW_SIZE=$DATA_SIZE*$BLOCK_SIZE
|
||||
let PART_NEW_END=$PART_START+$PART_NEW_SIZE
|
||||
ACT1=`parted -s "$NBD_DEV" rm 2`
|
||||
ACT2=`parted -s "$NBD_DEV" unit B mkpart primary $PART_START $PART_NEW_END`
|
||||
NEW_IMG_SIZE=`parted -s -m "$NBD_DEV" unit B print free | tail -1 | awk -F: ' { print substr($2,1,length($2)-1) } '`
|
||||
}
|
||||
export -f resize_qcow2
|
||||
|
||||
# create raw img from qcow2: make_bootable_image <in.qcow2> <out.img>
|
||||
function make_bootable_image() {
|
||||
|
||||
EXPORT_QCOW2="$1"
|
||||
EXPORT_IMAGE="$2"
|
||||
|
||||
echo "Connect block device to source qcow2"
|
||||
connect_blkdev "${EXPORT_QCOW2}"
|
||||
|
||||
echo "Resize fs and partition"
|
||||
CALL_FROM_MBI=1
|
||||
resize_qcow2
|
||||
sync
|
||||
CALL_FROM_MBI=
|
||||
|
||||
echo "Disconnect block device"
|
||||
disconnect_blkdev
|
||||
|
||||
if [ -z "$NEW_IMG_SIZE" ]; then
|
||||
echo "NEW_IMG_SIZE could not be calculated, cannot process image. Exit."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Shrinking qcow2 image"
|
||||
qemu-img resize --shrink "${EXPORT_QCOW2}" $NEW_IMG_SIZE
|
||||
sync
|
||||
|
||||
echo "Convert qcow2 to raw image"
|
||||
qemu-img convert -f qcow2 -O raw "${EXPORT_QCOW2}" "${EXPORT_IMAGE}"
|
||||
sync
|
||||
|
||||
echo "Get PARTUUIDs from image"
|
||||
IMGID="$(blkid -o value -s PTUUID "${EXPORT_IMAGE}")"
|
||||
|
||||
BOOT_PARTUUID="${IMGID}-01"
|
||||
echo "Boot: $BOOT_PARTUUID"
|
||||
ROOT_PARTUUID="${IMGID}-02"
|
||||
echo "Root: $ROOT_PARTUUID"
|
||||
|
||||
echo "Mount image"
|
||||
MOUNTROOT=${WORK_DIR}/tmpimage
|
||||
mkdir -p $MOUNTROOT
|
||||
|
||||
MOUNTPT=$MOUNTROOT
|
||||
PARTITION=2
|
||||
mount "${EXPORT_IMAGE}" "$MOUNTPT" -o loop,offset=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*start=[ ]*//' | sed 's/,.*//'` * 512 ],sizelimit=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*size=[ ]*//' | sed 's/,.*//'` * 512 ] || exit 1
|
||||
|
||||
MOUNTPT=$MOUNTROOT/boot
|
||||
PARTITION=1
|
||||
mount "${EXPORT_IMAGE}" "$MOUNTPT" -o loop,offset=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*start=[ ]*//' | sed 's/,.*//'` * 512 ],sizelimit=$[ `/sbin/sfdisk -d "${EXPORT_IMAGE}" | grep "start=" | head -n $PARTITION | tail -n1 | sed 's/.*size=[ ]*//' | sed 's/,.*//'` * 512 ] || exit 1
|
||||
|
||||
if [ ! -d "${MOUNTROOT}/root" ]; then
|
||||
echo "Image damaged or not mounted. Exit."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Setup PARTUUIDs"
|
||||
if [ ! -z "$BOOT_PARTUUID" ] && [ ! -z "$ROOT_PARTUUID" ]; then
|
||||
echo "Set UUIDs to make it bootable"
|
||||
sed -i "s/BOOTDEV/PARTUUID=${BOOT_PARTUUID}/" "${MOUNTROOT}/etc/fstab"
|
||||
sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${MOUNTROOT}/etc/fstab"
|
||||
sed -i "s/ROOTDEV/PARTUUID=${ROOT_PARTUUID}/" "${MOUNTROOT}/boot/cmdline.txt"
|
||||
fi
|
||||
|
||||
echo "Umount image"
|
||||
sync
|
||||
umount "${MOUNTROOT}/boot" || exit 1
|
||||
umount "${MOUNTROOT}" || exit 1
|
||||
|
||||
echo "Remove qcow2 export image"
|
||||
rm -f "${EXPORT_QCOW2}"
|
||||
}
|
||||
export -f make_bootable_image
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
#!/bin/bash -e
|
||||
|
||||
if [ ! -d "${ROOTFS_DIR}" ]; then
|
||||
if [ ! -d "${ROOTFS_DIR}" ] || [ "${USE_QCOW2}" = "1" ]; then
|
||||
bootstrap ${RELEASE} "${ROOTFS_DIR}" http://raspbian.raspberrypi.org/raspbian/
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
|
||||
console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 fsck.repair=yes rootwait
|
||||
|
|
|
|||
|
|
@ -2,6 +2,8 @@ ssh less fbset sudo psmisc strace ed ncdu crda
|
|||
console-setup keyboard-configuration debconf-utils parted unzip
|
||||
build-essential manpages-dev python bash-completion gdb pkg-config
|
||||
python-rpi.gpio v4l-utils
|
||||
python-gpiozero
|
||||
python3-gpiozero
|
||||
avahi-daemon
|
||||
lua5.1
|
||||
luajit
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
--- stage2.orig/rootfs/boot/cmdline.txt
|
||||
+++ stage2/rootfs/boot/cmdline.txt
|
||||
@@ -1 +1 @@
|
||||
-console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait
|
||||
+console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh
|
||||
-console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 fsck.repair=yes rootwait
|
||||
+console=serial0,115200 console=tty1 root=ROOTDEV rootfstype=ext4 fsck.repair=yes rootwait quiet init=/usr/lib/raspi-config/init_resize.sh
|
||||
|
|
|
|||
Loading…
Reference in a new issue