Optimise your ROS snap – Part 3

Welcome to Part 3 of our “optimise your ROS snap” blog series. Make sure to check Part 2. This third part is going to present safe optimisations consisting of removing duplicates and unnecessary files. We will present three different methods as well as their cumulative benefits for the performance of ROS snaps.

When snaps are bundling all their dependencies, unnecessary files are bundled too. Most dependencies are coming from APT packages. These packages are also bundling files that are not useful for our snap at runtime (documentations, build tools, etc.). All these unnecessary files count in the size of our snap and affect the “cold start”. Reducing the number of files in our snap would make it smaller and potentially start faster.

Remove unnecessary files for deployment

Debian packages are not only used for deployment. Some can be used for development or simply bundle files that are necessary only at build time. A C++ library Debian package most likely includes headers, CMakeLists.txt, examples and even some tests that are certainly useful at build time but no longer necessary once we deploy our final application. As an example, the Gazebo snap currently contains 7920 headers files! Using the package.xml files, the colcon plugin is deducing what Debian packages are necessary to keep for the snap, namely the exec_depends. These packages along with our compiled binaries are first placed in the stage, then moved to the prime area. All the files placed in the prime folder are going to be bundled in the final .snap file.

As an example, the Debian package ros-foxy-rclcpp, necessary at run-time for our snap, contains CMake files in opt/ros/foxy/share/rclcpp/cmake/ and tons of header files in opt/ros/foxy/include/rclcpp that are no longer necessary once the snap is shipped.

This is one package out of the hundreds a complex application may rely on. How many other  tests, examples, doc, man files, sources that are not necessary for our final application can we spare?

Optimisation

The idea here is to clean up the prime part once the prime step has been completed by the other parts. We can add a clean-up part to search for all the files and directories we mentioned above and simply delete them. Let us add the following part to our snapcraft.yaml:

cleanup:
  after: [gazebo, ros2-foxy-extension] # we want to make sure that we clean only after the other parts did their prime step
  plugin: nil
  build-packages: [fd-find] # I personally find that installing fd-find and calling it is generally faster than calling find once
  override-prime: |

    # remove every header and source file
    fdfind --type file --type symlink '.*\.(hpp|hxx|h|hh|cpp|cxx|c|cc)$' $SNAPCRAFT_PRIME --exec rm {}

    # remove every cmake file
    fdfind --type file --type symlink '(.cmake|CMakeLists.txt)$' $SNAPCRAFT_PRIME --exec rm {}

    # remove every test(s) directories
    fdfind --type directory "^tests?$" $SNAPCRAFT_PRIME --exec rm -rf {}

    # remove every example(s) directories
    fdfind --type directory "^examples?$" $SNAPCRAFT_PRIME --exec rm -rf {}

    # remove man files
    rm -rf $SNAPCRAFT_PRIME/usr/share/man

    # remove doc files
    rm -rf $SNAPCRAFT_PRIME/usr/share/doc

    # remove perl share files
    rm -rf $SNAPCRAFT_PRIME/usr/share/perl

    # remove /usr/src/
    rm -rf $SNAPCRAFT_PRIME/usr/src

    # remove additional ROS 2 sources
    rm -rf $SNAPCRAFT_PRIME/opt/ros/foxy/src/

The list of files and directories to delete obviously depends on the use case. This is not an exhaustive list, but let’s see the results that we get adding this optimisation to the previous one. We must make sure to test our final snap enough to be sure that we didn’t remove something that was necessary.

Results

In terms of timing we don’t see anything significant, but in terms of size we see a significant reduction (9% size on the snap file and 12% on the filesystem).

Remove unnecessary Debian packages for deployment

In the dependency tree of our Debian apt packages, they might be packages that are really only necessary for build time. The colcon snapcraft plugin will use rosdep to automatically download all necessary dependencies. Depending on how dependencies are declared by a given project, there might be weight we would like to drop. However, the problem is that if we remove them classically (say e.g. with apt), we will also remove the packages that depend on and thus pull them in the first place. One can list the dependencies of a package by running:

apt-cache depends my-package

Similarly, we can list the packages that depend on a given package by running:

apt-cache rdepends my-package

For example, let’s explore what potentially unnecessary dependency ros-foxy-ros-core is directly pulling to our prime stage.

Depends: ros-foxy-ament-cmake
Depends: ros-foxy-ament-cmake-auto
Depends: ros-foxy-ament-cmake-gmock
Depends: ros-foxy-ament-cmake-gtest
Depends: ros-foxy-ament-cmake-pytest
Depends: ros-foxy-ament-cmake-ros
Depends: ros-foxy-ament-lint-auto
Depends: ros-foxy-ament-lint-common
Depends: ros-foxy-launch-testing
Depends: ros-foxy-launch-testing-ament-cmake
Depends: ros-foxy-launch-testing-ros
Depends: ros-foxy-rosidl-default-generators

This one package is pulling a lot of dependencies, and I’ve listed here only the ones that I estimated to only be necessary at build/test time. We can also note that some general purpose packages like python3-colcon, python3-rosdep, etc are no longer necessary. As mentioned at the beginning of the section, if a package A depends on B, removing B means also removing A. We cannot use apt or dpkg since we do not want to remove dependencies or dependent packages. Plus, in our case, the packages are installed in the /root/prime directory.

Optimisation

One solution here is to list all the files installed by a Debian package that we know we don’t want and simply remove all the installed files in the /root/prime.

To do so, we will create two functions. One for deleting every file installed by a Debian package and one for applying the previous function to every package that matches a prefix (for example, to remove every package starting with ros-foxy-ament-cmake).

We will add the following code to our cleanup part:

function remove-apt-package (){
  # because of the previous cleanup, we might have deleted headers that were installed by Debian packages, hence we ignore listing a file that no longer exist
  set +o pipefail
  $(dpkg -L $1 | xargs -I {} bash -c 'ls -ld $SNAPCRAFT_PRIME{} 2> /dev/null || true' | grep -v "^d" | awk '{print $NF}' | xargs rm -f)
  # remove empty directories
  $(dpkg -L $1 | xargs -I {} bash -c 'ls -ld $SNAPCRAFT_PRIME{} 2> /dev/null || true' | grep "^d" | tail -n +2 | awk '{print $NF}' | xargs -I {} rmdir --ignore-fail-on-non-empty {})
  set -o pipefail
}
function remove-apt-package-with-prefix (){
  for pkg in $(dpkg --get-selections "$1*" | awk '{print $1}')
  do
    remove-apt-package $pkg
  done
}
remove-apt-package ros-foxy-rosidl-adapter
remove-apt-package ros-foxy-rosidl-generator-c
remove-apt-package ros-foxy-rosidl-generator-cpp
remove-apt-package ros-foxy-rosidl-generator-py
remove-apt-package ros-foxy-ament-cmake-gmock
remove-apt-package ros-foxy-python-cmake-module
remove-apt-package python3-pytest

# remove globs of packages
remove-apt-package-with-prefix ros-foxy-ament-cmake
remove-apt-package-with-prefix ros-foxy-testing
remove-apt-package-with-prefix python3-colcon
remove-apt-package-with-prefix python3-rosdep
remove-apt-package-with-prefix cmake

Here, the selection of unnecessary packages is not exhaustive. We should remove packages that we are sure are unnecessary in our application.

Results

Now, what was the benefit of that:

In terms of timing again nothing significant, but in terms of size we see a gain one more time (9% reduction on the snap file and 7% reduction on the filesystem compared to the snap with only the unnecessary files removed).

Clean-up content sharing duplicates

In the previous section regarding the kde-neon extension and the content sharing snap, we saw that it saves us from having to store additional files. Our kde-neon content sharing snap doesn’t contain only our Qt libraries, but also all its dependencies that we might have downloaded too in our Gazebo snap. The idea here would be to list every duplicate and simply delete the duplicated files from our Gazebo snap since we can find them in our content sharing. Additionally, we also can access the files from our base: core20.

Optimisation

We will list all the files in our core20 and kde-neon content snap and remove them if they are also available in our prime directory. The kde-neon extension is using the last content sharing snap available (with a name in the format kde-frameworks-QTVERSION-core20). Thus, we will have to deduce that name from the environment variable provided by the extension.

We can append the following code to our override-prime of cleanup part:

KDE_CONTENT_SNAP=$(echo $SNAPCRAFT_CMAKE_ARGS | sed -n 's/.*\/snap\/\(.*\)-sdk.*/\1/p') # deduce the Qt content sharing snap name
# Remove duplicated files available in content snap
for snap in "core20" $KDE_CONTENT_SNAP; do # List all content-snaps and base snaps you're using here
  snap install $snap
  # We don't delete symlink
  cd "/snap/$snap/current" && fdfind . --type f --exec rm -f "$SNAPCRAFT_PRIME/{}" \;
  # We delete only symlink pointing to nowhere
  find "$SNAPCRAFT_PRIME" -xtype l -delete
done 

Results

In terms of timing, again, nothing significant, but in terms of size we see a reduction again (38% reduction on the snap file and 32% reduction on the filesystem compared to the snap with only the unnecessary Debian packages removed). This is a huge gain, and the extra work to achieve this gain is not much. We can recommend any snap relying on content sharing to use this optimisation to save space.

Conclusion

In this blog post part, we have seen that there are a lot of files that are actually completely unnecessary and bloating our snap. Understanding what we are installing in our snap is paramount to optimise it. In the case of other ROS snaps, we might find other types of files and packages that are completely unnecessary and could be deleted. Reducing the size of a snap will benefit all the snap users and applying this to every ROS snap would save precious time and storage space.

Continue reading Part 4 of this series.