Optimise your ROS snap – Part 5

Welcome to Part 5 of our “Optimise your ROS snap” blog series. Make sure to check Part 4. This fifth part is going to cover two different optimisations. The first one, covers the compression algorithm. The second one, is about implementing extremely risky but efficient file deletion.

We are going to present two completely different optimisations in this article. The first one is a tradeoff that will totally depend on our final application use case. The second one is more of an experiment to see how far we can go in terms of minimising the size of a ROS snap.

Changing the compression algorithm

By using the kde-neon extension, our snap is enforced to use the compression method LZO. The details are explained in the blog post: why-lzo-was-chosen-as-the-new-compression-method.

What interests us here is that the XZ compression is normally the default, and we can actually still select it from snapcraft. The two compression methods are different and will provide different characteristics to our snap.

Optimisation

By default, our Gazebo snap is using the LZO compression method, so let’s change that for the XZ compression.

In our snapcraft.yaml we must add:

summary: Simulate before you build.
+ compression: xz
description: |

Results

The results are as follows:

We can see that the cold and hot start times increased, while the size of the snap file reduced by 28% just by changing the compression method. Robotics devices with limited internet access could benefit from a smaller snap file and thus smaller updates. In the case of Gazebo, since it’s a desktop application, we prioritise the time it takes to start rather than the bandwidth. For that reason, we won’t apply this compression method to the Gazebo snap.

Extreme trimming

Here, we are entering a more dangerous zone. While we have already removed a lot of files and Debian packages that we deemed unnecessary for our snap, there is certainly still a huge amount of files that are never used by our application. Instead of handpicking what looks unnecessary, we could do the opposite and only keep what is necessary. Hey, makes sense, no?

Optimisation

To list only what is necessary, we can use the Inotifywait program to monitor every access that took place in a given directory. In our case, monitoring the /snap/gazebo/current/ directory.

Make sure to do this also on the first run after install since an app can have a different behaviour on the first run.

We will run the Gazebo snap multiple times and with different scenarios while monitoring the files with:

inotifywait -r -m /snap/gazebo/current > /tmp/everything-access.txt

Once it’s done, we can stop inotifywait and apply some processing to our file to first remove the unnecessary data and remove duplicates:

cat /tmp/everything-access.txt| awk '{printf("%s%sn", $1, $NF)}' | sort | uniq > /tmp/necessary_files.txt

Then we ignore directories:

sed -i '/ISDIR/d' /tmp/necessary_files.txt

And finally, we adapt the path to our snap priming environment:

sed -i 's#/snap/gazebo/current#/root/prime#g' /tmp/necessary_files.txt

The necessary_files.txt is then a list of all the files (libraries, config file, texture, etc) used by our application during the different executions we ran. We can place the file in our snap/local/ folder and add the following to our snapcraft cleanup part:

# remove every file not listed as necessary
find $SNAPCRAFT_PRIME -type f -not -path "$SNAPCRAFT_PRIME/meta/*" -not -path "$SNAPCRAFT_PRIME/snap/*" | sort | comm -23 - /root/project/snap/local/necessary_files.txt | xargs rm -rf

This way, every file that is not picked up by inotifywait as being used will not end up in the snap. We simply ignored the meta/ and snap/ directory containing data that we shouldn’t delete (hooks, for example).

Results

The results are as follows:

Here, the results are impressive since the cold start is slightly reduced but most importantly, we reduce the size of the snap by 67% and the size of the installation by 73% compared to the “Cleanup content sharing duplicates” optimisation.

While the space saved is enormous, we must be cautious. We might have deleted something that we shouldn’t have, like a specific plugin, a texture or something useful in a scenario we didn’t test for. In the case of modular applications and with as many possibilities as Gazebo, we cannot safely recommend using this. In the case of an application with a much simpler usage, we could apply such optimisation, although we will need great care and lots of tests and support. Deemed too fragile, we won’t apply this optimisation to Gazebo.

Conclusion

The compression algorithm choice will highly depend on the final usage of the snap. It’s important to keep it in mind, especially since ROS snaps can be desktop applications as well as embedded applications on edge devices. We could see it rather like a parameter of a snap rather than an optimisation. On the other hand, what we called “extreme trimming” was an interesting experiment to run on our Gazebo snap. In the previous optimisations, we already removed many useless files and yet tons of them are still silently cluttering our snap. While this would be a risky optimisation, we can see it as a way to identify “intruders” in our final ROS snap.

Stay tuned, Part6 is coming