Controlling a ROS2 Robot with a WiFi Stick

This chapter pertains to human perplexing behaviors:

Device Introduction

The protagonist of this chapter is… a stick, here it is:

Since the Mainlining of Qualcomm MSM8916 became open-source, various antique devices have been repurposed. The Redmi phone from 10 years ago, for instance, has become a Klipper host, significantly increasing its value. Due to market competition, domestic operators have released a batch of 4G WiFi sticks priced at just 9.9 yuan. Enthusiasts discovered that the chip inside was a Snapdragon 410 and developed the openstick project to port Mainlining to the stick. Consequently, the sticks sold out quickly, and I followed the trend and bought one.

The configuration is modest; the Snapdragon 410 is Qualcomm’s first 64-bit ARM processor, released at the end of 2013, exactly 10 years ago. It has a Cortex-A53 architecture, quad-core 1.2GHz. The stick comes with 512MB of memory and 4GB of ROM, which is barely sufficient. Openstick is based on Debian 11 (bullseye), and there are plenty of flashing tutorials online, so I won’t provide a link. After flashing, about 2.5GB of space remains, and it comes with OpenJDK 1.8, so I ran Eight on it. All the examples could run. Then… it was left to gather dust. Why? Because I couldn’t think of any use cases. This stick only has a power supply USB interface (which can also be driven as a wireless network card) and no peripheral interfaces, so it can’t be used as a device server like a Raspberry Pi. Also, due to limited resources, running a proper database is challenging, so it can’t be used as an application server. There are expansion docks available online, but they are several times more expensive than the stick itself, making it more worthwhile to spend the money on something else. Thus, after running Eight, I couldn’t find any use for it and left it aside, which is a pity considering the CPU’s decent performance.

A few days ago, while rummaging through old gadgets, I found it again and suddenly thought of ROS. ROS is essentially a domain controller for a network bus, and as long as there is a network, it can drive all ROS peripherals within the same subnet, effectively using the network as an I/O bus. Doesn’t this perfectly expand the stick’s peripherals? The stick is a 4G hotspot with a WiFi subnet, and if it can run Eight on ROS, it could control an autonomous vehicle, right?

So, I started flashing ROS2. But upon closer inspection, I realized that although ROS2 supports Debian bullseye, it is tier 3, meaning no binary packages are provided, and it must be compiled manually. The remaining options were to compile locally on the stick or cross-compile. Local compilation resources are too limited, but setting up a cross-compilation environment is cumbersome and prone to errors, and the device’s dependency libraries are also an issue. So, I decided to compile locally on the stick.

Next came a nightmare. I hope no one else tries this. I will provide the compiled image for flashing (the compiled version is the mainstream Humble LTS version. The image is stored in the cloud space where the previous virtual machine is stored: Baidu Cloud, extraction code: bo5c Aliyun Drive, extraction code: unr9).

To compile the entire ROS2 suite and add rcljava, a lot of memory and disk space are required. The stick’s resources are insufficient. At least 2GB of memory and around 20GB of disk space are needed. How to solve this? First, solve the memory issue by setting up a 2GB swap using the remaining space on the stick’s ROM, then mount virtual memory. This leaves almost no disk space, so compilation must be done by mounting a network file system. Although openstick doesn’t support this, the kernel has FUSE, so apt install sshfs can be used, and a Linux server can act as the disk server. Although it will be slow, it solves the disk space issue. Then comes the long, arduous, and repeatedly failing compilation process, which took five days. This was a performance art piece, and the stick barely survived. Here are some key points from the compilation process, though I strongly advise against attempting this.

• The first point is about Python3. The stick’s installed Python3 lacks main and library files. You need to apt install --reinstall libpython3.9-stdlib libpython3.9-minimal python3.9-minimal. Also, note that the pre-configured symbolic link for Python in /usr/bin is incorrect and needs to be deleted and relinked to Python3.9.

• The second point is that GitHub is not accessible for known reasons. You need to set up a proxy and configure git config --global https.proxy. However, many downloads during the installation process do not use git but curl or wget, so proxychains-ng is needed. After installation, use commands like proxychains4 -f /etc/proxychains.conf rosdep update.

• The third point is when using colcon build, some components are huge, such as rclcpp, which uses C++ templates extensively, causing high memory usage during compilation. Parallel compilation can lead to system crashes due to insufficient memory. A single process generally requires 2GB of memory (hence the earlier requirement for at least 2GB of memory). If compilation fails, add MAKEFLAGS="-j1 -l1" and --executor sequential compilation parameters, like MAKEFLAGS="-j1 -l1" colcon build --merge-install --executor sequential for single-process sequential compilation.

• The fourth point is that the mcap_vendor package needs to download version 0.8.0, but for some reason, the hash in the src directory does not match the actual downloaded package, requiring modification of the hash in the src directory from da39a3ee5e6b4b0d3255bfef95601890afd80709 to b44637791da2c9c1cec61a3ba6994f1ef63a228c.

• The fifth point is that the original openstick comes with OpenJDK 1.8 but does not configure JAVA_HOME, so this parameter needs to be exported before compiling rcljava.

Although it seems like there aren’t many issues, compiling is still fraught with various problems, making it a pointless endeavour. So, use my image. I recommend downloading from Aliyun Drive, which has no speed limits. Download rootfs.7z, extract it to get a rootfs.img image, which is the main system partition of the stick. There are many flashing tutorials, so I won’t go into detail (note that this image is for the 4GB memory version and is not compatible with other versions). Use Qualcomm Premium Tool or similar firmware recovery software to write the image to the corresponding partition directly (Note: Do not use one-click flashing to overwrite rootfs.img, as it will cause partition damage, and you will have to flash in 9008 mode. The reason is that the partition backup image format differs from the ADB flashing image format. After flashing, you can create an ADB flashing package if interested). Note that some versions of the stick require firmware from the boot partition to be copied to /lib/firmware to use the 4G network normally.

The firmware does not include Eight, so download it separately, and the environment will be fully configured:

wget https://www.yeeyaa.net/console/static/download/eight-seat-1.0.0.jar

System Deployment and Usage

Next, I will introduce manual compilation, mainly referencing https://wiki.debian.org/DebianScience/Robotics/ROS2 and https://docs.ros.org/en/rolling/Installation/Ubuntu-Development-Setup.html#build-the-code-in-the-workspace.

First, install the necessary tools for compiling and installing ROS2:

sudo apt install -y colcon python3-rosdep2 vcstool

These packages are essential for compiling and installing ROS2. vcstool downloads and generates project source code based on configuration, rosdep downloads corresponding dependency libraries based on the source code, and colcon compiles the source code.

Next, create an installation directory, ensuring a src subdirectory is created within it, then:

proxychains4 -f /etc/proxychains.conf wget https://raw.githubusercontent.com/ros2/ros2/master/ros2.repos

Note that the documentation suggests using sed to remove certain components, but this can be done as needed. However, the documentation itself is incorrect, as removing too many components can result in missing dependencies during compilation. I didn’t remove any components during my compilation and compiled everything. Next:

vcs import src < ros2.repos

After downloading the source code, configure the dependencies:

rosdep init
proxychains4 -f /etc/proxychains.conf rosdep update
proxychains4 -f /etc/proxychains.conf rosdep install --from-paths src --ignore-src -y --skip-keys "fastcdr rti-connext-dds-6.0.1 urdfdom_headers"

If ros-dev-tools is not installed, please install it. Otherwise, start compiling, and the nightmare begins:

source install/setup.bash
colcon build --merge-install

Occasionally, download errors or compilation failures may occur, requiring a switch to:

proxychains4 -f /etc/proxychains.conf colcon build --merge-install

Good luck.

After compiling, remember to source install/setup.bash, then compile rcljava. Ensure JAVA_HOME is set in advance. The compilation process is similar to the methods in the previous chapters, so I won’t elaborate further.

At this point, you can turn on the robot car, connect it to the stick’s hotspot, and similarly activate the chassis and camera. Then, use ros2 topic list to check the available message queues within the current subnet:

root@openstick:~# ros2 topic list
/PowerVoltage
/camera/color/camera_info
/camera/color/image_raw
/camera/color/image_raw/compressed
/camera/color/image_raw/compressedDepth
/camera/depth/camera_info
/camera/depth/color/points
/camera/depth/image_raw
/camera/depth/image_raw/compressed
/camera/depth/image_raw/compressedDepth
/camera/depth/points
/camera/extrinsic/depth_to_color
/cmd_vel
/diagnostics
/joint_states
/mobile_base/sensors/imu_data
/none
/odom
/parameter_events
/robot_description
/robotpose
/robotvel
/rosout
/set_pose
/tf
/tf_static

If the above topics do not appear, it may be because you have not set the appropriate ROS_DOMAIN_ID. Refer to previous chapters for setting it in .bashrc.

After compiling rcljava, source /root/ros2_java/install/local_setup.bash, then run Eight seat as before:

java -Dframework.boot.scanner.node=nodeName -Dfile.encoding=UTF8 -Dframework.web.user=xxxx -Dframework.web.password=pppp -Dframework.web.url=https://www.yeeyaa.net/api -Djdk.util.zip.disableZip64ExtraFieldValidation=true -cp eight-seat-1.0.0.jar:$CLASSPATH aQute.launcher.pre.EmbeddedLauncher

OK, Eight is running on the stick. Now, you can bind the node at https://www.yeeyaa.net/.

This is my account binding list, where ros-openstick is the node name given to the stick, bound to the ros-robot-control application. The ros-wheeltec is the robot car, also bound to the same application. Both devices have an aarch64 architecture, i.e., arm64. Notably, the winnt4 and win98 virtual machines have an x86 architecture, meaning the virtual machines are downgraded to 32-bit.

There is a special existence, the loongson, with a mipsel architecture. That is the Loongson.

This is a netbook that was ahead of its time, equipped with a Loongson 2f CPU using the MIPS-III instruction set. The Loongson 2 series was the first 64-bit CPU from Loongson Technology, born in 2007. At that time, developing a CPU independently in China was like moving mountains, difficult and seemingly hopeless. This netbook, branded as YeeLoong, was the company’s first consumer-facing laptop, produced in 2008, and it was a commercial failure. Its rudimentary user interface, weak performance, poor ecosystem, and lack of attention made it unsuitable for that era. The product had low sales, and the inventory couldn’t be cleared even five years later. Now, it has become a historical artefact. I acquired this machine in 2013, a rare 8101 version. Unlike the more common 8089 version, it has a 10-inch screen and is quite collectible. Here it is, the same model as the one cherished by the gentleman in the photo.

To improve the Loongson ecosystem, Loongson engineers ported the JDK 1.6 runtime to the MIPSel instruction set, but they didn’t have the resources to implement JIT, resulting in low runtime efficiency (JIT significantly impacts Eight, as its components are highly reused and almost always optimized by JIT). This was the only JDK version on Loongson 2f at the time. Eight made extensive backports to support JDK 1.6, partly to run on domestic chips and pay tribute to the entrepreneurs who bravely forged ahead in the dark. Loongson has since evolved and no longer uses the MIPS instruction set, but its pioneering spirit endures.

Returning to the topic, after completing all this, only Eight remains. Eight’s characteristic is that once it’s involved, things are about to conclude. Indeed, after deploying the system, open http://stick_ip:7241/sub/ros.html, and it’s the same recipe, the same taste. Drive on, friends, and head towards your brain.

It should be noted that there is significant lag during image capture and transmission, possibly due to the 410’s performance or the lack of optimization in the openstick’s JDK, or poor network performance (additional note: it is indeed due to poor network signal in the corner, possibly related to the brass heatsink). While only a single core is heavily loaded, code optimization using a thread pool to handle image messages may be considered. Additionally, the 410 generates considerable heat under full load, so I spent 8 yuan on two brass heatsinks.

Finally, let’s look at the resource usage after the stick is fully equipped. As seen, both disk and memory are very limited, with 4GB storage providing 3.3GB usable space, almost entirely consumed by ROS. The memory is only 512MB, making it unsuitable as a server. Interestingly, the virtual memory is worth noting. With no disk space for virtual memory, where does the swap come from? Notice that only 384MB of the 512MB memory is used, so where did the rest go? It turns out to be mounted as a memory file system, with swap in this file system. Although it seems trivial, this file system is compressed, doubling the space. Thus, the swap space is still in memory but doubled, at the cost of efficiency. This is a way for low-resource systems to trade CPU power for memory.

Next, let’s look at Eight’s usage. Running Eight in this sparse environment shows it performs well. Running the previously introduced services only used about 200MB of memory, and in this environment with only 600MB of memory (including swap), it ran smoothly. The 410 CPU’s performance is decent, and Eight runs smoothly. The high single-core usage is due to the polling used for convenience in the code. Overall, deploying Eight on ROS on this simple stick has maximized its potential.

Finally, here’s a family photo, with each device running Eight.

Significance

This endeavor started out of boredom but ended with some significance. Upon reflection, the stick’s scenario holds considerable meaning.

As shown, it actually provides a broad system deployment and manipulation model.

• The WiFi stick, after deploying ROS2, is equivalent to the central nervous hub, capable of controlling all ROS devices within the same subnet (often its hotspot).

• Other devices within the subnet do not need to possess significant computational power; they only require low-cost microcontrollers that can connect to the network and deploy micro-ROS, thereby transferring control to the neural hub. This not only reduces costs but also facilitates unified coordination by the hub.

• Upon deploying Eight, the stick can use its built-in 4G (or other networking methods) to connect to the central control system on the internet within the coverage area of the mobile network. Consequently, the central control system (the brain) gains supreme control over the distributed neural hubs, enabling it to manage the operation of each peripheral nerve (edge node).

• Since Eight is a dynamically deployable system with a very small size (modules are often in the range of tens to hundreds of KB) and can continue to operate even when disconnected from the network, it is suitable for scenarios with poor bandwidth and unstable network conditions (such as mobile networks).

• The advantage of dynamically deploying the system, rather than running it in the cloud and simultaneously uploading data, is that it significantly reduces bandwidth requirements and prevents network-induced system failures, while ensuring that business logic can adapt to on-site conditions at any time. This is essential for applications based on unstable network environments.

• After the system is deployed, it runs locally at the deployment point, reducing bandwidth usage, creating a more secure and stable command system, enhancing on-site response speed, and avoiding interference.

• The command hub (stick) generally has a certain level of performance (CPU, GPU, RAM, etc.) and can run complex algorithms (such as machine vision and pattern recognition based on OpenCV). This allows for the full utilisation of the computational capabilities at the deployment point for on-site command.

• Only specific data that needs to be collected and submitted is filtered and processed before submission, reducing server load and network overhead.

In essence, with a single stick, we can establish a remote control system anywhere in the world in just a few seconds. This system can operate autonomously, be remotely controlled, and adapt at any time. All this is achievable for just 9.9 (network fees not included: ).