mensi.ch

Letting AI implement OpenWRT Support for Managed Switches

After my recent success with using AI to work on managed network switch firmware, I started experimenting more with how I could get Gemini CLI or Claude Code to do all the hard work for me to get OpenWRT ported to such devices.

I did eventually manage to get a mix of both these AI agents to get full support for the Horaco ZX-SW82TS-L2P including all LEDs, ports and PoE - but it involved hours of trying to coax the agents to make progress. While Claude Code tended to be better at managing knowledge and staying on track, it also burned through tokens and overage credits much faster than Gemini. During those experiments, I noticed that what you get out of Gemini depends a lot on the context and environment setup. So I wanted to try again on a different device.

The Horaco ZX-SWTG2C8F

This is a managed 8 port SFP+ switch. Opening it up, we can again immediately spot the familiar SPI flash chip due to the larger and smaller package options.

We can also see 2 RTL8231 GPIO expanders and what looks like an unpopulated RJ45 footprint that could be meant for a console port as we saw on the other device. Next to it, we see an unpopulated IC footprint - if you have some experience with hardware you might already guess what this is for, but we can also take a picture and ask Gemini, which will hopefully tell you that this looks like a typical setup for a MAX232 IC with its charge pump capacitors next to it.

Based on the MAX232 datasheet, we can identify the pins that correspond to the TTL side TX and RX pins. Soldering some fine wires to it and hooking it up to the oscilloscope reveals 3.3V UART signals - yay! With a suitable UART->USB adapter, we can start poking around and this time it even looks like rtk network on and tftpboot work out of the box.

The Harness

Since the goal is ultimately to let Gemini perform the work on its own, just hooking up serial isn't enough - we also need a way to reboot the device without me manually unplugging the power each time. I had a spare IKEA Zigbee smart socket, so I used that to power the switch. In combination with Zigbee2MQTT this lets me power cycle the switch without touching it.

The Porting Environment

Let's collect what information we have:

So let's start to organize a little directory structure for our workspace:

.
├── ghidra
│   └── ghidra_12.0.4_PUBLIC_20260303.zip
├── ghidra_projects
├── realtek_sdk
│   ├── dms-1250-oss-release
│   ├── README.md
│   ├── u-boot
│   └── zyxel-xgs-oss-release
├── SWTG2C8F
│   ├── NOTES.md
│   ├── openwrt
│   ├── PATCHES.md
│   ├── SWTG2C8F.bin
│   ├── TASKS.md
│   ├── TESTING.md

The idea here is that at the top-level, we have generic tools and reference material that is not specific to the switch only. You might have noticed I also just threw in the current release .zip file for Ghidra in case some binary analysis is necessary. In the switch-specific directory, we have a git clone of OpenWRT, the firmware dump, and some empty Markdown files we'll instruct Gemini to use for managing knowledge.

Now you likely could get going already with this - but typically the agents will struggle with managing context - particularly around the SDKs or the monolithic SPI flash dump. So first, we need to spend some time making this information digestible.

Mapping out the SDK

First, let's go into the realtek_sdk directory. We can leverage Gemini to populate a README.md file that helps Gemini itself to better understand the SDKs and the key information. I used this prompt to get started:

This directory contains 2 directories (dms-1250-oss-release, zyxel-xgs-oss-release) containing GPL 
releases of switch firmware, containing the Realtek SDK. A 3rd directory "u-boot" contains the upstream
u-boot repository checked out at the v2011.12-rc3 tag which should most closely match the U-Boot version
used in the Realtek SDK. Your task is to create a README.md file that describes and compares the contents
of each oss release. It might be beneficial to index the code with GNU global to make cross referencing   
more efficient. The resulting README.md should allow a skilled firmware engineer to quickly browse code
to understand how hardware initialization, device support, etc. works 
and finding the relevant code efficiently. This should greatly simplify implementing device support for openwrt.

This already generated a great overview, but was lacking a bit of detail in pretty much all of the sections. So I followed up with:

We'll now expand some sections. Remember to keep referencing concrete source code files or even code
excerpts to make the information is to understand and cross-reference. First, let's dig deeper on the
Bootloader. Figure out what custom commands, image headers or other notable things on top of u-boot's
upstream behavior have been customized. Extend README.md to fully document these customizations.

Which worked out great, and you can just keep doing this, asking it to expand section after section - for example, my next prompt was:

Great findings! Dive even deeper: Expand the README.md further on how flshow works, where it gets the partition layout
from, etc. Then also spend some time to understand how the LED configuration in the hardware profile works and how
GPIO expanders or shift registers or similar play their part. Remember to keep referencing code.

and

Awesome. Now let's move on to the linux kernel customizations. Analyze any behavior of interest for porting efforts
and document them, referencing code. Pay special attention to: How flash access works, MTD driver, partitons etc.
The rtcore.ko / rtk.ko or other custom kernel modules and any other customizations.

Gotta give some positive reinforcement every once in a while ;) The references to these kernel modules could be considered cheating a bit, since I saw these in my earlier work with the other switch where they were visible on the console log.

Now that we have this detailed README.md, we can just refer to it in the future, which will cause Gemini to just load this into context, giving a bit of a map to the SDK and it finds relevant information much faster.

Picking apart the Firmware

Now moving on to the firmware dump, the biggest challenge is to properly split it into the different MTD partitions and extracting the kernel and root filesystem. Often binwalk is used for this, but with this switch in particular, I got imperfect results. Since we know a lot about the structure from the Realtek SDK, we should be able to perfectly split up the pieces and extract everything properly instead of going by pattern matching and heuristics.

My goal here was to have Gemini create a set of scripts I can use to do the extraction. The first was extract_kernel.py, where I asked for a script scanning for the U-Boot Linux image header, accounting for the custom magic value the Realtek SDK sets (thus breaking binwalk). This took quite a bit of handholding, but it let me focus on the tricky parts while Gemini took care of the boring code. The result was a script that reliably extracts the linux image and rootfs for all firmware dumps of similar devices I have.

The next script, kernel_partitions.py is a bit more adventurous and I did not bother trying to understand how Gemini actually pulled it off - but I was essentially asking for some automated analysis of the linux image to find the baked in MTD partition table, by looking for known partition names like LOADER or BDINFO. This was very unreliable at first, but having it constantly verify the results against all firmware dumps I had eventually led to something that worked on all of them.

I have also experimented with getting this from U-Boot, but Gemini struggled a lot with the different addressing modes U-Boot goes through, where the same location in flash can be accessed by about 3 different address ranges depending on what stage during boot we are in.

With these two scripts and a hwprofile extraction script I had Gemini write for the previous device work, we have more stuff in our device directory:

├── SWTG2C8F
│   ├── BDINFO.bin
│   ├── hwprofile.c
│   ├── JFFS2_CFG.bin
│   ├── JFFS2_LOG.bin
│   ├── LOADER.bin
│   ├── NOTES.md
│   ├── OEMINFO.bin
│   ├── openwrt
│   ├── PATCHES.md
│   ├── rootfs
│   ├── RUNTIME.bin
│   ├── SWTG2C8F.bin
│   ├── SYSINFO.bin
│   ├── TASKS.md
│   ├── TESTING.md
│   └── vmlinux

Interaction with the Serial Port

Gemini is not fast enough to directly mess with the serial port - especially since we have just about a second or two to interrupt booting and getting to the U-Boot prompt. So we need some sort of manager process that takes care of the resetting and entering U-Boot while providing a simple CLI for Gemini to use.

I first tried a somewhat simple prompt just describing the problem - and it came up with something that worked. It was however plagued by timing issues and often Gemini would fall into a loop of trying to boot a custom built OpenWRT image, it not working for some timing issue and then Gemini trying again with artificially large sleep calls injected. This was good enough to actually perform all of the porting work but it was very slow and annoying.

What worked much better was writing out a design in Markdown and then just telling Gemini to implement that and verifying against a real device. You can see the design here and the resulting one-shotted script here.

I created a TESTING.md file which just contains a brief comment about how to use the script and the necessary values for MQTT server, topic and serial port to use, as well as where to put the built image to serve it over TFTP and how to boot it in U-Boot.

The Porting Process

So with that, we have:

  • A lot of supporting material with the SDK sources
  • The firmware split apart into nice chunks
  • An OpenWRT checkout to work on
  • A way to test on real hardware

So let's get started. This is the prompt I used:

We're working on reverse engineering a managed switch firmware in the SWTG2C8F directory.
It uses the Realtek SDK, for which we have sources from similar devices in the realtek_sdk
directory - see the realtek_sdk/README.md file to understand how it is generally structured.
In SWTG2C8F, I have already extracted the different MTD partitions into separate .bin files,
the kernel into a vmlinux file and the root filesystem into the rootfs directory. We also
have a heuristically extracted hardware profile in hwprofile.c. Now we're still missing some
clarity especially around the two RTL8231 chips on the board - they are presumably used for
LEDs and/or I2C or control signals for the 8 SFP+ cages on the switch. Your task is to examine
the firmware and devise a strategy on how we can determine the missing information needed to
come up with a functioning device tree and eventually OpenWRT port. Keep notes from your analysis
in SWTG2C8F/NOTES.md. Constantly revise and extend NOTES.md after every insight and discovery.
Also keep a TASKS.md file with tasks that you want to defer to later or to capture open questions.
If you think extra tooling is needed, propose what we'd need and how to set it up.

And off it went. It independently found the board_conf.ko kernel module where it found a lot of information about this specific switch. After a while, it was satisfied with the information extract regarding the SFP ports, proposed next steps and stopped. I just told it to continue with those next steps.

After a while, it seemed like it collected all the necessary information. I then prompted it to start the porting work:

Alright, I have now cloned the openwrt git repository into SWTG2C8F/openwrt.
Add support for our device in the proper place for realtek targets. There's been a lot of recent 
work on realtek devices, so examine the recent git log to understand the changes. When creating
the device tree, ensure you use a modern style, using the existing port macros etc where appropriate.
The flash layout should be compatible with the vendor firmware, i.e. leave LOADER, BDINFO etc alone
and only use the RUNTIME partition for openwrt. Finally, build the initramfs style image so I can
attempt to tftboot it. If compilation fails, fix any issues encountered. Remember to keep taking notes if you      
learn anything new.

I didn't quite tell it about the hardware access at this point since I wanted to test it myself first. There were some glaring issues at first around the memory layout (it just assumed a larger amount of memory) which I went back and forth on manually for a while. But eventually I told it about the TESTING.md file and it started happily iterating on its own and I didn't pay that much attention anymore. Occasionally, you could just hear the relay in the smart socket click, when it did another reset and boot cycle.

A big challenge was again dealing with the RTL8231 GPIO expanders, which already turned out to be a huge pain the last time around. These devices can run in multiple modes and the RTL930x has some integrated support for driving LEDs / polling them independently. I think Gemini eventually resorted to running rtk network on and then dumping memory registers via the U-Boot prompt to reverse engineer the configuration.

While Gemini can get a lot done independently with a setup like this, it still occasionally requires some steering. It seems to sometimes fall into a somewhat cyclic trial and error mode. You can break out of this by giving instructions on where to look for information next. For example, specifically telling it to use Ghidra in headless mode to analyze a binary.

Something that also works really well is to ask for verification of the work with specific measurable goals. For example, for a while, it struggled to get the I2C busses set up correctly. Saying that all SFP+ cages have modules inserted and to build an image with the full ethtool to read out the SFP modules worked wonders - it first got port 1 working but struggled with the rest, then eventually got all of them working.

Conclusion

Using AI agents to work on device support has made it more accessible for me, as I no longer have long, uninterrupted chunks of time I can just stay focused on reverse engineering these days. With AI, I just have to manage the overall process and goals, while AI does most of the heavy lifting.

In this case, it easy somewhat easy mode since there are released SDK sources under GPL for very similar looking devices, but I'd hope that the more general reference material, SKILLS.md and tooling is built, the better this process also works without these.


Using Gemini CLI to Figure Out How to Interrupt U-Boot

In my last post, I poked at the firmware of a network switch. While we could learn a lot from poking around the extracted data and even gain root access, one thing missing was U-Boot console access. Let's dive a bit deeper on what's going on there in this post.

The Console

The Horaco ZX-SW82TS-L2P switch we're looking at here has one of those RJ45 Cisco-style console ports on the front. Using a suitable USB serial adapter cable, we do get read access to U-Boot logs and then, once the switch boots into Linux, we get the switch's config CLI that you also get via telnet.

During the U-Boot phase, we do see a 3 second countdown (remember the bootdelay 3 we saw in the U-Boot environment from the last post), but no amount of mashing Esc, Enter or any other key for that matter seems to drop us into the U-Boot prompt.

This is somewhat weird though - why would you set up a 3 second boot delay if the prompt is inaccessible anyway? So my suspicion is that there is some special key you can use to get the prompt.

Taking a closer look at LOADER.bin

In principle, we do have the actual code that is executed on the device at startup in LOADER.bin, so we should be able to determine what mechanism is present to interrupt the boot. There's a problem though: I just don't have the time and stamina anymore to just spend endless nights cracking at this, and I also never dealt with MIPS assembly.

But these days, there's another option: LLMs. So I set up a fresh VM with all the artifacts extracted in the last post. I also set up Ghidra with GhidrAssistMCP and loaded LOADER.bin and finally installed Gemini CLI.

Before getting started, I also asked the "normal" web Gemini some questions about how the boot process in MIPS generally looks like, as I am more familiar with ARM. There I learned that the flash is directly accessible at address 0xBFC00000 but does not have DRAM at this point, so U-Boot generally has to bootstrap in some on-chip SRAM and setup DRAM before continuing. This makes the disassembly look a bit weird, since the addresses referenced are all over the place, depending whether they point to the early flash address range, the relocated one, SRAM, etc.

To give some additional help to Gemini CLI, I also ran objdump with the 0xBFC00000 starting location:

mips-linux-muslsf-objdump -m mips -EB -Mcpu=34kc -b binary --adjust-vma=0xBFC00000 \
-D readonly_reference_material/firmware/mtd_partitions/LOADER.bin > LOADER_BFC00000.objdump

Then finally, I started Gemini CLI and started with this prompt:

You are an expert reverse engineer. Analyze the U-Boot version in
readonly_reference_material/firmware/mtd_partitions/LOADER.bin to determine the overall
control flow, where it is relocated to, how it deviates from the sources in
readonly_reference_material/realtek_sdk/loader, how it can be interrupted during
bootdelay and what settings have to be used in ghidra to get full and correct analysis.
I've already created LOADER_BFC00000.objdump as a starting point.
Collect your insights in UBOOT.md. Be specific and go into a level of detail that allows a 
skilled firmware engineer to compile and adapt a custom u-boot version. At the minimum,
your analysis should include: The boot flow w.r.t to address relocations, decompression
or similar. The steps taken to initialize DRAM. Any other hardware initialization steps.
The bootdelay logic and interrupt keys that can be used to get a uboot prompt. The steps
executed in rtk network on and how they match to code in realtek_sdk. Context information
and explanation for people unfamiliar with the particular platform.

And off it went, poking at the loaded binary in Ghidra, extracting strings, hexdumping and cross-referencing everything with the Realtek SDK source from the ZyXEL GPL release for a similar switch Olliver Schinagl got from ZyXEL. This helped understand the overall process better, but Gemini was somewhat fixated on the way the bootdelay is implemented in the source code, where Hit Esc key to stop autoboot should be printed and you can interrupt with escape and wanted to move on to further analysis of GPIOs. Since that is not the case on the real device, there must be a difference. So I followed up with another prompt:

Before we move on to the kernel modules, revisit the bootloader. The switch hardware does
not print the message to press Esc, suggesting the code compiled for the device deviates
from the available source code. Perform in-depth reverse engineering on LOADER.bin to
determine how the actual compiled code behaves to explain the discrepancy.

And off it went again on a long stream of hexdumping, using Ghidra, etc. Just before I ran out of token quota for the day, it found something intriguing:

✦ I will check the content at file offset 0xdd6bc in LOADER.bin using od -t c to see
if it contains the string printed after the CTRL+C, z, h sequence is successfully entered.

That is interesting indeed, as I also saw a seemingly custom U-Boot command called zhaodandebug in strings output - so maybe that's the firmware engineer's name and they used the first 2 letters to unlock the U-Boot prompt?

And indeed, if we power up the switch and hit CTRL+C, z, h in sequence when the 3 second countdown starts, we get dropped into a U-Boot prompt:

U-Boot 2011.12.(3.6.8.55120) (Jul 11 2024 - 14:44:22)

Board: RTL9300 CPU:800MHz LX:175MHz DDR:600MHz
DRAM:  512 MB
SPI-F: MXIC/C22019/MMIO32-4/ModeC 1x32 MB (plr_flash_info @ 83f9a094)
Loading 65536B env. variables from offset 0xe0000
Net:   Net Initialization Skipped
No ethernet found.    <-- WHEN THIS SHOWS UP, PRESS CTRL+C, z, h RAPIDLY
RTL9300# # help
?       - alias for 'help'
base    - print or set address offset
boota   - boota  - boot application image from one of dual images partition automatically
bootm   - boot application image from memory
bootp   - boot image via network using BOOTP/TFTP protocol
cmp     - memory compare
cp      - memory copy
crc32   - checksum calculation
env     - environment handling commands
erase   - erase FLASH memory
flerase - Erase flash partition
flinfo  - print FLASH memory information
flshow  - Show flash partition layout
go      - start application at address 'addr'
help    - print command description/usage
iminfo  - print header information for application image
loadb   - load binary file over serial line (kermit mode)
loads   - load S-Record file over serial line
loady   - load binary file over serial line (ymodem mode)
loop    - infinite loop on address range
md      - memory display
mm      - memory modify (auto-incrementing address)
mtest   - simple RAM read/write test
mw      - memory write (fill)
nm      - memory modify (constant address)
ping    - send ICMP ECHO_REQUEST to network host
printenv- print environment variables
printsys- printsys - print system information variables

protect - enable or disable FLASH write protection
reset   - Perform RESET of the CPU
reset_all- Perform whole chip RESET of the CPU
rtk     - rtk     - Realtek commands

run     - run commands in an environment variable
saveenv - save environment variables to persistent storage
savesys - savesys - save system information variables to persistent storage

setenv  - set environment variables
setsys  - setsys  - set system information variables

sf      - SPI flash sub-system
sleep   - delay execution for some time
tftpboot- boot image via network using TFTP protocol
upgrade - Upgrade loader or runtime image
usb     - USB sub-system
version - print monitor, compiler and linker version
RTL9300# # help rtk
rtk - rtk     - Realtek commands


Usage:
rtk rtk network on
        - Enable the networking function
rtk netowkr off
        - Disable the networking function
rtk testmode [mode] [port]
        - Set default value for specific testing
rtk ext-devInit [deviceAddress]
        - set RTL8231 MDC address
rtk ext-pinGet [pinNum]
        - get external 8231 GPIO pin status
rtk ext-pinSet [pinNum] [status]
        - set external 8231 GPIO pin status
rtk i2c init sw [i2c_dev_id] [sck_dev] [sck_pin] [sda_dev] [sda_pin] [8/16 access type] [chipid] [delay] [rtl8231_address (for Ext-GPIO only)] [read_type 0~1]
        - create a i2c group and init
rtk i2c init hw [i2c_dev_id] [intf_id 0~1 for HW] [sda_pin] [8/16 access type] [chipid] [freq 0~3] [read_type 0~1]
        - create a i2c group and init
rtk i2c read [i2c_dev_id] [reg]
rtk i2c write [i2c_dev_id] [reg] [data]
rtk pinGet [pinNum]
        - get internal GPIO pin status
rtk pinSet [pinNum] [status]
        - set internal GPIO pin status
rtk ledtest [port] [led_index]
        - led test
rtk show hw_profile_list
        - show the current all supported hw_profile list
rtk phytestmode mode port channel
        - Set PHY into test mode; channel: 1=A,2=B,3=C,4=D,0=None
rtk boardid
        - Get board model id
rtk boardid id
        - Set board model id
rtk boardmodel
        - Get board model
rtk boardmode <str>model
        - Set board model
rtk 10g PORT [none | fiber10g | fiber1g | fiber100m | dac50cm | dac100cm | dac300cm]        - Set 10g port media
rtk phyreg get [port] [page] [reg]
rtk phyreg set [port] [page] [reg] [data]
        - Get/Set PHY register
rtk phymmd get [port] [addr] [reg]
rtk phymmd set [port] [addr] [reg] [data]
        - Get/Set PHY mmd register
rtk sdsreg get [sdsId] [page] [reg]
rtk sdsreg set [sdsId] [page] [reg] [data]
        - Get/Set MAC serdes register
rtk macsds set [sdsId] [sds_mode]
        - Set MAC serdes mode
rtk unit get
rtk unit set [unit]
        - Get/set unit ID

RTL9300# # rtk boardid
Board Model ID: <NULL>
RTL9300# # rtk boardmodel
Board Model: RTL9302D_2x8224_2XGE

I guess now I have to wait until my quota refills for more exploration, but that was surprisingly painless and productive!


Playing around with a ZX-SW82TS-L2P Network Switch

Having seeing some cool looking Chinese network switches on ServeTheHome, I've been checking out the new models every once in a while. This one has caught my attention and I got one:

You can find the exact same looking switch under multiple brands - I bought mine from Horaco. It is a managed 2.5G PoE switch with 2 SFP+ 10G cages. The label on the device itself states the model as "ZX-SW82TS-L2P". If we open it and go look at the PCB, we can also find "S1300WP-8GT-2S+ V1.03", which seems to match the scheme found in other popular Chinese switches, like for example the hasivo S1100WP-8XGT-SE. At the time of writing, hasivo has an Aliexpress listing with that model number and an identical looking device (apart from the color and branding).

The Web UI

The web UI looks surprisingly powerful, but in my case was all Chinese without any option to change. I asked the seller for an English firmware and got the response that one can switch the language via the console. It turns out that the switch also listens to telnet, so you can do this:

$ telnet 192.168.0.1
User Access Verification

Username:admin
Password:
Switch1>en
Switch1#factory language only-English
Switch1#reboot
Do you wish to reboot the switch? [Y/N]:

Logging in with the default admin/admin and the default IP address as printed on the label. Once the switch came back from reboot, the Web UI was indeed switched to English, yay!

The Console Port

The Switch comes with an RJ45 console port - this seems to be a Cisco/HP style RS232 RJ45 though I think only ground, RX and TX are connected. If we connect a USB to Console cable during startup, we get U-Boot logs:

U-Boot 2011.12.(3.6.8.55120) (Jul 11 2024 - 14:44:22)

Board: RTL9300 CPU:800MHz LX:175MHz DDR:600MHz
DRAM:  512 MB
SPI-F: MXIC/C22019/MMIO32-4/ModeC 1x32 MB (plr_flash_info @ 83f9a094)
Loading 65536B env. variables from offset 0xe0000
Net:   Net Initialization Skipped
No ethernet found.                                                                                        0
## Booting image from partition ... 0
## Booting kernel from Legacy Image at 81000000 ...
Image Name:   RTK_SDK
Created:      2025-04-27  13:18:11 UTC
Image Type:   MIPS Linux Kernel Image (lzma compressed)
Data Size:    6300104 Bytes = 6 MB
Load Address: 80000000
Entry Point:  803169a0
Verifying Checksum ... OK
Uncompressing Kernel Image ... OK

Starting kernel ...

console [ttyS0] enabled
bootconsole [early0] disabled
Calibrating delay loop... 531.66 BogoMIPS (lpj=2658304)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
TCP established hash table entries: 2048 (order: 1, 8192 bytes)
TCP bind hash table entries: 2048 (order: 3, 40960 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP: reno registered
UDP hash table entries: 256 (order: 1, 12288 bytes)
UDP-Lite hash table entries: 256 (order: 1, 12288 bytes)
futex hash table entries: 256 (order: 0, 7168 bytes)
ntfs: driver 2.1.31 [Flags: R/W].
jffs2: version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
msgmni has been set to 489
random: modprobe urandom read with 1 bits of entropy available
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0x0 (irq = 47, base_baud = 10764700) is a 16550A
SCSI Media Changer driver v0.25
RTK_SPI_FLASH_MIO driver is bypassed
RTK_NORSFG3 driver is used
=================================================================
init_luna_nor_spi_map: flash map at 0xb4000000
SPI NOR driver probe...
MXIC/C22019/MMIO32-4/ModeC add SPI NOR partition
MTD partitions obtained from built-in array
Creating 7 MTD partitions on "rtk_norsf_g3":
0x000000000000-0x0000000e0000 : "LOADER"
0x0000000e0000-0x0000000f0000 : "BDINFO"
0x0000000f0000-0x000000100000 : "SYSINFO"
0x000000100000-0x000000200000 : "JFFS2 CFG"
0x000000200000-0x000000300000 : "JFFS2 LOG"
0x000000300000-0x000000a00000 : "RUNTIME"
0x000000a00000-0x000002000000 : "RUNTIME2"
=================================================================
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
rtk_gen1_hcd_cs_init: rtk_gen1_hcd_cs_init()!
rtk_gen1_hcd_cs_init: register rtk_gen1_ehci ok!
usb_phy_configure_process: usb_phy_configure_process()!
rtk_gen1-ehci rtk_gen1-ehci: Realtek On-Chip EHCI Host Controller
rtk_gen1-ehci rtk_gen1-ehci: new USB bus registered, assigned bus number 1
rtk_gen1-ehci rtk_gen1-ehci: irq 28, io mem 0x18021000
rtk_gen1-ehci rtk_gen1-ehci: USB 2.0 started, EHCI 1.00
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 1 port detected
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
ohci-platform: OHCI generic platform driver
usbcore: registered new interface driver uas
usbcore: registered new interface driver usb-storage
TCP: cubic registered
Freeing unused kernel memory: 4964K (80427000 - 80900000)
init started:  BusyBox v1.00 (2025.04.27-05:16+0000) multi-call binary
Starting pid 26, console : '/etc/rc'
RTCORE LKM Insert...
    TC util init (isr)

Starting pid 62, console /dev/ttyS0: '/bin/sh'
Decompressing /mnt/switch.tar.gz ...
switch
switch/kernel
switch/kernel/kernel_rtl9312_7v6
switch/kernel/9310-sdk3.6.11
switch/kernel/kernel_9301_version.txt
switch/other
switch/other/certificate.pem.crt
switch/other/private.pem.key
switch/other/do_next
switch/other/AmazonRootCA1.pem
switch/module
switch/module/system_gpio.ko
switch/module/inos_ifksdk.ko
switch/module/i2c-ddm.ko
switch/module/reset.ko
switch/module/i2c-poe.ko
switch/module/i2c-rtc.ko
switch/module/i2c-encyt.ko
switch/app
switch/app/ospf6d
switch/app/ripngd
switch/app/mstpd
switch/app/snmpd
switch/app/ripd
switch/app/nsm
switch/app/lacpd
switch/app/webfiles.tar
switch/app/ospfd
switch/app/bgpd
switch/app/openssh.tar.gz
switch/app/imi
switch/app/pimd
openssh
openssh/sbin
openssh/sbin/sshd
openssh/libexec
openssh/libexec/sshd-session
openssh/ssh
openssh/ssh/ssh_host_rsa_key.pub
openssh/ssh/ssh_host_ed25519_key.pub
openssh/ssh/ssh_config
openssh/ssh/ssh_host_ecdsa_key.pub
openssh/ssh/ssh_host_ecdsa_key
openssh/ssh/ssh_host_rsa_key
openssh/ssh/sshd_config
openssh/ssh/ssh_host_ed25519_key
openssh/lib
openssh/lib/libcrypto.so.1.1
openssh/lib/libz.so.1
openssh/lib/libz.so.1.3.1
openssh/passwd
openssh/bin
openssh/bin/ssh
Start to run IFKSDK ...
Start to load DDM module ...
Start to load PoE module ...
Start to load encyt module ...
Start to load GPIO module ...
Start to load reset module ...
Start to load RTC module ...
Start to run MSTP ...
Start to run LACP ...
Layer 2 switch loading ...
Start to load configuration ...

ManagedSwitch 6.9.0
Creation date:May  7 2025 16:25:44



User Access Verification



Username:

Getting the Firmware

In the past, I was usually lucky enough to find a firmware update somewhere I could further poke at. In this case, I could find very little information about this device and no firmware update files at all. So let's have a look inside!

Looking at the PCB, the SPI flash chip immediately jumps out due to the two different possible footprints:

Searching for the IC marking reveals that this is likely a 25-series SPI flash chip from Macronix, the MX25L25645G (32MiB). This matches what we saw in the U-Boot log: SPI-F: MXIC/C22019/MMIO32-4/ModeC 1x32 MB.

Since I've never done this before, let's try desoldering the chip and dumping it! To get it off, I taped the area around the chip with Kapton tape to prevent accidentally blowing away components. I then used a hot-air station around 290°C and heated up the chip's legs. Once the solder started to look liquid, I carefully lifted the chip with tweezers.

Then, I used an 8-pin SOP clamp and a CH341A programmer to dump the memory with flashrom:

$ flashrom -p ch341a_spi -r firmware.bin -VV

which worked flawlessly and gave me a 32MiB file as expected.

Unpacking the Flash Dump

I started off with binwalk to get an idea what the dump contains:

$ binwalk firmware.bin
---------------------------------------------------------------------------
DECIMAL    HEXADECIMAL   DESCRIPTION
---------------------------------------------------------------------------
867884     0xD3E2C       CRC32 polynomial table, little endian
1048576    0x100000      JFFS2 filesystem, big endian, nodes: 589, total
                        size: 2093068 bytes
3145792    0x300040      LZMA compressed data, properties: 0x5D,
                        dictionary size: 67108864 bytes, compressed size:
                        6300104 bytes, uncompressed size: -1 bytes
10485760   0xA00000      JFFS2 filesystem, big endian, nodes: 21788, total
                        size: 23068672 bytes
---------------------------------------------------------------------------

Now this is not what I expected based on the U-Boot output we saw earlier:

MTD partitions obtained from built-in array
Creating 7 MTD partitions on "rtk_norsf_g3":
0x000000000000-0x0000000e0000 : "LOADER"
0x0000000e0000-0x0000000f0000 : "BDINFO"
0x0000000f0000-0x000000100000 : "SYSINFO"
0x000000100000-0x000000200000 : "JFFS2 CFG"
0x000000200000-0x000000300000 : "JFFS2 LOG"
0x000000300000-0x000000a00000 : "RUNTIME"
0x000000a00000-0x000002000000 : "RUNTIME2"

As it turns out, MTD does not have a partition table (like it says, loading from built-in array) so binwalk has no way of finding the MTD partitions and is instead looking for recognizable filesystems and such.

Since we already know from U-Boot where the partitions are, we can just extract them ourselves. As I was doing this on a Windows machine, here's some Powershell to achieve this:

$bytes = gc .\firmware.bin -Encoding Byte -Raw
sc .\firmware\LOADER.bin ([byte[]]($bytes | select -First 0xe0000)) -Encoding Byte
sc .\firmware\BDINFO.bin ([byte[]]($bytes | select -Skip 0xe0000 -First 0x10000)) -Encoding Byte
sc .\firmware\SYSINFO.bin ([byte[]]($bytes | select -Skip 0xf0000 -First 0x10000)) -Encoding Byte
sc .\firmware\JFFS2_CFG.bin ([byte[]]($bytes | select -Skip 0x100000 -First 0x100000)) -Encoding Byte
sc .\firmware\JFFS2_LOG.bin ([byte[]]($bytes | select -Skip 0x200000 -First 0x100000)) -Encoding Byte
sc .\firmware\RUNTIME.bin ([byte[]]($bytes | select -Skip 0x300000 -First 0x700000)) -Encoding Byte
sc .\firmware\RUNTIME2.bin ([byte[]]($bytes | select -Skip 0xa00000 -First 0x1600000)) -Encoding Byte

Which was horribly slow but seemed to do the trick. dd on Linux would probably have been much faster.

The U-Boot env

From the U-Boot log, we know that the BDINFO.bin file should contain the U-Boot environment. Looking at it in a hex editor looks like there's 4 bytes of something and then \0-byte separated key-value pairs. Let's whip up some silly Python to parse it:

f = open('BDINFO.bin', 'rb')
d = f.read()
for var in d[4:].split(b'\0'):
    if var == b'':
        continue
    print(var.decode('ascii'))

running it gives us:

baudrate=38400
boardmodel=RTL9302D_2x8224_2XGE
bootargs=console=ttyS0,38400n8
bootcmd=boota
bootdelay=3
bootmsg=1
ethaddr=00:E0:4C:00:00:00
ipaddr=192.168.1.1
ledModeInitSkip=1
serverip=192.168.1.111
stderr=serial
stdin=serial
stdout=serial

which looks somewhat interesting, as we potentially have 3 seconds to then use the tftboot command to boot from 192.168.1.111. So far I didn't have any luck interrupting u-boot though.

The main JFFS2 Filesystem

RUNTIME2.bin is the largest partition with around 23 MB and contains a JFFS2 filesystem. To extract it, we can use the jefferson PyPI package.

As we've already seen in the U-Boot log, there's a large switch.tar.gzfile. The script doing the decompression is likely the file do, as it contains:

echo "Decompressing /mnt/switch.tar.gz ..."
tar zxvf /mnt/switch.tar.gz -C /tmp/

which matches the log output and the verbose decompression that follows. We can of course also do the same to look at the contents. Intriguingly, tar tells us it is ignoring some extra data at the end. This could hint at some sort of checksum or signature mechanism for firmware updates.

After extraction, do executes /tmp/switch/other/do_next which matches more of the log output we've seen. One interesting thing it does is unpack openssh.tar.gz so presumably SSH can somehow be turned on.

The rest of the script is loading some kernel modules and starting some daemons. The custom ones seem to be /tmp/switch/app/imi and /tmp/switch/app/nsm.

We can also find all the Web UI endpoints, which seem to all be ELF binaries run via CGI - but for example looking at upgradeImg.do, it mostly seems to do its thing by issuing console commands to 127.0.0.1:2700 so it seems likely the Web UI is basically a frontend to the console.

All binaries seem unstripped, so understanding them should be somewhat easier.

The imi daemon

Based on the strings in this binary, it might be the main console server. Loading it into Ghidra, one thing in the main function immediately jumps out: It seems to expect some sort of authentication code on /dev/i2c-encyt.

Some more observations around the startup:

  • In imi_openssh_start: The SSH port seems to be controllable in /mnt/switch_info/swinfo.txt with an ssh_port: line but defaults to 22. The sshd seems to be started unconditionally.

  • In http_sock_init: The console open to the webinterface seems to be listening on 0.0.0.0:2700

  • There seems to be some sort of low level management interface in imi_ztpm_handle_bpdu looking for magic Ethernet frames.

Since port 22 and 2700 do not seem open on the running device, I assume there must be some additional mechanism or packet filtering involved.

Switching gears a bit, let's look at references to switch.tar.gz. The function writeUpgradeFile looks interesting. It first subtracts 0x1c from the filesize and calls imi_switch_file_additional_region_check. This function checks that these last 0x1c bytes at the end of the file start with 0x90abcdef. If we go look at the switch.tar.gz we extracted, it does indeed have that magic number at the expected position. The next 4 bytes are then checked against a number in string form that depends on the version of the kernel file in /tmp/switch/kernel/. In my case it's "9312". Finally, with suffix being these last files as uint32_t[], the filesize must be equal to suffix[3] + suffix[4] + suffix[5] + 0x1c.

Another check is done on suffix[2]: It is also interpreted as a string and used for another file existence check. In my case it is "7v6\0" and the resulting file /tmp/switch/kernel/kernel_rtl9312_7v6.

suffix[3] seems to contain a length from the start and just before that is an MD5. For the extracted example, this points just to this suffix, so the MD5 should be before 0x90abcdef. The MD5 computation is a bit silly though since strlen is used to determine the amount of bytes to be hashed. Here, the first \0 already appears after the first 3 bytes of switch.tar.gz, so only these 3 bytes are hashed. The hash indeed matches with what is stored at the end.

patch.tar.gz

There's also a patch feature where a patch.tar.gz can be downloaded from TFTP. The last 16 bytes are the MD5 of the file with the length again determined with strlen. Inside, a patch/patch.txt is expected, with the first two lines being:

  • # This is a patch.txt file.
  • cp /home/patch/patch.txt /mnt/patch.txt

The rest of the file is read line by line and every line is passed to system(). So let's give it a quick try to see if we can execute commands on the switch. First, I installed and started tftpd-hpa on an Ubuntu machine. We know from the web UI that ping seems to be installed, so let's have it ping our machine with this:

# This is a patch.txt file.
cp /home/patch/patch.txt /mnt/patch.txt
# Ping
ping 192.168.1.123

Then, we can create an archive:

$ tar -czf patch.tar.gz patch/patch.txt

It's important that the patch.txt is in a patch subdirectory. Now we need to append the magic strlen MD5, which we can do with a bit of Python:

import hashlib

with open("patch.tar.gz", "rb") as f:
data = f.read().split(b'\0')[0]
hash = hashlib.md5(data).digest()

with open("patch.tar.gz", "ab") as f:
f.write(hash)

Running this gives us the final file we can put into /srv/tftp/ to make it available over TFTP. Now connect to the switch over telnet and run (assuming the TFTP server is at 192.168.1.123):

Switch>en
Switch#download patch 192.168.1.123 patch.tar.gz
Do you wish to continue? [Y/N]: Y
Downloading file. Don't shutdown the switch. Please wait for a moment ...
Total data bytes sent/received: 203.
Decompressing file. Don't shutdown the switch. Please wait for a moment ...
Updating files. Don't shutdown the switch. Please wait for a moment ...

At this point, it will hang due to the infinite pings, but you should see those with tcpdump arriving at the TFTP host. Yay!

Getting SSH access

As we've seen in the do_next script and openssh.tar.gz, we do have an sshd binary available in /tmp/openssh/sbin/sshd and an SSH client in /tmp/openssh/bin/ssh. At first, I tried to just run sshd on a different port - but that wasn't accessible from the outside. Running on the standard port 22 also didn't work - on the serial console I could see that the port was already in use.

I then tried starting sshd on a different port, then connecting back to my TFTP host with a ssh key file I shipped within patch.tar.gz and opening a reverse tunnel with the -R option. Doing this while actually running the client in the background with an & at the end worked... Just to reveal that they did some custom vty logic to basically provide the CLI on any terminal. I didn't have much luck with the -T option trying to bypass PTY allocation :(

Building a custom binary

Back when I was messing with a Rigol oscilloscope, I experimented with cross-compiling little rust binaries and running them on the scope. We can try doing the same here. So let's get the toolchain:

$ sudo apt install gcc-mips-linux-gnu gcc-multilib-mips-linux-gnu
$ rustup target install mips-unknown-linux-gnu
$ cargo new rshell
$ cd rshell

This time, we will set the linker in .cargo/config.toml:

[target.mips-unknown-linux-gnu]
linker = "mips-linux-gnu-gcc"
rustflags = ["-C", "target-feature=+crt-static"]

And apply the same optimization settings to get a small binary in Cargo.toml:

[package]
name = "rshell"
version = "0.1.0"
edition = "2024"

[dependencies]

[profile.release]
lto = true
strip = "symbols"
codegen-units = 1
opt-level = "z"

And finally compile:

$ cargo build --target=mips-unknown-linux-gnu -Zbuild-std --release

And we get a ~1M binary in target/mips-unknown-linux-gnu/release/rshell. However, running this via patch.tar.gz just prints "Illegal Instruction" on the serial console. So it turns out the MIPS core used in these Realtek chips might be Lexra cores that do not fully support MIPS :(

Back to basics: C hello world

The RTL93xx chip is also used in other switches, for example the Zyxel XGS1210 series where some OpenWRT support seems to exist. I didn't really find anything hinting at requiring the RTL SDK toolchain to compile for these, so I'm wondering whether GCC actually avoids the problematic instructions while Rust doesn't. So let's give it a try with C and the MIPS GCC cross compiler we already installed.

#include <stdio.h>

int main() {
  printf("Hello World!");
  return 0;
}

and compile with

$ mips-linux-gnu-gcc -o hello main.c -static

sadly that runs into the same Illegal Instruction :(

Now the problem here is that I don't necessarily know the right search terms, so trying to find more information had a tendency to just end up with a million browser tabs. But luckily it's 2025 and tools like Gemini 2.5 Pro exist. Gemini proposed that this particular core does not have hardware floating point support, and that could also be a problem. After going back and forth a couple of times, the proposal was to build a MUSL softfloat toolchain:

$ git clone https://github.com/richfelker/musl-cross-make.git
$ cd musl-cross-make/
$ make TARGET=mips-linux-muslsf
$ make TARGET=mips-linux-muslsf install
$ export PATH=`pwd`/output/bin:$PATH
$ export LIBRARY_PATH=`pwd`/output/lib

We can then go back to our source code and compile our hello world:

$ mips-linux-muslsf-gcc -march=34kc -static main.c -o hello

And packaging this as a patch.tar.gz and running it on the switch actually works and prints Hello World! on the serial console, yay!

Getting a shell

We can use something like this to get a shell on the switch. In my experimentation it seemed like while packets egressing from the switch worked fine for any port, the responses didn't make it to the binary. However, if we have our machine listen on the telnet port 23 and have the reverse shell binary connect to it, we can get a root shell on the switch.

Further Work

As far as I've seen, the other binaries and kernel modules in the firmware are all unstripped with nice symbols - so it should be in the realm of the possible to get something like OpenWRT working. Though that requires a lot more work of course.