Avocent IP KVM Hacking: use SIP from any OEM on Dell 1082DS (and probably others)
Jan 31st 2020 — Avocent, Vertiv, Dell, KVM, firmware, hardware_hacking
Many KVM switches (local and IP) from major companies like Dell, HP or IBM are manufactured by Avocent. Those KVMs use special cables known as SIPs or RIPs to communicate with managed machines.In normal scenario only OEM and Avocent cables work on particular device, but as I discovered - at least in one case this was simple to bypass and allow any existing SIP to work.
This article summarizes my journey and discoveries while hacking Dell 1082DS IP KVM.
Background (or Why?)
I'm a homelabber with 24U rack half full of devices. Recently I acquired IBM KVM Console for cheap, and quickly got annoyed by connecting cables to each device I need to interact with.
I had old KVM laying around - it had 4 USB + DVI-I ports. After some digging (and disassembly) I found that those should be really DVI-D as inside there's HDMI switch... So quite useless with VGA monitor and all VGA devices in rack.
After researching local classifieds and auctions on our ebay-eqiv. (Allegro) I stumbled across cheap old HP, Dell and IBM KVM Switches, that reminded me one I seen some time ago on local market. With some research I found them all to be made by Avocent, well-known manufacturer of IP-enabled PDUs, KVMs, etc.
Unfortunately this research led me into two important things: First, IP interface uses Java (obviously...), second (and more worrying) those old KVMs has three passwords: console (serial), remote (ip) and for OSCAR, overlay interface for local console. While there are ways to reset first two, last one requires sending device back to manufacturer. I needed mostly local KVM (IP feature would be only a nice addition), but this looked like an gamble, even for $10-$20 equivalent for single device.
Later I found Dell 1082DS which is the subject of this post. It's still an Avocent product, but it's built on newer platform - local interface was visually identical to remote one, so I suspected it runs some kind of embedded *nix device with web browser to access it locally. And there was a seller with asking price of about $25 for a piece in "fully tested and working condition", so in worst case I'd return it, right? And with the right cable those 2nd generation products can even emulate USB Mass Storage devices, so I really wanted to check them out.
Those KVM switches uses special cables (called SIP in documentation). Those happens to be very expensive in Poland, but I found seller that offered IBM PS/2 SIPs for $1/piece, so I took a chance. Before I even received KVM, I found some HP SIPs on local flea market. Also for $1/piece so I acquired some of those too.
While docs for each of manufacturers mentions only it's own and Avocent cables as compatibile, considering how much I paid for cables - worst case scenario was that I'll return Dell KVM and hunt for IBM/HP one.
Initial disappointment
1082DS had arrived a couple of days later. A few minutes for powerup, console looks OK, first signs of X11 and GTK widgets are here (as suspected). So it's time to check cables, right? So I plugged IBM one, nothing happens. Second - nothing. HP - ditto.
Digging through options I found that they are visible in Tools -> Diagnostics. In section called suspect devices. Sigh. Can't be that easy.
Quick firmware analysis
As I seen obvious signs of Linux here, I decided to check firmware images. Dell has updates available with .fl extension. HP happens to have almost identical KVM (only visible difference is two PSUs integrated) known as 1X1EX8 KVM IP CONSOLE SWITCH G2, so I grabbed similar file from HP website.
Quick binwalk later I found uboot and linux squashfs embedded inside both files. And some filesystem-digging later, I had even more hopes - firmware stores most of hardware configuration in /config. Well, just look at this excerpt:
(...)
#Avocent PDUs
USING_AVO_SPC = yes
USING_CYCLADES_SPC = yes
# must say yes to USING_CYCLADES_SPC if using the following
USING_CYCLADES_REBRANDED = no
#Third-party PDUs - must use licensing scheme for Avocent branded but not necessarily for OEMs
#License requirement is turn off for DELL
USING_SERVERTECH_SPC = yes
SERVERTECH_LICENSE_REQUIRED = yes
SERVERTECH_ACCOUNT_OVERRIDE = no
USING_BAYTECH_SPC = no /* Baytech is deprecated and should be removed */
BAYTECH_LICENSE_REQUIRED = no
USING_APC_SPC = yes
APC_LICENSE_REQUIRED = no
USING_DELL_EATON = yes
EATON_LICENSE_REQUIRE = no
(...)
# OEM: 0=AVO 1=CPQ 2=DELL 3=HP 4=IBM 5=FTS 7=BBOX 8=DSR 9=APC 10=LENOVO 11=FCL 12=LCACs
# RIP: 1=PS/2 2=Sun 3=USB 7=SRL 20=USB2 21=PS2M 26=MPSRL 27=Pacer 28=Pacer2 29=Pacer2PS2
# 30=Pacer3 31=USBHS 32=USBFS
rip.cascade.enabled = yes
rip.interop = 0 1 0 2 0 3 0 7 0 20 0 21 0 27 0 28 0 30 0 31 0 32 8 1 8 2 8 3 8 7 8 20 8 21 8 26 8 27 8 28 8 30 8 31 8 32
user.accounts.enabled = yes
vmedia.allowed = yes
vmedia.enabled = yes
vmedia.locked = no
vmedia.reserved.allowed = yes
vmedia.write.allowed = yes
(...)This is just a part of
Of course I included rip.interop in this excerpt for a reason. This is an obvious indication of thing we want to change. And comparing it with HP and APC config (they left APC in Dell firmware too), the most important question was: how to modify it? It's obviously in squashfs, there's probably some kind of checksum on firmware and so on...
But when there's linux, there's probably serial console somewhere, right? So let's start with this and see if I can find any way to modify settings on running device.
What's inside?
Of course this meant I have to find serial connection. 1082DS includes 4 external serial ports - two to communicate with managed PDUs, one for local serial management and one for modem (out-of-bands). I don't have cables or pinouts for those so I had to open device anyway.
There's 4 pin header that grabbed my attention, as it usually indicates debug UART. Not this time! While it was indeed right connector, this is full-blown RS-232 levels serial communication. Quick probing with osciloscope later - Pin 1 is TX, 2 for GND, 4 for RX. So I dug my PL2032 serial cable instead of usual USB-TTLs, powered the device, and...
U-Boot 1.3.0-rc3 (Sep 24 2012 - 09:28:24) Polaris 2.1.20060
CPU: AMCC PowerPC 405EX Rev. D (PVR=12911475) at 400 MHz (PLB=200, OPB=100, EBC=33 MHz)
Security support
Bootstrap Option F - Boot ROM Location EBC (8 bits)
16 kB I-Cache 16 kB D-Cache
Board: Avocent Polaris - AMCC PPC405EX
I2C: ready
DRAM: Auto calibration - \ | / - \ | /
RQFD: 47 - RQS Read Delay
RFFD: 460 - Read Feedback Fractional Delay
RDSS: T2 sample - Read Sample Cycle Select
WDTR: 270 degrees advance - Write Data/DQS/DM Phase Corase timing
CLKP: 180 degrees - Write Clock Phase
DRAM: 256 MB (ECC not enabled, 400 MHz, CL4)
FLASH: 512 kB
NAND: NAND device: Manufacturer ID: 0x2c, Chip ID: 0xda (Micron NAND 256MiB 3,3V 8-bit)
Scanning device for bad blocks
256 MiB
SERIAL: Serial 1 Init
Net: ppc_4xx_eth0, ppc_4xx_eth1
Type "bootp" to get an IP address
Type "run net_nfs" to boot Linux and mount root filesystem via NFS
Hit any key to stop autoboot: 3 2 1 0
mounting system0
Copy /system0/uImage.polaris to 0x00200000...
[DONE]
## Booting image at 00200000 ...
Image Name: Linux-2.6.23
Image Type: PowerPC Linux Kernel Image (gzip compressed)
Data Size: 2916952 Bytes = 2.8 MB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Uncompressing Kernel Image ... OKwhoami? root
Some time later noise on serial interface stopped, so I pressed enter. And this followed:
# whoami
root
# mount
rootfs on / type rootfs (rw)
/dev/mtdblock3 on /mnt/m3 type yaffs2 (rw)
/dev/loop0 on / type squashfs (ro)
proc on /proc type proc (rw)
sys on /sys type sysfs (rw)
/dev/mtdblock5 on /mnt/m5 type yaffs2 (rw)
tmpfs on /tmp type tmpfs (rw)
none on /etc type unionfs (rw,dirs=/mnt/m5/etc=rw:/etc=ro)
none on /var type unionfs (rw,dirs=/mnt/m5/var=rw:/var=ro)
none on /user_data type unionfs (rw,dirs=/mnt/m5/user_data=rw)
none on /dev type unionfs (rw,dirs=/tmp/dev=rw:/dev=ro)
none on /dev/pts type devpts (rw)
none on /proc/bus/usb type usbfs (rw)
# ls /config
app.APC.cfg firmware.cfg redbirdA_hwconfig.cfg
app.cfg oem.APC.cfg redbirdD_hwconfig.cfg
boards.APC.cfg oem.cfg ripinfo.cfg
boards.cfg polaris_hwconfig.cfg syslog
edid redbird8_hwconfig.cfgWe are getting there! Have you noticed rw mtdblock5 and all those unionfs'es? Indeed, it's 32MB R/W flash partition and they mount some directories R/W this way.
Init scripts
I noticed main application is started from scripts in /etc/rc.d/. There's even interactive mode implemented when you can skip any of those scripts during boot. There's also S20-initcfg that loads our app.cfg and using some binaries loads configuration into ASICs.
First attempt
So I created config directory in /mnt/m5, mounted it as overlay and updated rip.interop in app.cfg to list all possible configurations. Since at this moment I was able to do it after device fully boots, I grabbed commands from S20-initcfg and... crash. Seems that device doesn't like overwriting ASICs memory on runtime.
2nd attempt
Remember when I mentioned it's possible to skip any of init files during boot? Let's look at that:
You have 5 seconds to interupt non-interactive booting (press Enter)...
Boot paused.
Do you wish to access the [D]ebug menu, Repeat [L]ast Boot Sequence, [C]ontinue Interacitve Boot, or [R]esume Auto-boot?
You have 30 seconds to respond...
c
Do you wish to execute /etc/rc.d/S05-fips_setenv? [y|n|a|q|s] a
Executing All Remaining Items Non-Interactively...This particular exceprt is from another run, but you get an idea. "S" drops into a shell, so before hardware init I did just that, mounted my config overlay and let the init script finish. After that I plugged HP cable... and it was detected!
First success, more digging
Not only cable was detected, but it asked me to upgrade it's internal firmware with one that was stored inside Dell (!) KVM firmware. This took a few minutes and it totally worked. I could connect to my laptop connected with VGA for display, and USB-PS/2 adapters for input.
After that I tried IBM cables. Suspect device. What? I triple checked rip.interop and it seemed fine. But I remembered something I saw before.
Debug menu
Remember this interactive booting menu? It mentioned Debug menu too. Let's look there again:
You have 5 seconds to interupt non-interactive booting (press Enter)...
Boot paused.
Do you wish to access the [D]ebug menu, Repeat [L]ast Boot Sequence, [C]ontinue Interacitve Boot, or [R]esume Auto-boot?
You have 30 seconds to respond...
d
Uber Debug Menu
1) OEM Spoofing (Only operates in NFS boot) [Disabled]
2) No Watchdog [Disabled]
3) RIP Firmware Test [Disabled]
4) Full OEM Override [Disabled]
5) FW OEM Override [Disabled]
6) Java Viewers Only [Disabled]
7) Core Dump To Flash [Disabled]
8) NFS Mounting of /tmp/nfs (Only operates in non-NFS boot) [Disabled]
9) ARI Mode Only [Disabled]
X) Exit Menu
Select Option to Modify or Exit.
4
Uber Debug Menu
1) OEM Spoofing (Only operates in NFS boot) [Disabled]
2) No Watchdog [Disabled]
3) RIP Firmware Test [Disabled]
4) Full OEM Override [Enabled]
5) FW OEM Override [Enabled]
6) Java Viewers Only [Disabled]
7) Core Dump To Flash [Disabled]
8) NFS Mounting of /tmp/nfs (Only operates in non-NFS boot) [Disabled]
9) ARI Mode Only [Disabled]
X) Exit Menu
Select Option to Modify or Exit.
x
Do you wish to access the [D]ebug menu, Repeat [L]ast Boot Sequence, [C]ontinue Interacitve Boot, or [R]esume Auto-boot?
You have 30 seconds to respond...
c
Do you wish to execute /etc/rc.d/S05-fips_setenv? [y|n|a|q|s] a
Executing All Remaining Items Non-Interactively...Full OEM Override... What it does is writes empty file /etc/oem_override. With that added to the puzzle, now IBM cables works too!
Hack persistence
So to recap: By mounting overlayfs on /config and overriding list of supported modules I got some cables working. By enabling OEM Override I was able to fix support for other cables. But this still requires manual interaction into boot progress, via serial port inside device. Far from ideal. If only we had /etc in RW and could write something into rc.d scripts...
Wait, we actually have this!
It required some thinking, as during early boot, /etc/rc.sh is called. It enumerates all executable scripts from /etc/rc.d and calls them one by one. At this moment mtd5 (and all overlays, including /etc) is not mounted yet, this is responsibility of one of early rc.d scripts. So I can't add file there, but I suspect I can overwrite any of files that are called after /etc overlay is mounted and before hardware init. So somewhere between S00-initfs and S20-initcfg. I've choosen S11-initnetwork, as all it does is bringing loopback interface up.
Hack removal
After more digging into init scripts, I found that when u-boot reset to defaults flag is set, S00-initfs will remove overlay contents for /etc before mounting it. Thus hack is not permanent; it's enough to reset device to its defaults. /mnt/m5/config will stay in flash forever, but it won't get loaded anymore.
Complete hack code
So this is all of my research summed up into kind of shell script. You need to execute those commands after device boots fully and you are in root console.
It's tested only on Dell KVM 1082DS with firmware 1.16.0, checked on two devices. Should be similar on different versions of firmware, and for other vendors.
Obviously you do it on your own risk. If it breaks you have to keep both pieces.
If you have different firmware revision or different device with similar firmware, I recommend you to read my findings and adapt them for your case.
#create directory for overlay config
mkdir /mnt/m5/config
cd /mnt/m5/config
#create config copy
cp /config/app.cfg app.cfg
#replace rip.interop line with all known options
sed -i "s/rip.interop.*/rip.interop = 0 1 0 2 0 3 0 7 0 20 0 21 0 26 0 27 0 28 0 29 0 30 0 31 0 32 1 1 1 2 1 3 1 7 1 20 1 21 1 26 1 27 1 28 1 29 1 30 1 31 1 32 2 1 2 2 2 3 2 7 2 20 2 21 2 26 2 27 2 28 2 29 2 30 2 31 2 32 3 1 3 2 3 3 3 7 3 20 3 21 3 26 3 27 3 28 3 29 3 30 3 31 3 32 4 1 4 2 4 3 4 7 4 20 4 21 4 26 4 27 4 28 4 29 4 30 4 31 4 32 5 1 5 2 5 3 5 7 5 20 5 21 5 26 5 27 5 28 5 29 5 30 5 31 5 32 7 1 7 2 7 3 7 7 7 20 7 21 7 26 7 27 7 28 7 29 7 30 7 31 7 32 8 1 8 2 8 3 8 7 8 20 8 21 8 26 8 27 8 28 8 29 8 30 8 31 8 32 9 1 9 2 9 3 9 7 9 20 9 21 9 26 9 27 9 28 9 29 9 30 9 31 9 32 10 1 10 2 10 3 10 7 10 20 10 21 10 26 10 27 10 28 10 29 10 30 10 31 10 32 11 1 11 2 11 3 11 7 11 20 11 21 11 26 11 27 11 28 11 29 11 30 11 31 11 32 12 1 12 2 12 3 12 7 12 20 12 21 12 26 12 27 12 28 12 29 12 30 12 31 12 32/g" app.cfg
#enable oem_override. this can be done from debug menu at boot
#for some reasons hp cables worked with just interop updated, IBM required override to operate
touch /etc/oem_override
#install boot-hack
cd /etc/rc.d/
ls -lah S11-initnetwork
cp /sbin/initnetwork.sh ./S11-initnetwork
sed -i '2i#boothack start' ./S11-initnetwork
sed -i '3i echo Avocent any-SIP hack' ./S11-initnetwork
sed -i '4i echo 2020 kitor.pl' ./S11-initnetwork
sed -i '5i echo Reset device to defaults and delete /mnt/m5/config for complete remove.' ./S11-initnetwork
sed -i '6i mount -t unionfs -o dirs=/mnt/m5/config=rw:/config=ro none /config' ./S11-initnetwork
sed -i '7i#boothack end' ./S11-initnetwork
#reboot
sync
reboot