Firmware Modification Kit

Firmware Mod Kit – Modify the Files in Firmware Binaries!

Firmware Modification Kit
Giving YOU the power to customize your firmware
documentation version 0.73 beta
all scripts, untrx, wrt_vx_imgtool, crcalc, binwalk and docs (c) 2006-2011
Craig Heffner <heffnercj@gmail.com>, Jeremy Collake <jeremy.collake@gmail.com>, and other contributors (see project page)
other portions (c) their respective holders

THERE ARE NEW DOCS: This documentation may be superseded or extended by documentation at the Google Code project site. An essentially new, and much improved, rendition is now available. Firmware-Mod-Kit-Ng 😉

Contents

I. Description
II. Why is this useful?
III. Where to get the kit
IV. Requirements
V. Compatibility
VI. Instructions
    A. extracting firmware
    B. modifying firmware
        1. installing packages
        2. removing packages
    C. re-building firmware
VII. Caveats
    A. binary compatibility issues
    B. maximum firmware size
    C. solutions to file system incompatibilities
VIII. Acknowledgements and more
IX. Links
X. Revision history

 

I. Description

This kit gives the user the ability to make changes to a firmware image without recompiling the firmware sources. It works by extracting the firmware into its component parts, then extracting the file system image (assuming its in one the supported formats). The user can then make modifications to the extracted file system, and rebuild the firmware image.

Although this kit completely automates the process, it is also useful for those who need a handy collection of utilities to manipulate firmware images. The following utilities are included in this kit, along with scripts to automate their use:

ASUSTRX An extended version of ASUSTRX that can build both ‘normal’ TRX files and, optionally, those with an ASUS addver style header appended. It can also, uniquely, force segment offsets in the TRX (with -b switch) for compatibility with Marvell ASUS devices like the WL-530g. This tool replaces both ‘normal’ trx tool and addver. Current versions included are: 0.90 beta.
ADDPATTERN Utility to pre-pend Linksys style HDR0 header to a TRX.
ADDVER ASUS utility to append a header to a TRX image that contains version information. ASUSTRX includes this capability. Current version: unversioned.
BINWALK Firmware analysis tool, used for searching firmware images for known file types. Used by the extract-ng.sh script to identify valid file systems inside firmware images. Current version included is 0.3.9. Tool home page: http://binwalk.googlecode.com.
CRAMFSCK CRAMFS file system image checker and extractor. Current versions included are:  2.4x.
CRCALC Re-calculates and patches header checksums for TRX and uImage firmware headers.
MKSQUASHFS Builds a squashfs file system image. Current versions included are: 2.1-r2, 3.0.
MKCRAMFS Builds a cramfs file system image. Coming in next version. Current versions included are: 2.4x.
MOTOROLA_BIN A utility that prepends 8 byte headers to TRX images for Motorola devices WR850G, WA840G, WE800G. Current version: unversioned.
UNCRAMFS Alternate tool to extracts a cramfs file system image. Use cramfsck instead whenever possible as it seems to be more reliable. Current versions included are: 0.7 (for cramfs v2.x).
UNCRAMFS-LZMA Extracts an lzma cramfs file system image, specifically those included in OpenRG based firmware. Current version included is 0.7rg.
UNSQUASHFS Extracts a zlib squashfs file system image. Current versions included are 1.0 for 3.0 images and 1.0 for 2.x images (my own blend).
UNSQUASHFS-LZMA Extracts an lzma squashfs file system image. Current versions included are 1.0 for 3.0 images and 1.0 for 2.x images (my own blend). Note: Not all squashfs-lzma patches are compatible with one another. I’m working on adding support for all common squashfs-lzma variations.
UNTRX Splits TRX style firmwares into their component parts. Also supports pre-pended addpattern HDR0 style headers. This was developed exclusively for this kit. Current versions included are: 0.45.
WRT_VX_IMGTOOL Utility to generate VxWorks compatible firmware images for the WRT54G(S) v5 series.

 

Tools/changes unique to this kit

This kit doesn’t just collect existing tools, it also offers new ones and modifications to existing ones. Custom code currently is:

ASUSTRX – Modified to allow ‘-b’ switch to force segment offsets (needed for some devices, like the WL-530g).
CRCALC – A new tool to re-calculate and patch the checksum fields for common firmware headers. Currently TRX and uImage headers are supported.
UNTRX – A new tool to extract the contents of a TRX image into its component parts.
UNSQUASHFS – Custom blend that supports extraction of 2.x squashfs images. This tool wasn’t officially made available until squashfs 3.0, so up until now it wasn’t available for 2.x images.
WRT-VX-IMGTOOL – A new tool to view, extract, build, and fix VxWorks compatible firmware images for the WRT54G(S) v5 series.
SPLITTER3 – Custom file splitter for certain types of linux firmware images.

II. Why is this useful?

A list of some of the many reasons this might be desired are:

  • add initialization scripts
  • install new packages
  • extend/change HTTP managed console
  • remove un-needed packages
  • mix-and-match packages from various flavors of the firmware image.

 

III. Where to get the kit

The best way to download and use this kit is to check out the sources from the repository. The automated scripts all automatically build the executables, and the sources. It is easy as downloading something and using it that way. Also, if you are unable to use a simple SVN checkout, then it is likely that this kit is a danger to you because it is for advanced users.

To anonymously check out the firmware-mod-kit:

 mkdir firmware_mod_kit
 cd firmware_mod_kit
 svn checkout http://firmware-mod-kit.googlecode.com/svn/trunk/ firmware-mod-kit-read-only

Kit downloads and links:

Project hosting and repository (new): http://code.google.com/p/firmware-mod-kit/
Guest SVN Checkout: svn checkout http://firmware-mod-kit.googlecode.com/svn/trunk/ firmware-mod-kit-read-only
User forums can be found here.

Packages:

All of the many OpenWrt packages should work with OpenWrt images and many other firmware images.

An old (2009) specific collection of DD-WRT packages for use with the kit: http://www.bitsum.com/files/ddwrt_packages.tar.gz .

 

IV. Requirements

I’ve worked to make sure the requirements are low enough for anyone to use this kit. However, to reduce distribution costs and better represent the multi-platform capabilities of this kit, the binary tools in the kit do automatically build themselves the first time you run one of the scripts. Any machine set up to build anything C or C++ on the machine will probably be ready. The few requirements are all basic items that everyone has easy access to:

  • A compatible platform. The following are specifically supported (or specifically unsupported):
    • LINUX     (recommended)
    • OS X         **warning: I don’t test every build under OS X. It’s up to you to email me if you have problems under it. Current builds may have issues.**
    • CYGWIN  **alpha stage testing (!! built firmwares may not boot !!)**
  • GNU C (gcc)
  • GNU C++ (g++)
  • GNU make
  • Standard C runtime library development
  • Standard C++ runtime library development
  • Zlib library development (for Ubuntu it is zlib1g-dev)
  • TAR and GZIP (optional, used by ipkg_install.sh)

OS/X is not supported at this time. The kit is probably not far from compatibility, but easier to just install a linux VM.

 

V. Compatibility

Any TRX style firmware image format should that uses squashfs-lzma as a root file system work fine. Firmwares that use regular zlib squashfs or unsupported file systems will work if the steps that extract the file system and rebuild the file system are tweaked.

The kit can build firmware images for the following devices, amongst many others:

  • WRT54G v1 – v6
  • WRT54GS v1 – v6
  • WRTSL54G v?
  • ASUS WL-330g
  • ASUS WL-500g/p
  • ASUS WL-520g
  • ASUS WL-530g
  • Belkin 7230-4 (some versions)
  • Belkin 7231-4 (?)
  • Buffalo WHR-G54S
  • Buffalo WHR-HP-G54
  • Siemens SE505
  • Trendnet TEW-632BRP
  • D-Link DIR-615 (maybe)
  • … lots of others*

The kit is known (or thought) to be compatible with the following firmwares:

DD-WRT v23 tested – versions v23 SP1 and later are compatible (soon older versions too). WARNING: Builds of DD-WRT later than 08/04/2006 (v23 SP2) have some protections against changing the web UI.
DD-WRT v24 tested
OpenWrt White Russian tested
OpenWrt Kamikaze untested (should work) – not really necessary, based on OpenWrt has its Image Builder.
FreeWrt untested – should work ok
HyperWrt untested
Ewrt untested
Sveasoft Alchemy untested
Sveasoft Talisman untested
Linksys / other vendor not supported by scripts yet – haven’t added cramfs handling
ASUS WL-330G untested – should work ok
ASUS WL-520G untested – should work ok
ASUS WL-530G supported
ASUS WL-550G untested  – should work ok
Trendnet TEW-632BRP tested
DLink DIR-615 untested
many others* untested

* Even if the kit doesn’t automatically support the firmware or device you want, you can manually use the tools inside to manipulate many firmware images, or request support be added.

 

VI. Instructions

I’ve scripts that simply and abstract the firmware modification process. Basically, it’s a one-step process to extract the firmware, and a one-step process to rebuild the firmware. There are also scripts to install or remove individual packages or entire directories of packages.

The procedure of calls is:

  1. extract_firmware.sh ….
  2. (optional) ipkg_install[_all]).sh …
  3. (optional) ipkg_remove[_all].sh   …
  4. build_firmware.sh

 

A. Extracting the firmware (extract_firmware.sh, extract-ng.sh)

Simply run extract_firmware.sh with the following parameters. You must run this tool from inside the directory it exists in.

$ ./extract_firmware.sh firmware.bin working_directory/
 

– firmware.bin is the firmware image you want to extract, i.e. DD-WRT.v23_MICRO_GENERIC.BIN.
– working_directory is the working directory you want to use. This is where the intermediate files and the extracted file system will be stored. Note that files in the working directory may be deleted!

It doesn’t matter which firmware image you supply, i.e. for a WRT54G or an ASUS WL-500G. These images are usually all the same and differ only in the header format. The rebuilding process will create images for the various models.

An extraction log is created as extract.log.

If extract_firmware.sh fails, the newer extract-ng.sh can be used. Although newer and less thoroughly tested than extract_firmware.sh, it is more flexible and can often extract file systems that extract_firmware.sh cannot:

$ ./extract-ng.sh firmware.bin working_directory/
 

 

B. Making changes to the firmware’s file system

Modifying the firmware is simple. The file system is stored in the working directory you supplied to extract_firmware.sh. Here are the subfolders of this directory:

    rootfs/
       
This is where the file system is. Make modifications here.
    image_parts/
       
This is where intermediate files are stored. If you need to replace the kernel (not at all recommended), you can do so by replacing the appropriate file here (usually segment2).
    installed_packages/
       
If you use the ipkg_install.sh tool, this is where packages you’ve installed after firmware extraction will reside.

One can manually change the files in rootfs and/or use the automated package scripts.

 

Installing packages (.IPK)

In compliment to manually making changes, one can use pre-built IPKG format package files with the ipkg_install/remove scripts. Although not all will work and a repository for packages verified to work has not yet been produced, many packages work just fine.

Packages are pre-built collections of files pertaining to a set of software. OpenWrt and DD-WRT both use packages, and most are cross-compatible. These packages are stored in a tar/gzip archive of a pre-defined structure that includes some control files. You can extract and copy the files manually, or use the ipkg_install.sh script included with this kit. For a list of some of the available packages (not all may work), see http://downloads.openwrt.org/whiterussian/packages/ and http://ftp.selfnet.de/pub/openwrt/backports/rc5/ . We are currently in the process of compiling a set of packages known to be good for DD-WRT … to help in this matter please visit here.

For information on how to create IPKs, see the IPK Creation Kit.

Example use:

$ ./ipkg_install.sh some_package-1.2.5.ipk working_directory/
 

some_package-1.2.5.ipkg would be the filename of the package.
working_directory is the same working directory you supplied to the extract_firmware.sh script.

Notes:

A script, ipkg_install_all.sh is also available to install an entire folder of packages.

The collection of installed packages will be stored in working_directory/installed_packages .

 

Removing packages (.IPK)

To remove packages there is the ipkg_remove.sh script.

Example use:

$ ./ipkg_remove.sh some_package-1.2.5.ipk working_directory/
 

some_package-1.2.5.ipkg would be the filename of the package.
working_directory is the same working directory you supplied to the extract_firmware.sh script.

Notes:

A script, ipkg_remove_all.sh is also available to install an entire folder of packages.

The collection of installed packages stored working_directory/installed_packages will be updated to reflect the change if successful.

 

C. Re-building the firmware (build_firmware.sh, build-ng.sh)

Rebuilding the firmware is as easy as extracting it. Use the build_firmware.sh script to automate the process.  You must run this tool from inside the directory it exists in.

Example use:

$ ./build_firmware.sh output_directory/ working_directory/
 

– output_directory is the path to which the created firmware images should be stored. If images already exist here, they will be over-written. Firmware images for various models will be emitted (these images are all the same but have different header patterns so they are accepted by the target models).
– working_directory is the working directory supplied to extract_firmware.sh.

A building log is created as build.log.

If extract-ng.sh was used to extract the firmware image, build-ng.sh must be used to rebuild it:

$ ./build-ng.sh working_directory/
 

 

VIII. Caveats

There are a few things to watch out for when modifying firmware images. If you make reasonable changes these problems will probably never affect you. Be warned, this section may be technical in parts.

A. Binary compatibility issues

  • If installing a binary package: 
    • It is possible the uclibc or glibc library was pruned (unused objects removed). In the unlikely event the binary package requires some of these previously unused (and now missing) functions, you may need to install to the firmware file system a new uclibc or glibc library. If you are installing another pruned uclibc or glibc library, then it is remotely possible some pre-existing package requires objects not found in this new library. In such a very rare case you can re-link the library or find a full copy of the library to install.
    • Different firmwares may use different kernel versions, or have different kernel modules installed. If you get too wild with your mix-and-matching (i.e. use older versions with newer, or mixtures of different firmwares all-together), problems can result. To install a new kernel (not at all recommended), replace the segment2 (or other) file found found in the intermediate directory with an appropriate one (can be from another firmware as long as lzma compression is used).
    • Make sure any dependencies are also installed to the file system, along with initialization scripts.
  • If making web UI changes:
    • WARNING: DD-WRT builds dated later than 08/04/2006 have some protections against modifying the web UI.

     

B. Maximum firmware size

Currently, the maximum firmware image size supported by this kit is 5,898,240 (0x5A0000) bytes, but can be adjusted through the “-m [size]” parameter of the asustrx tool.

C. Solutions to file system incompatibilities

! This section is very much a work in progress !

The squashfs file system is case sensitive (i.e. teSt.bin ! = test.bin) and adheres to EXT2 naming rules. To support proper extraction of the file system, the working directory should be on a case sensitive file system and support symbolic links. The scripts will warn you if you attempt to use as working directory on a case INsensitive file system. You’ll see a number of errors if you extract to a file system that doesn’t support symbolic links.

Note: Case insensitivity doesn’t always cause problems, especially if the file system preserves case ok. It depends on the firmware and version. Most of the time for DD-WRT as long as a case preserving file system is used as a working directory, all should work fine.

These solutions apply to those using CYGWIN, or some other environment that may not have a case sensitive file system by default. Linux users typically do not have this problem since EXT2 and EXT3 file systems are case sensitive.

WARNING: THESE ARE IN EARLY STAGES OF TESTING AND MAY NOT WORK.

  Applies to Description
SOLUTION 1 CYGWIN, OTHER It’s reported this does not work under CYGWIN due to the CYGWIN utilities all being case INsensitive (thanks pagedude).

Use a network share to a case sensitive file system. This is, by far, the easiest solution. Simply use an appropriate network share for the working directory.
 

SOLUTION 2 CYGWIN For NT partitions it’s reported you can enable case sensitivity through the following registry key, but I do not know if this works or not.

 \HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\obcaseinsensitive [REG_DWORD] = 0

SOLUTION 3 OS X (not usually needed, at least for DD-WRT) One can either Turn on HSFX for the boot disk, or (more easily) create a temporary HSFX case sensitive file system to use as a working directory.

Example of how to create a temporary HSFX case sensitive file system:

to create:
$ hdiutil create -size 20m -fs HFSX -volname db90h -nouuid test-image
to mount:
$ hdiutil attach test-image.dmg
to un-mount:
$ hdiutil detach /Volumes/db90h

(thanks solarflare for this information)

SOLUTION 4 CYGWIN
 
Mount an ext2 files system. Eko, of DD-WRT forums, contributes this tip. Make a partition on a hard drive or USB thumb drive, then install “Ext2 Installable File System For Windows“. 10MB should generally be plenty of working directory space, but 20MB is suggested for more optimal file system performance.
 

    

IV. Acknowledgments and more

The authors and maintainers of this kit are Jeremy Collake (db90h) and Craig Heffner.  Special thanks to Solarflare, TexHex <floh@fbesser.de>, and others.

If YOU would like to contribute to this kit, send me an email at jeremy.collake@gmail.com or visit the current project hosting at http://code.google.com/p/firmware-mod-kit/ . Developers are welcome.

 

X. Links