Command Line Archival in Mac OS X

© Amit Singh. All Rights Reserved.Written in Early 2003

Forked Resources

/bin/pax on Mac OS X is a command line utility that can read and write file archives and copy directory hierarchies. Pax supports various output archive formats:

Although pax is a useful utility, it does not handle "resource forks" on HFS+ file systems. HFS+ files can have two parts (forks): data and resource. The "data fork" contains what a file would usually contain on traditional file systems. The "resource fork" can contain arbitrary information: icons, preview pictures, key-value pairs, program segments etc.

Other command line utilities such as cp, cpio, dump and tar also will not handle resource forks. Much of commercial software (most of Adobe's offerings, for example) make use of resource forks, and as such, if any of these utilities are used to handle files with resource forks, things are not going to work properly.

hfstar and hfspax are modified versions of the respective utilities that handle resource forks.

MacOSX::File is a perl module that allows you to get and set HFS+ file attributes. psync is an accompanying perl script that uses MacOSX::File to implement incremental backup and restore.

/usr/bin/ditto (present by default on Mac OS X) and /Developer/Tools/CpMac (present if Developer Tools are installed) can be used to copy files and directories with resource forks preserved.

Finally, there must be (are?) various commercial software packages/tools that allow full archival of HFS+ file systems.

A Quick n' Dirty Archiver

Even with the complications described above, enough useful utilities are present by default on Mac OS X that a reasonably powerful archiver can be written rather straightforwardly. What follows is an example depicting use of such utilities. The specific utilities used are ditto, hdid and hdiutil.

mkdmg Screenshot

We would be developing a small shell script (using /bin/zsh, the Z shell) called mkdmg that creates a disk image (dmg) from files and/or directories. For argument parsing simplicity, the script would only accept one file or directory as argument. Please understand that this script should be considered alpha level software and should be used at your own risk!

Annotated Source: mkdmg

#! /bin/zsh # # mkdmg - create a disk image from files/folders # Copyright (c) 2003-2004 Amit Singh. All Rights Reserved. # Scratch folder where temporary archives will be created # SCRATCH=/tmp/.mkdmg.$$ # Reset path # PATH=/bin:/sbin:/usr/bin:/usr/sbin

We would be using /tmp/.mkdmg.$$ as a temporary location where intermediate archive files are stored. Be warned that if you try this script on folders with excessively large amounts of data in it, you need to have at least twice as much temporary disk space.

# Utility function for output # croak() { echo -n "\n$1" } # Utility function for checking return status # chkerror() { if; [ $? -ne 0 ]; then; halt; fi } Utility function for clean up # halt() { rm -rf $SCRATCH # defaults write com.apple.finder ShowRemovableMediaOnDesktop 1 # chkerror # FINDERPID=`ps -auxwww | grep Finder.app | \ # grep -v grep | awk '{print $2}'` # chkerror # kill -HUP $FINDERPID 2>/dev/null >/dev/null # chkerror exit 1 }

One possible (though unnecessary) "tweak" could be to prevent the disk image icon from appearing on the Desktop during the execution of the script. This requires setting the value of the key ShowRemovableMediaOnDesktop to 0 in the domain com.apple.finder using defaults. The Finder application should be sent a HUP signal after doing so. If this tweak is enabled, the cleanup function should undo the tweak.

main() { # Check if exactly one command line argument was specified # if [ $ARGC -ne 1 ] then echo "usage: mkdmg <file|directory>" exit 1 fi # Check if the specified file/directory exists # if [ ! -e $1 ] then echo "*** $1 does not exist." exit 1 fi SRC=$1 NAME=`basename $SRC` NAME="NAME" ARCH="$NAME Archive" echo -n "Using source $SRC" # Change directory to a scratch location # cd /tmp # Create a scratch directory # mkdir $SCRATCH croak "Creating temporary directory $SCRATCH"

If "/foo/bar" is the complete path of the source file/folder, the output archive will be named "bar.dmg" with the volume name being "bar Archive".

# Estimate how much space is needed to archive the file/folder # SIZE=`du -s -k $SRC | awk '{print $1}'` chkerror SIZE=`expr 5 + $SIZE / 1000` chkerror croak "Using $SIZE MB" # Create a disk image, redirecting all output to /dev/null # hdiutil create "$SCRATCH/$ARCH.dmg" -volname "$ARCH"\ -megabytes $SIZE -type SPARSE -fs HFS+ \ 2>/dev/null >/dev/null chkerror croak "$SCRATCH/$ARCH.dmg created"

The above code estimates the required size of the target volume, and creates a sparse disk image with an HFS+ file system on it.

# Optionally disable display of removable media on Desktop # # defaults write com.apple.finder ShowRemovableMediaOnDesktop 0 # chkerror # FINDERPID=`ps -auxwww | grep Finder.app | \ # grep -v grep | awk '{print $2}'` # chkerror # kill -HUP $FINDERPID 2>/dev/null >/dev/null # chkerror # # Mount sparse image # hdid $SCRATCH/$ARCH.dmg.sparseimage 2>/dev/null >/dev/null chkerror croak "$SCRATCH/$ARCH.dmg.sparseimage attached" # Find out allocated device # DEV=`mount | grep "Volumes/$ARCH" | awk '{print $1}'` croak "Device in use is $DEV" # Use ditto to copy everything to the image # Resource forks will be preserved # ditto -rsrcFork $SRC "/Volumes/$ARCH/$NAME" \ 2>/dev/null >/dev/null chkerror croak "Copied $SRC to /Volumes/$ARCH/$NAME" # Detach the disk image hdiutil detach $DEV 2>/dev/null >/dev/null chkerror croak "$DEV detached" # Compress the image (maximum compression) hdiutil convert "$SCRATCH/$ARCH.dmg.sparseimage" \ -format UDZO -o "/tmp/$ARCH.dmg" -imagekey \ zlib-devel=9 2>/dev/null >/dev/null chkerror croak "Disk image successfully compressed" croak "/tmp/$ARCH.dmg is ready" echo halt } main $1

Download

mkdmg.txt