What are resource forks in Mac OS X? How can they be used? What applications are using them? Are they being deprecated? Are developers encouraged to make use of them? Are they the future? Why is there a lack of up-to-date documentation? These are just a handful of questions that I had about the underlying Mac OS X filesystem. It took me several hours and hundreds of Google queries only to return with conflicting information. Through a process of gleaning information from hundreds of emails on mailing lists and usenet groups, sifting through outdated documentation, and hands-on experimentation I formed a coarse understanding of this technology. The following are my compiled observations which may or may not be correct.
I am writing this under the assumption that the target system in question is running the HFS+ filesystem.
Extended attributes vs. resource forks
First, I present an excerpt from the Wikipedia page on Resource forks:
The resource fork is a construct of the Mac OS operating system used to store structured data in a file, alongside unstructured data stored within the data fork.
From what I have gathered, there was a distinct difference between resource forks and extended attributes at one point. However, this appears to no longer be the case as I will attempt to explain in the next section. In short, the terminology usage has become muddied, and this seems to be the source of most confusion.
Resource forks are being deprecated
There is quite a bit of noise saying that resource forks are going away. Based on the information I found, this assertion is only partly true. Indeed, resource forks have already been phased out by way of Apple silently changing the underlying API implementation. Whereas this information was previously stored in the resource fork, it has now been moved to the data fork. From what I can tell, the majority of this data now manifests in the “Resources” directory that can be found within application bundles. However, there is still more resource fork data and this is now stored as an extended attribute under the key “com.apple.ResourceFork.”
At this point I must interject that it appears that Mac OS X continues to support named forks. There does not seem to be any limits on the number or size that these may be, but I’m not aware of any applications that make use of them. The majority of metadata appears to be contained within the extended attributes tree. This metadata is key-value based and generally very small in size.
Below is a bullet list of my observations.
- Apple’s resource fork was simply one of the many different forks that the Mac OS X filesystem can apparently handle.
- Apple’s resource fork is dead, but
- Apple’s APIs for interacting with their resource fork now store this information as an extended attribute (com.apple.ResourceFork) rather than using a separate fork.
- Mac OS X continues to support named forks.
- I have not seen any applications (Apple or otherwise) that use a named fork beyond the original resource fork.
- Extended attributes are used extensively by Apple products.
Accessing extended attributes (EA) and thereby com.apple.ResourceFork
Apple provides an API for interfacing with extended attributes, but they can be accessed without writing an application by using Terminal and the xattr command. For example, xattr -p com.apple.ResourceFork example.txt will display the data of the com.apple.ResourceFork attribute [if it exists]. Using the @ option with the ls command will give you a listing of all attribute keys for each file [e.g. ls -la@].
A neat trick that is still available in Mac OS X Leopard [10.5] from the days when ResourceFork was stored in its own fork is using ..namedfork/rsrc. If you don’t provide a name for the fork, then it defaults to Apple’s ResourceFork. For example, cat example.txt/rsrc will produce the same output as xattr -p com.apple.ResourceFork example.txt.
Gotcha!
File sizes calculated in terminal do not include the size of extended attributes, including the ResourceFork. This must be obtained separately through alternative means. For example, using ..namedfork/rsrc.
Clear as mud
I hope that this article sheds some light on the situation. Extended attributes are used quite a bit by Apple products. For example, Spotlight comments are stored as an extended attribute key-value pair. When a file is deleted using Finder, a key-value pair is written that indicates where the file originated. Time Machine uses extended attributes to keep track of backups, and the list can continue.
Extended attributes should not be feared and are a great way to store additional metadata. Unfortunately, support for extended attributes is not ubiquitous. Even some of Apple’s own bundled command line programs do not fully or properly support extended attributes. As such, from a programming point of view it might be wise to avoid use of extended attributes unless the data is transient. Any important data should be stored along with the file in the data fork.
Here’s hoping that EA support improves in the near future!
Please leave a comment if the information I have gathered is incorrect, or can be better clarified.
{ 8 comments… read them below or add one }
Here’s a command that will list all files with resource forks, in case you want to Rsync to a Linux machine and are wondering what might be missing in the process:
find . -type f -exec ls \-l {}/rsrc \; 2> /dev/null | awk ‘ $5 != “0″ { print $0 }’
First ‘cd’ into the directory you want to search, then run the command above
It would be nice to have a list of actions that can accidentally strip off a file’s extended attributes, such as copying to a Windows machine and back. (It used to be that even cp was not resource-fork-aware, but I believe that is fixed now.)
Indeed. The situation is inconsistent enough now on the same operating system, but becomes even more muddled when crossing OS boundaries.
I think rsync3 has extended attribute support now as an optional item.
As I am currently writing a paper on EA and used your website as inspiration, I’d like to clarify a bit on the fork part by showing an example I made.
snip
So, does this mean that all files with a /rsrc automatically gets the com.apple.ResourceFork Extended Attribute?
$ touch emptyfile
$ ls -la@ emptyfile
-rw-r–r– 1 klaag staff 0 13 Jan 11:23 emptyfile
echo “hello resource” > textfile
$ cat textfile
hello resource
$ cp textfile emptyfile/rsrc
$ ls -la@ emptyfile
-rw-r–r–@ 1 klaag staff 0 13 Jan 11:26 emptyfile
com.apple.ResourceFork 15
$
$ xattr -lp com.apple.ResourceFork emptyfile
com.apple.ResourceFork: hello resource
$
Yes, it seems that simply making something in /rsrc enables the com.apple.ResourceFork Extended Attribute.
snip
Are you aware of the mdls command? Where does all that data come from? mdls seems to have output even when there are no xattrs or resource forks for a given file.
From my searching, mdls can return information about files that do not necessarily need to be stored explicitly as metadata or with a resource fork. For example, the demonstration on MacRumors:Guides shows that mdls can be used to get the width of an image. From what I remember, this is not information that is going to be explicitly stored as metadata since this attribute does not differ from what the image width actually is.
Are there any advantages of this resource fork/ data fork aspect to apples operating system over however windows do it?
I haven’t really explored the NTFS alternate data streams, so I did a quick search and ran across the following article that you might find interesting: http://www.symantec.com/connect/articles/what-you-need-know-about-alternate-data-streams-windows
I have not spent any time using ADS on Windows, but I’ve found Apple’s implementation less than ideal. For example, not all utilities are aware of or work properly with the extended attributes. For example, the version of rsync that comes with OS X does not include full support (rsync 3 fixes this). This probably stems from the fact that not all GNU/BSD utilities support xattr , at least not yet.
They are both approaches at solving the metadata problem. Unfortunately, there seems to be some uncertainty surrounding them (are they going to be here tomorrow?). Ideally I envision a universal filesystem incorporating metadata management that greatly increases interoperability, but this is not realistic within the next 5 years. A more achievable goal would be to either drop the idea completely or fully incorporate the capabilities into the filesystem and APIs with a pluggable translation layer to make moves between filesystems seamless.