Mac OS X Resource Forks

What are resource forks in Mac OS X? How can they be used? What applications are using them? Are they being deprecated? Are developers encouraged to make use of them? Are they the future? Why is there a lack of up-to-date documentation? These are just a handful of questions that I had about the underlying Mac OS X filesystem. It took me several hours and hundreds of Google queries only to return with conflicting information. Through a process of gleaning information from hundreds of emails on mailing lists and usenet groups, sifting through outdated documentation, and hands-on experimentation I formed a coarse understanding of this technology. The following are my compiled observations which may or may not be correct.

I am writing this under the assumption that the target system in question is running the HFS+ filesystem.

Extended attributes vs. resource forks

First, I present an excerpt from the Wikipedia page on Resource forks:

The resource fork is a construct of the Mac OS operating system used to store structured data in a file, alongside unstructured data stored within the data fork.

From what I have gathered, there was a distinct difference between resource forks and extended attributes at one point. However, this appears to no longer be the case as I will attempt to explain in the next section. In short, the terminology usage has become muddied, and this seems to be the source of most confusion.

Resource forks are being deprecated

There is quite a bit of noise saying that resource forks are going away. Based on the information I found, this assertion is only partly true. Indeed, resource forks have already been phased out by way of Apple silently changing the underlying API implementation. Whereas this information was previously stored in the resource fork, it has now been moved to the data fork. From what I can tell, the majority of this data now manifests in the “Resources” directory that can be found within application bundles. However, there is still more resource fork data and this is now stored as an extended attribute under the key “com.apple.ResourceFork.”

At this point I must interject that it appears that Mac OS X continues to support named forks. There does not seem to be any limits on the number or size that these may be, but I’m not aware of any applications that make use of them. The majority of metadata appears to be contained within the extended attributes tree. This metadata is key-value based and generally very small in size.

Below is a bullet list of my observations.

  • Apple’s resource fork was simply one of the many different forks that the Mac OS X filesystem can apparently handle.
  • Apple’s resource fork is dead, but
  • Apple’s APIs for interacting with their resource fork now store this information as an extended attribute (com.apple.ResourceFork) rather than using a separate fork.
  • Mac OS X continues to support named forks.
  • I have not seen any applications (Apple or otherwise) that use a named fork beyond the original resource fork.
  • Extended attributes are used extensively by Apple products.

Accessing extended attributes (EA) and thereby com.apple.ResourceFork

Apple provides an API for interfacing with extended attributes, but they can be accessed without writing an application by using Terminal and the xattr command. For example, xattr -p com.apple.ResourceFork example.txt will display the data of the com.apple.ResourceFork attribute [if it exists]. Using the @ option with the ls command will give you a listing of all attribute keys for each file [e.g. ls -la@].

A neat trick that is still available in Mac OS X Leopard [10.5] from the days when ResourceFork was stored in its own fork is using ..namedfork/rsrc. If you don’t provide a name for the fork, then it defaults to Apple’s ResourceFork. For example, cat example.txt/rsrc will produce the same output as xattr -p com.apple.ResourceFork example.txt.

Gotcha!

File sizes calculated in terminal do not include the size of extended attributes, including the ResourceFork. This must be obtained separately through alternative means. For example, using ..namedfork/rsrc.

Clear as mud

I hope that this article sheds some light on the situation. Extended attributes are used quite a bit by Apple products. For example, Spotlight comments are stored as an extended attribute key-value pair. When a file is deleted using Finder, a key-value pair is written that indicates where the file originated. Time Machine uses extended attributes to keep track of backups, and the list can continue.

Extended attributes should not be feared and are a great way to store additional metadata. Unfortunately, support for extended attributes is not ubiquitous. Even some of Apple’s own bundled command line programs do not fully or properly support extended attributes. As such, from a programming point of view it might be wise to avoid use of extended attributes unless the data is transient. Any important data should be stored along with the file in the data fork.

Here’s hoping that EA support improves in the near future!

Please leave a comment if the information I have gathered is incorrect, or can be better clarified.

14 comments… add one

  • Tim Jul 11, 2014

    I have added a comment here, which may be helpful.

  • Jason Chiang Apr 1, 2014

    I spent about 2 hour to find out what really take the size in file. One of my jpeg image is showing 64kB in finder but console ls cmd showing 39kB. Now I saw xattr that shows me where those size come from in finder. It’s okay with me that apple has EA in filesystem but I wish that there is an switch that could control if finder should display the file actual size or the size combined with EA. Because my colleague is having really big issue to find out the real file size for a bunch jpeg size. She is not familiar with console operations…This is really frustration… I might end up remove all the EA of those jpeg files

  • Syed Aug 30, 2012

    Dear All,

    I am confused in differentiating Extended attributes, resource forks and alternate data stream.

    From reading different sources I got two different ideas:

    1). these three are same things, in linux it is called extended attributes, in mac it is called resource fork, and in windows it is called alternate data stream (ads).

    2). Or, only resource fork and ADS are same things and extended attribute is different from these two. And the difference between extended attribute and the other both (fork, ADS) is the size difference only. As extended attributes are smaller in size.

    Please tell me which is correct. If both are incorrect then please send me the correct differences between these three.

    Thanks to all

    Syed

    • Jon Stacey Sep 4, 2012

      Syed,

      It’s confusing and difficult to explain. Here’s my short answer that’s even more confusing:

      Extended attributes and filesystem forks are different things, but they can also be the same thing in core concept and depending on the implementation.

      A tad more detail:

      There is a difference between a resource fork and an extended attribute. My understanding is that an extended attribute is typically constrained in size because this information is stored inline with the filesystem tree. A filesystem fork on the other hand can typically be as large as the maximum filesystem file size. Some systems such as Windows may store their extended attributes as a resource fork which can blur the line.

      So, there is a technical distinction between a fork and an extended attribute. However, it seems that we generally use the term ‘extended attribute’ to refer to the same concept. That concept is attempting to answer the question, what do we do with that extra information that is important and specific to a certain file, but does not belong within the data stream itself?

      Some information is pretty generic and can be applied operating system wide such as the time a file was created. [side note: each Operating system might have different interpretations of what this means though. Take a look at the difference in ctime between Unix and Windows systems.]. This can be rolled into the filesystem inline if the creators wanted to do it that way. But what do we do with other metadata that’s not so generic and can be quite arbitrary, and perhaps even large? That’s where the concept of extended attributes and resource forks enters the scene.

      So, to answer you questions directly:

      1. Extended Attributes on Linux, Resource Forks on Mac OS X, and Alternate Data Streams on Windows can roughly be categorized as the same thing–the concept of handling arbitrary metadata for a given file. However, I believe that they can be called “extended attributes” regardless of what operating system we’re talking about. Ultimately, these are filesystem features.
      2. Alternate data streams on Windows are filesystem forks, just as Mac OS X resource forks are filesystem forks. Some systems such as Windows store extended attributes in a filesystem fork.

      So, to recap. Filesystem forks and extended attributes are different things technically, but are part of the same concept of trying to answer the question what do we do with the metadata? Storing extended attributes in a fork is one possible implementation method and one that I believe Windows uses.

      That’s my best explanation as I understand it. No guarantees on accuracy. Be sure to read up on Wikipedia to verify and further clarify the matter.

      Edit: I should also note to readers that after writing this response, I think some of the details of my original observations in this blog post are slightly incorrect. As I demonstrate in this response, there is some grey area between the difference of an extended attribute and a resource fork. There is a technical difference between the two. Some operating systems build one on another which can further muddy the waters, but in general the overarching concept of how to handle the metadata is the same.

      • I am thankful to you for the reply, your reply would definitely help me a lot.

        Regards

        Syed Rahman Mashwani
        PhD Student
        University of Peshawar, Pakistan.

      • Tim Jun 30, 2014

        I don’t think that is right.

        What it looks like to me is:

        (1) Previously, (at least before OS X), resource forks were stored in the file system.

        (2) Mac OS X (or HFS+, I don’t know) introduced Extended Attributes, which are stored in the directory structure.

        (3) Once Extended Attributes are available, resource forks are stored as Extended Attributes. You can see that in the ls -@ command.

        (4) I guess there is a special trick in ls, which allows the one special Extended Attribute ‘com.apple.ResourceFork’ to be accessed as though it were a separate file below the normal file and called ‘rsrc’.

        So Extended Attributes and Resource Forks are different, it is just that Extended attributes are now used to implement the old Resource Forks.

  • Doug May 16, 2012

    Here’s a command that will list all files with resource forks, in case you want to Rsync to a Linux machine and are wondering what might be missing in the process:

    find . -type f -exec ls \-l {}/rsrc \; 2> /dev/null | awk ‘ $5 != “0” { print $0 }’

    First ‘cd’ into the directory you want to search, then run the command above

  • Matt Neuburg Mar 13, 2012

    It would be nice to have a list of actions that can accidentally strip off a file’s extended attributes, such as copying to a Windows machine and back. (It used to be that even cp was not resource-fork-aware, but I believe that is fixed now.)

    • Jon Stacey Mar 15, 2012

      Indeed. The situation is inconsistent enough now on the same operating system, but becomes even more muddled when crossing OS boundaries.

      I think rsync3 has extended attribute support now as an optional item.

  • Krister Laag Jan 13, 2011

    As I am currently writing a paper on EA and used your website as inspiration, I’d like to clarify a bit on the fork part by showing an example I made.

    snip
    So, does this mean that all files with a /rsrc automatically gets the com.apple.ResourceFork Extended Attribute?
    $ touch emptyfile
    $ ls -la@ emptyfile
    -rw-r–r– 1 klaag staff 0 13 Jan 11:23 emptyfile
    echo “hello resource” > textfile
    $ cat textfile
    hello resource
    $ cp textfile emptyfile/rsrc
    $ ls -la@ emptyfile
    -rw-r–r–@ 1 klaag staff 0 13 Jan 11:26 emptyfile
    com.apple.ResourceFork 15
    $
    $ xattr -lp com.apple.ResourceFork emptyfile
    com.apple.ResourceFork: hello resource

    $
    Yes, it seems that simply making something in /rsrc enables the com.apple.ResourceFork Extended Attribute.
    snip

  • Bill Sep 2, 2010

    Are you aware of the mdls command? Where does all that data come from? mdls seems to have output even when there are no xattrs or resource forks for a given file.

    • Jon Stacey Mar 15, 2012

      From my searching, mdls can return information about files that do not necessarily need to be stored explicitly as metadata or with a resource fork. For example, the demonstration on MacRumors:Guides shows that mdls can be used to get the width of an image. From what I remember, this is not information that is going to be explicitly stored as metadata since this attribute does not differ from what the image width actually is.

  • Paul Crowley Nov 5, 2009

    Are there any advantages of this resource fork/ data fork aspect to apples operating system over however windows do it?

    • Jon Stacey Nov 8, 2009

      I haven’t really explored the NTFS alternate data streams, so I did a quick search and ran across the following article that you might find interesting: http://www.symantec.com/connect/articles/what-you-need-know-about-alternate-data-streams-windows

      I have not spent any time using ADS on Windows, but I’ve found Apple’s implementation less than ideal. For example, not all utilities are aware of or work properly with the extended attributes. For example, the version of rsync that comes with OS X does not include full support (rsync 3 fixes this). This probably stems from the fact that not all GNU/BSD utilities support xattr , at least not yet.

      They are both approaches at solving the metadata problem. Unfortunately, there seems to be some uncertainty surrounding them (are they going to be here tomorrow?). Ideally I envision a universal filesystem incorporating metadata management that greatly increases interoperability, but this is not realistic within the next 5 years. A more achievable goal would be to either drop the idea completely or fully incorporate the capabilities into the filesystem and APIs with a pluggable translation layer to make moves between filesystems seamless.

Leave a Comment


+ three = 12