Adobe Photoshop Computer Graphics

Re: Centralized x Distributed cache in Bridge.

Joseph Chamberlain, DDS wrote:
> On 12/9/05 2:47 AM, in article
> 1134125249.323369.56800@g49g2000cwa.googlegroups.com, "Barry Pearson"
> wrote:
>
> > That appears to mix up two concepts: the location of the cache
> > (centralised or distributed); and the location of ACR settings and
> > other metadata (database or sidecar XMP files or within a DNG file).
> > The former is a Bridge preference. The latter is an ACR preference
> > combined with a choice of file-type.
[snip]
> Isn't the purpose of .xmp sidecar files to hold or store "all" information
> that relates to the raw images ?

The XMP metadata holds information that should have a long association
with the image. The cache is mainly to speed up Browser or Bridge. For
example, the cache, I believe, holds the thumbnails in a rapidly
accessible form. The cache can be purged, because the thumbnails can be
regenerated from the DNG later.

> My impression was that sidecar files held raw conversion information created
> by Camera Raw but also held information attached to the raw images by Bridge
> such as sort order, keywords and other metadata.

Not all of that. For example, the sort-order isn't held with the image
itself, because if you hand that file to someone else, the sort order
is meaningless.

The distributed cache for Bridge is held in 2 files in each folder,
(not per image):
Adobe Bridge Cache.bct
Adobe Bridge Cache.bc

There is sometimes another file in a folder, (not per image), and from
its name appears to be related to the sort order:
.BridgeSort

The XMP sidecar is in text form, and is worth having a look at to see
what is there. If you open a DNG file with Word, (or presumable any
text editor or other word processor), you should see the same text in
it, if it is holding XMP metadata. (Word won't show the sensor data, of
course! But the XMP metadata in in clear form).

> Although I am tempted to use DNG and find the idea the best approach to
> working with RAW images, I must confess that at this time I am a little
> skeptical of committing my RAW files to the DNG file format and the DNG
> converter. I am afraid that either the file format itself or the converter
> may corrupt my images and prevent me from extracting my original RAW files
> in the future should I need to do so.

You need to be comfortable with what you do. I've been doing this for
several months with success, but I am not going to try a "hard sell" to
persuade you to do this!

I do my own test on ACR and the DNG Converter when they are released,
and don't adopt them until I am satisfied that they work OK. I don't
just download and start using them. I check that ACR appears to give
the sort of results I expect. And I test the following routes, to
ensure that the DNG Converter itself is not screwing things up. (I test
by using blending mode "difference" on the two versions, and verifying
that all pixels are zero).
PEF > ACR > PSD (layer 0)
PEF > DNG > ACR > PSD (layer 1)

[snip]
> I remember reading Bruce Fraser's book on Camera Raw for Photoshop CS2 and
> he mentions the two options. One to have a centralized cache or database
> with all the metadata that applies to the RAW files or to have a distributed
> one where each sidecar file is placed in the same folder where the image it
> is related to is located. This is the database I am referring to.

What is important is that the "database" and the "cache" are different
things. You can actually safely delete the cache, (or copy your DNGs
without exporting the cache), and still retain your settings.

But I have just looked at page 70 of Bruce's CS2 book, at "Export
Settings". And I believe he has made a mistake! He says "when you have
the Camera Raw Preferences set to save edits in the Camera Raw cache".
Go and have a look at Camera Raw Preferences, at the drop-down box for
the location of settings - it (correctly) says "Camera Raw database".
You have a perfect right to be confused.

The database is a file called "Database" in "Documents and
SettingsApplication DataAdobeCameraRaw".

The central cache appears to be in the "Documents and
SettingsApplication DataAdobeCameraRawCache" folder.

[snip]
> How can one export in a single sidecar file "ALL" settings and information
> that has been applied to an image so that in case this image and the
> corresponding sidecar file are transferred to another system Bridge and
> Photoshop CS2 will open the image with the same settings that were applied
> to the image in the first system ?
[snip]

Data that is specific to one image, and needs to be retained even if
the cache is lost or purged, is held in the database or sidecars or
inside the DNG itself. Hence edits, settings, etc. If you hold it in
the database, you then need to export it so that it can be copied
across.

Data that can be thrown away and remade, or relates to a set of images,
such as thumbnails & previews, or sort parameters, is held in the
cache. They don't need to be copied across, or don't really make sense
to copy across.

Some data appears to vary according to file type: for example, I notice
that ratings are held in the DNG file as XMP metadata. I suppose that
makes sense, but I believe that in some cases ratings are held in the
cache. (?)

It is not a simple picture.

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/

Article References :

Centralized x Distributed cache in Bridge.

See Also : Re: RAW images

Hecate wrote:
> On 20 May 2005 00:16:50 -0700, "Barry Pearson"
> wrote:
> >
[snip]
> The problem with DNG is keeping your files in that format, apart from
> the fact that you're relying on the algorithms that the DNG converter
> uses.

What "algorithms"? I'm informed that the sensor data in my DNGs is a
binary copy of the sensor data in my camera's Raw files. AFAIK that is
always the case. The DNG Converter adds extra data, so that the Raw
processor doesn't need its own knowledge of that camera model, but it
doesn't alter the original sensor data.

> Storing in a format that is new, proprietary (whatever Adobe
> says) and not widely used isn't sensible. Camera manufacturers are
> never going to support because it would cut earnings from their own
> proprietary formats, and unless Adobe were to make it widespread by
> say, giving the code to the OSF, it's not going to become any sort of
> "standard".

AFAIK TIFF 6.0 became a de-facto standard without Adobe giving any code
to anyone. DNG will become a de-facto standard faster, both because of
the DNG Converter and because of the more rapid adoption by non-Adobe
products, about 25 at last count.

I too would like to see a non-Adobe DNG Converter, especially one built
from open source. But it hardly needs Adobe to give any code to anyone!
There is already a lot of source around for reading and writing
TIFF-like files. There is Dave Coffin's source for decoding Raw files
for 150+ cameras. Anyone wanting to know what Adobe puts in the DNG
files it creates can look at the specification and/or reverse-engineer
individual DNG files. Surely the real advantage would arise from having
an independent version?

I think DNG is becoming widely used. Surely millions of DNG files are
created every day? Both from the DNG Converter, and from people saving
as DNG from ACR 3.x. (DNG now has a similar position in Adobe's Raw
workflow that PSD has in Adobe's photo-editing).

Which camera manufacturers have earnings from their own proprietary
formats? Nikon does, with Nikon Capture (etc), perhaps $100. But I
thought the others were free? I didn't pay extra for the software with
my camera. I didn't think Canon charged extra? Raw processing software
appears to be a cost, not a profit, for the camera manufacturers. I
discussed this at the following - I won't repeat it here:

http://avondale.typepad.com/rawformat/2005/04/canon_and_other.html#c5234772

But why would use of DNG interfere with this anyway? The only reason
for a camera manufacturer not to use DNG is if they intend to COERCE
their customers to use their software by encryption. If they don't
intend to use encryption, there is no disadvantage in using DNG,
because they know their formats are going to be reverse-engineered and
supported by other Raw processors anyway.

(Hasselblad and Leica have said they will support DNG. Given the niche
position of these companies, and the financial problems at Leica, this
isn't a major step forward, but it it shows that there isn't a
universal anti-DNG position among the camera manufacturers).

> As a corollary, I won't trust Adobe to produce the information my
> camera has recorded, exactly as it has been recorded because it isn't
> *really* open source.

The post below from John Francis explains how DNG relates to the
camera's own Raw format. He is speaking from experience of examining
the results, not just theory.

http://tinyurl.com/9rmnn

My own tests were less definitive, but still useful. (See the thread
below John's). I obtained several RAW image files from a number of
cameras. I input each into Photoshop CS. I converted each of the RAW
files into DNG using the converter. Then I input those versions as
separate layers in the first set of Photoshop files, and used
"difference". There was zero difference across every pixel in all the
cases I tried. As far as Photoshop CS is concerned, there is no
detectable difference between an original RAW file and a corresponding
DNG file. Obviously that didn't rule out the possibility that both the
Converter and Camera Raw were corrupting the sensor data in exactly the
same way! But that would really be talking about bugs, not any plot to
distort the data.

And why not have the same worries about the camera manufacturers'
software? If you save your Raw file from Nikon Capture, might that not
corrupt the data too? Is Nikon known to produce more reliable software
than Adobe? (And why, if I rotate a D2X NEF file and save it, do I get
a smaller file? Is this because Nikon Capture does better compression
than the D2X?)

Canon have just announced that they sometimes even LOSE images in the
EOS-1Ds Mark II and EOS-1D Mark II cameras! Who CAN we trust?

Some people simply embed the original Raw file in their DNGs, so that
they can extract what the camera wrote later. This combines full safety
with the convenience of a single file. (As long as you trust Adobe not
to corrupt the original file during this process).

--
Barry Pearson
http://www.barry.pearson.name/photography/
http://www.birdsandanimals.info/