This is a read-only archive. Find the latest Linux articles, documentation, and answers at the new Linux.com!

Linux.com

Feature: Graphics & Multimedia

The tricky task of supporting Photo CDs on Linux

By Nathan Willis on January 03, 2008 (4:00:00 PM)

Share    Print    Comments   

In the photography world, a prominent proprietary file format is Kodak's Photo CD (.PCD). Once the premiere format for film scanning, it is now a difficult-to-work-around relic. Recently I set out to resurrect some old PCD images on a Linux system -- a challenge that serves as an object lesson in the importance of open standards in any kind of digital archive.

For years PCD was the preferred target format for professional photo labs scanning slides and negatives. It was the output of high-end, all-in-one scanning systems built by Kodak, supporting all types of film from APS size up to large-format.

Although Kodak no longer sells the equipment that labs used to create PCD files, a great deal of information about the format is still available on the company's Web site. The format itself is unusual, which is part of why it is rarely supported by third-party software.

Each PCD file is multi-resolution, with a Base size, Base/4 and Base/16 reductions, and Base*4, Base*16, and (optionally) Base*64 high-resolution copies all embedded into a single file. In spite of the names, Base*4, Base*16, and Base*64 sizes are not algorithmic enlargements of Base; the highest-resolution image (whether it is Base*16 or Base*64) is the native scan. The top end gives the equivalent of 6.3 megapixels for Base*16 and 25 megapixels for Base*64.

And there is another wrinkle. The varying sizes are not stored separately like the thumbnails inside JPEG files. Instead, each of the lower resolutions is a subset of the pixels contained in the native-size image.

PCD also uses its own encoding method, PhotoYCC -- a system similar to YCbCr but that can represent luminance data above 100% brightness. In that sense, you might think of it as a precursor to the high-dynamic-range formats in wide use today. PhotoYCC defines 100% brightness in terms of a reference device, independent of the scene photographed.

An unofficial Windows solution

The extra latitude that encoding method provides is helpful for an image with extremely bright (e.g., glowing) objects. But the peculiarity of the design choice fooled a lot of applications and libraries, which incorrectly clipped the PhotoYCC data and produced TIFF and JPEG conversions with blown-out highlights.

Ted Felix was unhappy enough with the contemporary conversion offerings that he waded through Kodak's white papers and technical documentation and figured out a way to correct the poor conversion. He modified the Windows .DLL that Kodak supplied to software companies for loading PCD images, directly altering the look-up table that scales the luminance values.

Felix's patched library will work as a drop-in replacement for the original, turning a variety of software alternatives from highlight-destroyingly-useless into dependable conversion tools.

His site includes a list of the supported apps, all but one of which are commercial, and none of which are open source. The list does not include Adobe Photoshop, which in Felix's tests produced blown-out highlights like most of the competition, but which uses a different conversion routine and is therefore not fixable with his patched DLL.

Linux-friendly options

One supported Windows app is IrfanView, a lightweight and free (but closed source) image viewer praised by many photographers for its accuracy. IrfanView does run under Wine, so if you have no other way to access a PCD image on a Linux system, it is an option.

A native solution is preferable, though, and that's where it gets difficult. Years ago there was hpcdtoppm, a command-line utility for converting PCD images to Portable Pixmap (PPM) format. It was included in the Netpbm package, but several distros (including, notably, Debian, Ubuntu, and SUSE) removed it from their versions of Netpbm because of its restrictive licensing.

If your distro does include hpcdtoppm, you can check whether the included version suffers from the blown-highlights problem with either your own image or with one of Felix's test PCDs. Felix links to a patch for hpcdtoppm, so if the converted output is bad, consider applying it. But beware, you might have to manually edit the code rather than apply the patch with patch if your version of hpcdtoppm differs considerably.

The more reliable solution is ImageMagick (IM), which is actively maintained and a standard component in almost all Linux distros. IM can convert a PCD file with convert image001.pcd image001.tiff.

I found IM's conversions subjectively too bright in the highlights -- although not as bad as some of the other offenders documented by Felix. The conversion is done in magick/colorspace.c using the YCCMAP table about halfway into the file.

If you plot the YCCMAP table as a function, you can see where the highlight compression occurs. Essentially, the compressed look-up table performs a gamma-correction when converting from PhotoYCC to RGB, and it uses the same gamma correction for every image. Felix's replacement look-up table is linear, so none of the highlight information is compressed.

I discussed the subject with the IM developers, who say they tested their look-up table against Kodak's reference image and got expected results.

Obviously conversion factors like curves are the type of thing photographers can argue the merits of all day long, but when converting from an uncorrected scan into a work-mode image, I prefer not to lose any information. Luckily with open source you can make the changes yourself.

If you patch colorspace.c to use a linear look-up table, you can apply any necessary gamma-correction in the image editor, and make it specific to the image contents and usage. I created a linear YCCMAP look-up table, which you can apply to colorspace.c using this patch. It is diff'ed against IM's trunk, which you can check out from anonymous SVN. I am happy with the results it provides -- to be certain, your mileage may vary, but at least Photo CDs are a stable optical media, so you don't have to worry too much about ruining your only copy of the file.

What to learn

I became interested in PCD support because of one of the scenarios described in the sidebar: I wanted to use an image I liked, but for which the original was lost and the PCD was my best remaining scan. But the process got me thinking about digital archives in general, and the critical importance of the archive format.

PCD was not flawed technically, it just was a single-vendor format, and when Kodak lost interest in maintaining it, it was stone cold dead. Even while it was alive, though, many software makers shipped a faulty implementation of the conversion utility for it, in spite of the public documentation (Ted Felix slogged through Kodak's own support docs, remember). And once it was no longer in common usage, the knowledge of how to properly interpret it was soon lost, and bugs started creeping into even the open source projects like ImageMagick.

There is a moral to heed in that story. Today, the major digital camera manufacturers each use a single-vendor RAW format, in most cases poorly documented. How easy will it be to access those .NEF and .CR2 files 10 years from now? Even Adobe's camera-maker-neutral DNG format, whose specification is publicly available, is written and released by Adobe alone. I'm reasonably certain that Adobe will still be around in 10 years, but I can't say the same for DNG. I just hope ImageMagick is still here, too.

Share    Print    Comments   

Comments

on The tricky task of supporting Photo CDs on Linux

Note: Comments are owned by the poster. We are not responsible for their content.

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 63.135.47.118] on January 03, 2008 10:13 PM
Your argument applies equally well to the whole ODF-OOXML format war. I always tell people to save their documents in ODF format, or, if they must use Microsoft formats, to use the old binary (doc, xls, etc) formats. Most non-technical people just shrug and go on using their shiny new Office 2007 and its ooxml formats, but it won't be my problem when they lost all their files in 10 or 20 years.

#

Re: The tricky task of supporting Photo CDs on Linux

Posted by: bricks on January 07, 2008 03:07 PM
Could not agree with this more. I think the common PC user has no clue what a file format even is. Someone recently mentioned to me after someone sent them a .docx file from Word 2007 that they were going to have to go buy Office 07 now because they needed to be able to open these files when people sent them. People just have no clue. <a href="http://www.javasigns.com/">Vinyl Banners</a>

#

Re: The tricky task of supporting Photo CDs on Linux

Posted by: Nathan Willis on January 08, 2008 10:18 PM
I suppose there are parallels, but there's also one huge (to me) difference: Kodak was not malevolently keeping PCD closed or trying to quelch competitors, in fact it was quite the opposite. Thus, even in the absence of vendors *trying* to lock users into a proprietary/closed/evil/sad format, users can still end up with no way to access their information. -- Nate

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 70.91.34.61] on January 04, 2008 04:07 PM
This is a great article. I had several CDs using the Kodak photo cd format, and I thought they were the cat's meow. I especially liked that I could choose the image size as I opened it. Thanks for providing information on opening those files with Linux applications.

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 67.84.159.91] on January 04, 2008 10:46 PM
http://linux.bytesex.org/fbida/xpcd.html has a command line tool to extract to ppm or jpg. The X app won't compile for me anymore (Fedora 8).

#

Re: The tricky task of supporting Photo CDs on Linux

Posted by: Nathan Willis on January 06, 2008 12:54 AM
Yeah; I had the same trouble here. That's why I cut xpcd from the story -- seems like no one has been able to get it working in quite some time. Which is a shame in one sense, although it's also survival of the fittest at work, I suppose.

-- Nate

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 68.43.220.46] on January 07, 2008 05:20 AM
Xpcd may no longer compile, but the command-line utility pcdtoppm certainly does, and it can
still be used to convert PCD files.

However, using pcdtoppm on Felix's test image (BlownJohn.pcd) causes the blown out highlights,
and so its use may be problematic.

The latest netpbm no longer includes hpcdtoppm for the reasons cited in the article, but it can be
downloaded from the following link and compiled separately:

http://www.ibiblio.org/pub/Linux/apps/graphics/convert/

Using the patch mentioned in the article, hpcdtoppm converts Felix's test file without blowing out
the highlights.

I'll have to re-convert some of my old PCD disks that I still have using the patched hpcdtoppm.

Thanks for an informative article.

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 58.179.212.57] on January 07, 2008 12:10 PM
Excellent article. My father is a typesetter, and was when Kodak introduced PhotoCD - back in those days, Kodak thought that the format was going to be adopted by consumers who wanted to look at photos on their TV. He was probably the first person to tell them that the market they should be targeting was print production, and the rest is history.

I often wondered what happened to PhotoCD, but I had no idea it was such a complicated specification, and I also was unaware that open-source software had such trouble with it. It has certainly raised my awareness of how proprietary formats can ultimately bite their user on the behind.

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 58.179.212.57] on January 07, 2008 12:18 PM
My father was a typesetter, and was present when Kodak announced the PhotoCD at a conference in Australia. Back then, Kodak positioned PhotoCD as a consumer device, for people who wanted to look at photos on their TV. My father would've been one of the first people, if not the first, to tell Kodak that PhotoCD would be much more suitable for pre-press. The Kodak rep's response? "We don't think PhotoCD has real application in the printing industry". The rest is history. But it has certainly raised my awareness of the dangers of trusting your data with proprietary technology that could fall out of favour. The few people who bought home Photo CD players have probably learnt this lesson too.

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 58.179.212.57] on January 07, 2008 12:20 PM
(sorry for the double-post; Songbird crashed while posting the message the first time, and it didn't appear on the page until after I wrote the second comment)

#

Links to fixed binaries?

Posted by: Anonymous [ip: 210.17.193.132] on January 08, 2008 05:53 PM
Links to conversion-fixed binaries would be welcome here, since this is where many folks will end up after searching about Linux and PhotoCDs. Perhaps even a brief guide for using the CLI tools since many new Linux users may be unfamiliar with the terminal. And yeah, I've got a few PhotoCDs as well. Never even realized they had better scans than the smallish jpegs hidden inside a proprietary tortilla. I guess that's why I stopped getting them... Well done Kodak! :-)

#

Re: Links to fixed binaries?

Posted by: Nathan Willis on January 08, 2008 10:13 PM
Binaries will get out of date as IM is continuously developed; the patch will not. Well, at least not until IM rewrites convert from scratch or somesuch. -- Nate

#

Re(1): Links to fixed binaries?

Posted by: Anonymous [ip: 210.17.203.181] on January 10, 2008 08:57 PM
Well releasing a generic fixed pcdtoppm, or one for major distros, would go a long way toward helping non-experts use their PhotoCDs. Or maybe it's just better to hope that ImageMagick will include this fix to some upcoming release and wait till distros pick it up.

But the article is great for helping hackers do it. Vintage Linux.com material.

#

The tricky task of supporting Photo CDs on Linux

Posted by: Anonymous [ip: 192.168.1.241] on January 09, 2008 01:03 PM
Thanks Nat! At last some real article here, tired of "securing joomla" stuff.

#

This story has been archived. Comments can no longer be posted.



 
Tableless layout Validate XHTML 1.0 Strict Validate CSS Powered by Xaraya