Help for 23

I've mentioned this before in the past, but I didn't want to drag up an old message thread.

I'm still noticing that after uploading my photos that already have tags with accented characters in them will have those accented characters munged in various ways. I can correct the tags by hand during upload, but the problem is that any existing tags with certain characters in them will

For example, I've just uploaded a test photo (http://www.23hq.com/clvrmnky/photo/3997546) which I tagged with some of the following keywords in Adobe Lightroom and then exported to JPEG: "Français, café, l'hôtel, naïve."

According to "Get Info" in OS X, these keywords are present (and correct) in the JPEG IPTC header. Likewise when I use a thirdparty IPTC/EXIF viewer. Once I upload the photo, however, the tags look like this in the upload form before saving:

"naã¯ve l'hã´tel cafã© franã§ais testshots testshot tests test na•ve l'h™tel cafŽ cafe butterfly "wings of paradise butterfly conservatory" "test shots" franais" (I'm not sure how these chars will end up displaying in this interface, but you get the idea.)

Note how the individual tags with extended chars are partially duplicated with the chars removed, as well as weird munged characters in the other.

Once I save the photo (without making any changes to the tags in the upload form) these munged tags will be saved to the photo in 23.

I'm not sure what to do. I'm pretty sure Lightroom is not doing anything special, and the IPTC fields are supposed to support such characters. Other applications and the OS seem to understand the contents of the JPEG header. I'm hoping that this can be remedied somehow.

Now, I tried uploading the same photo to Some Other Photo Sharing Site, and I get very similar results. Among the partially duplicated keywords, though, it actually displays the correct ones.

So, something is going on here I can't figure out, and I admit it might be the format of the JPEG. I'd just like to know what, exactly.

I know that I18n stuff can be subtle to get right, so if you need me to provide more test data or explain myself better (I'm pretty tired right now -- up late studying for a mid-term.)

Does anyone else see this?

Thanks.

Thank you for the very detailed description -- please let us know what you find out. We'd love to have another leg up on that SOPSS you mention ;-)

It may help for you to know that we're using exiftool to read EXIF and IPTC.

An update. What an ordeal trying to sort this out.

This problem looks related to IPTC fields, missing "Coded Character Set" field, and the fact that Lightroom may default to MacRoman for character data on a Mac. It appears that the improperly encoded IPTC:Keyword data is not understood by any application, especially if the Coded Character Set field is not set.

Since no app or web site can know what char set the data is in, it probably guesses UTF-8 or Latin-1, which is (of course) totally wrong.

This is a posted bug in Lightroom, which should write these out in UTF-8 and/or set the Code Character Set (at the very least) and/or give the user some control over how potentially dangerous data is stuffed into EXIF/IPTC/XMP. (I vote for ignoring IPTC altogether since it is rather outdated.)

A workaround is to postprocess the images with ExifTool to just remove the IPTC:Keywords filed completely. In fact, I just remove the entire IPTC block with a Lightroom export plugin (http://regex.info/blog/lightroom-goodies/metadata-wrangler) since most of that data is duplicated in the XMP block.

Here are some references:

- http://regex.info/blog/lightroom-goodies/flickr (search for "accented characters")
- http://www.lightroomforums.net/showthread.php?t=2313
- http://www.flickr.com/help/forum/30559/?search=lightroom