Author Topic: Removing IPTC Keyword duplicates  (Read 2429 times)


  • Newbie
  • *
  • Posts: 2
Removing IPTC Keyword duplicates
« on: November 07, 2012, 07:54:29 PM »
Is there a way to use EXIFTool to remove IPTC keyword duplicates?  I am not sure how they got into my metadata, but now I believe the duplicates are causing me problems because certain programs won't let me delete keywords that are duplicates.

My followup question is can EXIFTool be configured to remove leading spaces from keywords?  Again, I am not sure how it happened but it appears that sometimes I now have what certain programs consider to be duplicates even though they are not...i.e.  " Germany" is not the same as "Germany" but programs treat them as the same.   I did not notice this issue when it happened, but (as above) now certain programs are not letting me delete or change keywords if I have (1) "Keyword" as well as (2) "LeadingSpace+Keyword"

Thanks in advance - your software is a great tool and probably my only chance to fix these issues in batch without lots of manual effort.

Phil Harvey

  • ExifTool Author
  • Administrator
  • ExifTool Freak
  • *****
  • Posts: 14596
    • ExifTool Home Page
Re: Removing IPTC Keyword duplicates
« Reply #1 on: November 08, 2012, 07:18:54 AM »
Here is a config file that will do it with this command:

exiftool "-keywords<mykeywords" DIR

Code: [Select]
%Image::ExifTool::UserDefined = (
    'Image::ExifTool::Composite' => {
        MyKeywords => {
            Require => 'Keywords',
            ValueConv => q{
                my @list = ref $val eq 'ARRAY' ? @$val : $val;
                my ($item, %found, @rtn);
                foreach $item (@list) {
                    $item =~ s/^\s+//;   # remove leading white space
                    push @rtn, $item unless $found{$item};
                    $found{$item} = 1;
                return \@rtn;
1; # end

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).


  • Newbie
  • *
  • Posts: 2
Re: Removing IPTC Keyword duplicates
« Reply #2 on: November 08, 2012, 02:55:18 PM »
Wow - that is truly amazing.  Your software is, again, great.  Thanks for saving me so much time.


  • Newbie
  • *
  • Posts: 1
  • working in the stock photo industry
Re: Removing IPTC Keyword duplicates
« Reply #3 on: July 06, 2018, 05:31:42 AM »
Thanks. This works and saved me a lot of time.