#Java 8, 90 bytes

s->{for(char c=96;++c<123;s=s.matches(".*"+c+".*"+c+".*")?s:s.replace(c+"",""));return s;}

Explanation:

Try it online.

s->{                         // Method with String as both parameter and return-type
  for(char c=96;++c<123;     //  Loop over the lowercase alphabet
    s=s.matches(".*"+c+".*"+c+".*")?
                             //   If the String contains the character more than once
       s                     //    Keep the String as is
      :                      //   Else (only contains it once):
       s.replace(c+"",""));  //    Remove this character from the String
  return s;}                 //  Return the modified String

Java 8, 90 bytes

s->{for(char c=96;++c<123;s=s.matches(".*"+c+".*"+c+".*")?s:s.replace(c+"",""));return s;}

Explanation:

Try it online.

s->{                         // Method with String as both parameter and return-type
  for(char c=96;++c<123;     //  Loop over the lowercase alphabet
    s=s.matches(".*"+c+".*"+c+".*")?
                             //   If the String contains the character more than once
       s                     //    Keep the String as is
      :                      //   Else (only contains it once):
       s.replace(c+"",""));  //    Remove this character from the String
  return s;}                 //  Return the modified String