Pyth, 1212 9 bytes
e@.*m-SdeS@Fm.-m/d{
Try it Saved 3 bytes thanks to Mr. Xcoder.
Explanation
e@.*m-SdeS@Fm.-m/d{
m Sd /d Count the Sortoccurrences Aof andeach Belement.
-.m .-d{ Take Removeonly those that appear the duplicatedminimum elementsnumber fromof eachtimes.
@.* @F Apply the above Taketo theA intersection.
eand B and take the intersection.
eS Take the last (largest).