Java 8, 133 bytes

a->b->{long r;for(java.util.Collections c=null;;a.remove(r))if(b.contains(r=c.max(a))&c.frequency(a,r)*c.frequency(b,r)==1)return r;}

Explanation:

Try it here.

a->b->{                  // Method with two ArrayList<Long> parameters and long return-type
  long r;                //  Result-long
  for(java.util.Collections c=null; 
                         //  Create a java.util.Collections to save bytes
      ;                  //  Loop indefinitely
       a.remove(r))      //    After every iteration, remove the current item
    if(b.contains(r=c.max(a)) 
                         //   If the maximum value in `a` is present in `b`,
       &c.frequency(a,r)*c.frequency(b,r)==1)
                         //   and this maximum value is unique in both Lists:
      return r;          //    Return this value
                         //  End of loop (implicit / single-line body)
}                        // End of method