#MATL, 21 bytes
35,5Y216,j6L)!16ZA-)h
###Example
This has been executed on Octave:
>> matl
> 35,5Y216,j6L)!16ZA-)h
>
> #FFAA20
#0055DF
###Explanation
35, % nunber literal: ASCII code of '#'
5Y2 % '0123456789ABCDEF'
16, % number literal
j % input string
6L) % remove first element
! % transpose
16ZA % convert from hex to dec
- % subtract from 16
) % index into '0123456789ABCDEF' to convert back to hex
h % concatenate 35, which is '#'