Jelly, 8 bytes

ASŒRḅ@Ðḟ

Try it online! or as a test-suite!

##How?

ASŒRḅ@Ðḟ || Full program (monadic link).

AS        || Sum the absolute values.
  ŒR      || And create the symmetric inclusive range from its negative value.
       Ðḟ || And discard those that yield a truthy value...
     ḅ@   || When plugging them into the polynomial (uses base convertion).

Based off Luis' answer. An alternative.