Jelly, 8 bytes

ASŒRḅ@Ðḟ

Try it online!

##How?

ASŒRḅ@Ðḟ || Full program (monadic link).

AS        || Sum the absolute values.
  ŒR      || And create the symmetric inclusive range from its negative value.
       Ðḟ || And discard those that yield a truthy value...
     ḅ@   || When plugging them into the polynomial.

Based off Luis' answer.