Python 3 - 506506 479 bytes
q=range
def p(n):
for d in q(2,n//2+1):
if n%d==0:return 0
return 1
w=int(input())
m=w**2
z=len;m=w**2;z=len(str(m))
def k(n):
if n==1:return'-'*(z-1)+'>'
if p(n):return str(n).zfill(z)
return'.'*z
l=[k(n+1)for n in q(m)]
s=[[0for];s=[[0for n in q(w)]for n in q(w)]
x=y=w];x=y=w//2
t=[n2;t=[n//2for n in q(2,w*w)]
c=[[1];c=[[1,0],[0,-1],[-1,0],[0,1]]
j=0
p=01]];j=p=0
for r in t:
d=c[p%4]
p+=1
i=0d=c[p%4];p+=1;i=0
while i<r and j<m:
s[y][x]=l[j]+' '
x+=d[0]
y+=d[1]
i+=1
j+=1';x+=d[0];y+=d[1];i+=1;j+=1
s=[''.join(l)for l in s]
for l in s:print(l)
I am not sure if the byte count is correct.
Also, this can most likely be made shorter (I'm new to Python).
Right now I'm just substituting all non-primes with periods. You can change this to any character.
Note: You should only enter odd numbers as input, as otherwise the program breaks.
Furthermore, my function to determine whether a number is prime is the most primitive it can be, for the sake of less characters, so it is not efficient. Though it ran just fine on my computer with input values less than 100 in just under a second.
Note: Input values above 25 or 30 won't display correctly unless you have a ridiculously wide screen.