#CJam, 3132 bytes
Just following the spec to the point. Running the instructions on a larger set so as to not affect the nth number.
ri:N4*N)2#,:)N{))/2/{:)~\@}%:+}/N(=
[Try it online here][1]
[1]: http://cjam.aditsu.net/#code=ri%3AN4*%2C%3Ahttp://cjam.aditsu.net/#code=ri%3AN)2%23%2C%3A)N%7B))%2F2%2F%7B%3A)~%5C%40%7D%25%3A%2B%7D%2FN(%3D&input=66%3D&input=4