Haskell - 147 142 138 characters
fi=zz.bu
bu=zz.(:).(++"zz")
[]#zz=zz;zz#__=zz
zZ%zz=zZ zz$zZ%zz
zz=(([[],[]]++).)
zZ=zipWith;z=zZz=zipWith3(((#).show)[1..]$zZ(++))(bu%"Fi")(fi%"Bu")$map show[1..]
The code is 19 characters longer than it needs to be, but I thought the aesthetics were worth it! I believe all three "objectives" are satisfied.
> take 20 z
["1","2","Fizz","4","Buzz","Fizz","7","8","Fizz","Buzz","11","Fizz","13","14",
"FizzBuzz","16","17","Fizz","19","Buzz"]