# C, 336 <del>585</del> <del>611</del> <del>615</del> <del>623</del> <del>673</del> <del>707</del> bytes

Source code:
<!-- language-all: lang-c -->

    p[256][256]={0};char*X="p[256][256]={0};char*X=%c%s%c,Y[999],c,j,*a;main(){sprintf(Y,X,34,X,34);for(a=Y;*a;a++)p[*a][*(a+1)]++;j=*Y;while(c=j){putchar(c);for(;;j++)if(rand()>>16<p[c][j])break;}}",Y[999],c,j,*a;main(){sprintf(Y,X,34,X,34);for(a=Y;*a;a++)p[*a][*(a+1)]++;j=*Y;while(c=j){putchar(c);for(;;j++)if(rand()>>16<p[c][j])break;}}

With newlines and whitespace added for legibility/explanation:

	01	p[256][256]={0};
	02	char*X="p[256][256]={0};char*X=%c%s%c,Y[999],c,j,*a;main(){sprintf(Y,X,34,X,34);for(a=Y;*a;a++)p[*a][*(a+1)]++;j=*Y;while(c=j){putchar(c);for(;;j++)if(rand()>>16<p[c][j])break;}}",
	03	Y[999],c,j,*a;
	04	main(){
	05		sprintf(Y,X,34,X,34);
	06		for(a=Y;*a;a++) p[*a][*(a+1)]++;
	07		j=*Y;
	08		while(c=j){
	09			putchar(c);
	10			for(;;j++)
	11				if(rand()>>16 < p[c][j])
	12					break;
	13		}
	14	}


The variable `p[][]` will contain the occurrences of each character following another. `X` contains the entire source, with `%c%s%c` instead of the assignment for X. The `%c` characters will be replaced with 64, the double quotation mark.

Line `06` tallies the occurrences of one character following another and starts the Markov model at the first character of the program (`'p'`).

The while loop from `08` to `13` first outputs the current state of the Markov model, then finds the next state. A state is chosen as the next transition if the next random number is less than the number of occurrences for that state.

I was able to save a bunch of bytes by using sprintf.

Sample output:

`p[j]+;;*a;forintforear(c=Y;;whif(;;j,j+;j][256<pr(Y;j,X,X,cha][j+)in(cha;fora+++;cha=j=j]=*X={0}}}}"p[*ain()p[9]={s%c)p[*antc,chin(c][c%c%sp[*ar*X=Y[*a;j,34,Y;f();j+){0}`