C, 336 585 611 615 623 673 707 bytes

Source code:

p[256][256]={0};char*X="p[256][256]={0};char*X=%c%s%c,Y[999],c,j,*a;main(){sprintf(Y,X,34,X,34);for(a=Y;*a;a++)p[*a][*(a+1)]++;j=*Y;while(c=j){putchar(c);for(;;j++)if(rand()>>16<p[c][j])break;}}",Y[999],c,j,*a;main(){sprintf(Y,X,34,X,34);for(a=Y;*a;a++)p[*a][*(a+1)]++;j=*Y;while(c=j){putchar(c);for(;;j++)if(rand()>>16<p[c][j])break;}}

With newlines and whitespace added for legibility/explanation:

01  p[256][256]={0};
02  char*X="p[256][256]={0};char*X=%c%s%c,Y[999],c,j,*a;main(){sprintf(Y,X,34,X,34);for(a=Y;*a;a++)p[*a][*(a+1)]++;j=*Y;while(c=j){putchar(c);for(;;j++)if(rand()>>16<p[c][j])break;}}",
03  Y[999],c,j,*a;
04  main(){
05      sprintf(Y,X,34,X,34);
06      for(a=Y;*a;a++) p[*a][*(a+1)]++;
07      j=*Y;
08      while(c=j){
09          putchar(c);
10          for(;;j++)
11              if(rand()>>16 < p[c][j])
12                  break;
13      }
14  }

The variable p[][] will contain the occurrences of each character following another. X contains the entire source, with %c%s%c instead of the assignment for X. The %c characters will be replaced with 64, the double quotation mark.

Line 06 tallies the occurrences of one character following another and starts the Markov model at the first character of the program ('p').

The while loop from 08 to 13 first outputs the current state of the Markov model, then finds the next state. A state is chosen as the next transition if the next random number is less than the number of occurrences for that state.

Sample output:

p[j]+;;*a;forintforear(c=Y;;whif(;;j,j+;j][256<pr(Y;j,X,X,cha][j+)in(cha;fora+++;cha=j=j]=*X={0}}}}"p[*ain()p[9]={s%c)p[*antc,chin(c][c%c%sp[*ar*X=Y[*a;j,34,Y;f();j+){0}

I was able to save a ton of bytes by using printf and a slower but more random approach to state switching.