Gol><>, 2020 18 bytes
Credit to Jo King.
iE;:`#=q$o`;=+`~P%
How it works
iE;:`#=q$o`;=+`~P%
Accumulator (m) is stored at the bottom
iE; Take input (n) as char; halt if EOF
:`#=q If n == '#'... (preserving n on the top)
$o Output m as char, discarding it
`;=+ If n == ';', increment m
`~P% m = m % 127
Repeat indefinitely
Previous submission, 20 bytes
iE;:`#=Q$`~P%o|`;=?P
How it works
iE;:`#=Q$`~P%o|`;=?P
Accumulator (m) is always stored at the bottom
iE; Take input (n) as char, halt if EOF
:`#=Q | If n == '#'... (preserving n on the top)
$`~P%o Swap m to the top, output m%127 as char
`;=? If n == ';'... (discarding n; m is exposed to the top)
P Increment m
Repeat indefinitely