R, 57 bytes

function(s,t=s%in%utf8ToInt("aeiou"))s-t*32*!cumsum(t)%%2

Try it online!

Input is vector of character codes (for comparison to pajonk's answer). Would be 77 bytes with input as string (for comparison to Maël's answer).

Edit: this approach would be only 48 bytes by porting dingledooper's great modulo trick...