K (ngn/k), 57 bytes (Not competitive, yet)

{""/{c:"bcdfghjklmnpqrstvwxz";v:"aeiouy";(1?c),(1?v)}'!5}

Try it online!

Not particularly concise, but looks pretty well. Takes inspiration from the PHP answer. Because ngn/k's random-related functions are all deterministic, so the output isn't randomly generated (and thus the title).

Explanation:

{""/{c:"bcdfghjklmnpqrstvwxz";v:"aeiouy";(1?c),(1?v)}'!5}    Main function. Takes no input
                                                      !5     Generate a array with range from 0 to 5 ([0..5])
                                                     '       For each number...
    {                                               }        Execute a function that...
                                        ;(1?c),(1?v)         Chooses one random consonant, followed by one random vowel
     c:"bcdfghjklmnpqrstvwxz";v:"aeiouy"                     Consonant and vowel constants
 ""/                                                         Joined each of these strings together