#05AB1E, 48 bytes

g≠iā<DδmUεXøINǝø}Xšεā<sUœε©2.ÆíÆ.±Xε®Nèè}«P}O}ć÷

Sometimes 05AB1E's lack of almost all matrix builtins is pretty annoying.. ;)
Inspired by @Arnauld's JavaScript answer.

Try it online or verify almost all test cases (removed the last two largest ones, since they time out on TIO).

Explanation:

First handle the edge case of a single-element input-list (would cause issues with the « later on in the code):

g                # Get the length of the (implicit) input-list
 ≠i              # And if it is NOT 1, continue with:
                 #  ... (see below)
                 # (implicit else:)
                 #  (output the implicit input-list as implicit output)

Next we'll get the exponentiation matrix of the list [0, input-length):

ā                #  Push a list in the range [1, (implicit) input-length] (without popping)
 <               #  Decrease each value by 1 to make the range [0, input-length)
  Dδ             #  Apply double-vectorized on itself by first duplicating:
    m            #   Take the power of the two values
     U           #  Pop and store this exponentiation matrix in variable `X`

Next we'll create a list of this matrix, with every column one by one replaced with the input-list:

ε      }         #  Map over the input-list that was still on the stack
 X               #   Push the exponentiation matrix from variable `X`
  ø              #   Zip/transpose it; swapping rows/columns
     ǝ           #   Replace the transposed row of the exponentiation matrix
    N            #   at the current map-index
   I             #   with the input-list
      ø          #   And then zip/transpose back

We'll prepend the original exponentiation matrix to this list:

Xš               #  Prepend the matrix `X` in front of this list

And we'll calculate the determinant of each inner matrix in this list:

ε              } #  Map over the list of matrices:
 ā               #   Push a list in the range [1, matrix-length] (without popping)
  <              #   Decrease it by 1 to make the range [0, matrix-length)
   sU            #   Swap to get the matrix again, and pop and store it in variable `X`
     œ           #   Get all permutations of the [0, matrix-length) list
      ε          #   Inner map over each permutation:
       ©         #    Store the current permutation in variable `®` (without popping)
        2.Æ      #    Get all 2-element combinations of this permutation
           í     #    Reverse each inner pair
            Æ    #    Reduce it by subtracting
             .±  #    And get it's signum (-1 if a<0; 0 if a==0; 1 if a>0)
       X         #    Push the matrix from variable `X`
        ε        #    Map over each of its rows:
         ®       #     Push the current permutation of variable `®`
          Nè     #     Get the value in the permutation at the current map-index
            è    #     And use that to index into the current matrix-row
        }«       #    After the map of rows: merge it together with the signum list
          P      #    And take the product of this entire list
      }O         #   After the map of permutations: sum all values

Now that we have all determinants of the matrices, we get the default one again to divide all others by it:

ć                #  Extract head: pop and push remainder-list and first item separated
 ÷               #  Integer-divide each value in the remainder-list by this head
                 #  (after which the result is output implicitly)