# Python 3, **238 bytes**

    from operator import*
    from functools import*
    from itertools import*
    f=lambda l:sum(reduce(mul,[l[a][b] for a,b in zip(range(len(l)),j)])*(-1)**(sum((sum(y<j[x]for y in j[x:]) for x in range(len(l))))) for j in permutations(range(len(l))))

[Try it online!][1]

My solution uses the definition of a determinant for calculations. Unfortunately, the complexity of this algorithm is `n!` and I can not show the passage of the last test, but in theory this is possible.



  [1]: https://ideone.com/kAUdl0