Python 2, 323 bytes

exec u"def I(s,a,b=1j):c,d=s;d-=c;c-=a;e=(d*bX;x=(d*cX;return e*(0<=(b*cX*e<=e*e)and[a+x*b/e]or[]\nE=lambda p:zip(p,p[1:]+p);S=sorted;P=E(input());print sum((t-b)*(r-l)/2Fl,r@E(S(i.realFa,b@PFe@PFi@I(e,a,b-a)))[:-1]Fb,t@E(S(((i+j)XFe@PFi@I(e,l)Fj@I(e,r)))[::2])".translate({70:u" for ",64:u" in ",88:u".conjugate()).imag"})

Takes a list of vertices through STDIN as complex numbers, in the following form

[  X + Yj,  X + Yj,  ...  ]

, and writes the result to STDOUT.

Same code after string replacement and some spacing:

def I(s, a, b = 1j):
    c, d = s; d -= c; c -= a;
    e = (d*b.conjugate()).imag; x = (d*c.conjugate()).imag;
    return e * (0 <= (b*c.conjugate()).imag * e <= e*e) and [a + x*b/e] or []

E = lambda p: zip(p, p[1:] + p);
S = sorted;

P = E(input());

print sum(
    (t - b) * (r - l) / 2
    
    for l, r in E(S(
        i.real for a, b in P for e in P for i in I(e, a, b - a)
    ))[:-1]
    
    for b, t in E(S(
        ((i + j).conjugate()).imag for e in P for i in I(e, l) for j in I(e, r)
    ))[::2]
)

Explanation

For each point of intersection of two sides of the input polygon (including the vertices), pass a vertical line though that point.

(In fact, due to golfing, the program passes a few more lines; it doesn't really matter, as long as we pass at least these lines.) The body of the polygon between any two consecutive lines is comprised of vertical trapezoids (and triangles, and lines, as special cases of those). It has to be the case, since if any of these shapes had an additional vertex between the two bases, there would be another vertical line through that point, between the two lines in question. The sum of the areas of all such trapezoids is the area of the polygon.

Here's how we find these trapezoids: For each pair of consecutive lines, we find the segments of each side of the polygon that (properly) lie between these two lines (which might not exist for some of the sides). In the above illustration, these are the six red segments, when considering the two red vertical lines. Note that these segments don't properly intersect each other (i.e., they may only meet at their end points, completely coincide or not intersect at all, since, once again, if they properly intersected there would be another vertical line in between;) and so it makes sense to talk about ordering them top-to-bottom, which we do. According to the even-odd rule, once we cross the first segment, we're inside the polygon; once we cross the second one, we're out; the third one, in again; the fourth, out; and so on... In other words, if we group the segments into consecutive pairs, each pair is the legs of one of the trapezoids.

Overall, this is an O(n³ log n) algorithm.