Ruby 2.7, ^695...672 668 bytes

R=*0..7
L,*e='+-'*8+?+
l,*b={}
"?>=<765/.'".bytes{b[_1],b[63-_1]=A=?X,E=?O}
loop{puts'
 '+[*?a..?h]*' ',R.map{|r|L+"
|#{b[8*~r,8].map{_1||?.}*?|}| #{8-r}"},L
eval"M,*m=->c,r,w=0,*s{9.times{[x=c+u=_1/3-1,y=r+v=_1%3-1]-R==e&&(k=b[x+8*y]
w<1?k!=E&&s<<[x,y]:b[z=c+8*r]==A&&(w<2?l[A]!=z&&(#{$f='[x+=u,y+=v]-R==e&&!b[x+8*y]'}&&(k==A||k&&N[c,r]-N[x,y]!=e)&&s<<[#{$g='c,r,w,x,y'}]
x,y=c,r
s<<[#$g]while#$f):s<<k))}
w>1?(A,E=E,A;([p,A]&s)[0]):s}
N,*q=->c,r,n=M[c,r]{n==n|=n.flat_map{M[*_1]}or redo;n}
R.product(R){M[*_1,2]&&(p A;Q);m+=s=M[*_1,1];M[*_1,2]&&q+=s}
A,E=E,A
q!=e&&m=q
m==e&&(p E;Q)
($><<A+?>;#$g=gets.bytes.map{~-_1%8})until[[#$g]]-m==e
b[c+8*r],b[l[A]=x+8*y]=p,A"}

Try it online! (676 bytes because the older Ruby on TIO doesn't support numbered block parameters.) A simple example game is included that illustrates most of Quagmire's rules. Just for clarity, the input is commented and invalid moves are indented.

Input is in the format x#:y#, terminated either by newline or EOF. The input prompt is O> for player 1 and X> for player 2. (In an interactive terminal the input appears as it is typed, after the prompt.) The prompt is repeated until a valid move is entered, then the updated board is printed. When the game ends, the winning player's symbol is printed in double quotes; I like to imagine it is raising its arms in victory.

Explanation

While perhaps not a brilliant board game, it ought to be a fun code golf!

Steel your Brain

The first 4 lines perform initialisation. The x,*y=z construct is equivalent to x=z;y=[] but 2 bytes shorter. R is a fixed list of row numbers and L is a fixed string representing the edges of the board. e is just an empty array, which is used often enough that defining this single-letter variable is shorter than using []. The value of each square on the board (O, X, or nil) is stored in b. Empty squares, stored as nil, are replaced by . on output. The starting configuration is encoded by the 10-byte string ?>=<765/.', whose codepoints give the indices of squares containing X's. l is a hash that stores the index of the piece moved on each player's previous turn (which cannot be moved on their next turn). A and E store the 'active' and 'enemy' piece symbols and are interchanged each turn. As constants (uppercase names), their scope extends beyond the block in which they are defined (more about this below).

Now the main loop begins, running until one player wins. Following the 3 lines that display the board, the remainder of the program is enclosed in an eval string, which allows a couple of chunks of code to be recycled. The first 6 lines of this string contain the real meat of the program: the lambdas M and N.

Mary had a little lambda

Let's start with N. N finds the set of all squares that are connected to the square at column c and row r (0-indexed). Here, connected means that there is a path between all the squares that is not blocked by enemy pieces. N fans out from the current square, adding its neighbours and their neighbours (by repeatedly calling M with w set to 0; see below) to the set n until n no longer changes (n==n|=n.flat_map{M[*_1]}or redo). Source and destination squares must be unconnected as a prerequisite for jumping an enemy piece.

M probes the board around the square at column c and row r. Its exact behaviour is determined by the switch w, which may take a value of 0, 1, or 2:

• When w equals 0, M returns an array s of the co-ordinates of all neighbours (8-neighbourhood) of the current square that do not contain enemy pieces (k!=E). Lambda N uses this information to find the set of all squares that are connected to the current square, as described above.

• When w equals 1, M returns an array s containing all possible moves for the active player originating from the current square. For a move to be possible, the current square must contain one of the player's pieces (b[z=c+8*r]==A) and that piece must not have been moved on the previous turn (l[A]!=z). Necessary conditions on the destination square (column x, row y) are that it lies within the board ([x+=u,y+=v]-R==e, i.e. x and y are both in R) and is empty (!b[x+8*y]). Because these conditions have to be tested in two places—once for jumps and once for normal moves—to save bytes they are stored as a string in $f, which is then interpolated into the eval string. In addition to the above conditions, jumping requires either another active piece to jump over (k==A) or, if an enemy piece is being jumped, the source and destination squares must be unconnected (k&&N[c,r]-N[x,y]!=e).

  Moves are saved in the format [c,r,w,x,y] to match the input format. w is the only integer variable defined whose value is known a priori. Consequently, it is the only sensible choice for the separator character's position by the method used to process user input (see below). This fact informs the selection of : as the separator; in fact any character with a codepoint of 2 (mod 8) will work. The snippet c,r,w,x,y, which would otherwise appear four times in the code, is instead saved as a string in $g and interpolated.

• When w equals 2, M interchanges the active/enemy pieces (A,E=E,A) and then tests whether an enemy piece in the current square is in quagmire, returning a truthy/falsey value accordingly. The piece is in quagmire if there are no empty neighbours and at least one neighbour is an active piece (([p,A]&s)[0]). The reason for interchanging active/enemy pieces here is that quagmire checks are performed twice per square, once to find whether the opponent has left a piece in quagmire (and hence lost the game) and again to find whether the active player has been quagmired by the opponent's previous move.

With their uppercase names, M and N (like A, E, L, and R) are constants in Ruby. Constants are used here because both lambdas call each other; constants, unlike local variables, share (global) scope. (The scope of local variables depends on the order in which they are defined. To use local variables m and n instead, one would have to ensure that n had already been assigned some temporary value before m was defined.) The lambdas are placed inside the loop, rather than outside, so that the arrays m and q, which need to be reset after every move, can be initialised cheaply. A side effect is that the 'constant' lambdas are defined again on every pass through the loop, producing warnings on STDERR (these warnings may be suppressed by calling Ruby with the -W0 option). Similar warnings are produced each time A and E are interchanged.

Into the quagmire

The remainder of the program is relatively straightforward. On each turn, loop over all squares of the board by forming pairs of column and row indices (R.product(R)). If any square contains a quagmired enemy piece (M[*_1,2]) the game is won; print the active player's symbol (p A) and terminate by throwing a tantrum (Q is undefined). Otherwise, add all possible moves (neglecting quagmire for the moment) for the active player to m and keep a copy of possible moves for the current square in s (m+=s=M[*_1,1]). Then check whether the current square contains a quagmired active piece (M[*_1,2]); if so, add the previously saved moves in s to q (q+=s). Once the loop has finished, interchange the active/enemy pieces to advance the turn (A,E=E,A).

We now have two arrays of moves for the active player: m contains all possible moves (neglecting quagmire) whereas q contains possible moves by any quagmired pieces. If q is nonempty it takes precedence over (replaces) m (q!=e&&m=q). If the active player has no possible moves (m==e) the game is lost; print the opponent's symbol and terminate (p E;Q).

Finally it's time for some input! Until a move in m is entered (until[[#$g]]-m==e), display the prompt ($><<A+?>), convert the input to codepoints (gets.bytes), then convert these to 0-indexed column/row numbers (map{~-_1%8}). The final line updates the source and destination squares, saving the destination index in l.