05AB1E, 2323 22 bytes
Ýí€ûnR**`âOIPns-D<ds_‚O.±{Åγ¦
Try it online Outputs the pair in reversed order, so \$[boundaryCount, interiorCount]\$.
Try it online or verify all test casesverify all test cases.
Ý # Convert both values in the (implicit) input-pair to [0,value]-ranged
# lists
í # Reverse each inner list
€û # Palindromize each inner list
n # Square each inner value
R # Reverse the list
** # Multiply each to the (implicit) input-pair twice
` # Pop and push both lists separately to the stack
â # Create all possible pairs using the cartesian product
O # Sum each inner pair together
IP # Push the product of the input-pair
n # Square that
s- # Subtract the sums of the list from that value
D.± # Duplicate this list
< Get the sign (-1 if negative; 0 if 0; 1 if #positive) Decreaseof each value by 1
{ d # Check for each value whetherSort it'sthese non-negative1s, (>=0s, 0)
and s1s
Åγ # SwapThen sorun-length theencode duplicatedit, pushing list is[-1,0,1] at(which thewe topwon't againuse)
_ # Check for each value# whetherand it'sthe equallist toof 0lengths
‚ ¦ # Pair those two lists together
O Remove the first #value Sum(for the checkamount of each inner list-1s)
# (after which the resulting pair is output implicitly)
Try it online with step-by-step debug lines.Try it online with step-by-step debug lines.