# [JavaScript (Node.js)], 80 bytes

<!-- language-all: lang-javascript -->

    x=>g=(y,i=0,j=0,v=x[i],d=v-y[j],t=d>y[j+1]-v)=>1/v?g(y,i+!t,j+t)+!t*(d>0?d:-d):0

* It run in O(|X|+|Y|): Every recursion run in O(1), and it recursive |X|+|Y| times.
  * `x`, `y` are passed by reference, which do not copy the content
* `1/v` is falsy if `x[i]` is out of range, truthy otherwise
* `t` -> `d>y[j+1]-v` -> `v+v>y[j]+y[j+1]` is false as long as following conditions meet. And which means `y[j]` is the number closest to `v` in `y`
    * `v` is less than `(y[j]+y[j+1])/2`, or
    * `y[j+1]` is out of range, which would convert to `NaN`, and compare to `NaN` yield `false`
        * that's why we cannot flip the `>` sign to save 1 more byte
* `t` is always a boolean value, and `*` convert it to `0`/`1` before calculating

[Try it online!][TIO-jj5cv6l8]

[JavaScript (Node.js)]: https://nodejs.org
[TIO-jj5cv6l8]: https://tio.run/##ZcpBDoMgFATQfW/RHZRPC7WmrcnHgxAWRtRAjDSVED09xXUXk5lknu9St/Zf94l8CXbII@YN1YRkB4cCfEnCTTsDFhPftTcQ0aoymDQ8UVTyltrp4OwcwbNIS1@IVaK1Dbe0EbkPyxrm4TqHiYxES6jhbSjRFTzgaSg9/YE7VAcQ8Cp3/gE "JavaScript (Node.js) – Try It Online"