Haskell, 211 197 bytes

m(')':r)=([],r)
m s|(a,u)<-e s,(b,v)<-m u=(a:b,v)
f s|(a,r)<-m s=(tail(a>>"`"):a>>=id,r)
e('\\':v:_:s)|(a,r)<-f s=('\\':v:a,')':r)
e('(':s)=f s
e(v:s)=([v],s)
main=getLine>>=putStrLn.fst.f.(++")")

Initially I had a function to perform the whole transformation, but now that is inlined into the main function which makes it a complete program. Using getLine and putStrLn also avoids the need to type and read double backslashes. The other functions follow the (simplified) pure functional parsing pattern of returning a parsing result and the unparsed rest of the string. f parses a list of expressions (possibly a multiple application) . It uses a helper function m that also parses a list of expressions and has a list of parses as result. e handles the remaining cases: lambdas, parenthesized expressions, and variables. Instead of generating a parsing tree, we directly generate the desired output.

Compile with ghc -o lam lam.hs and give input to stdin:

$ ./lam
\f.(\x.xx)\x.f(xx)
\f`\x`xx\x`f`xx

Haskell, 211 197 186 bytes

(Saved 10 bytes by using interact as suggested by @Challenger5, and one more by not having a final newline.)

m(')':r)=([],r)
m s|(a,u)<-e s,(b,v)<-m u=(a:b,v)
f s|(a,r)<-m s=(tail(a>>"`"):a>>=id,r)
e('\\':v:_:s)|(a,r)<-f s=('\\':v:a,')':r)
e('(':s)=f s
e(v:s)=([v],s)
main=interact$fst.f.(++")")

Initially I had a function to perform the whole transformation, but now that is inlined into the main function which makes it a complete program. Dealing with a complete program instead of using a function String -> String at the interpreter also avoids the need to type and read double backslashes. The other functions follow the (simplified) pure functional parsing pattern of returning a parsing result and the unparsed rest of the string. f parses a list of expressions (possibly a multiple application) . It uses a helper function m that also parses a list of expressions and has a list of parses as result. e handles the remaining cases: lambdas, parenthesized expressions, and variables. Instead of generating a parsing tree, we directly generate the desired output.

Try it online!

Using interact ruined clean I/O behaviour. If you don't use the TIO link but run it at the command line of your computer, you have to make sure that there is no trailing whitespace, it would be treated like a variable. You can do something like echo -n '\f.(\x.xx)\x.f(xx)' | ./prog. Alternatively, you could replace the last line by the previous version main=getLine>>=putStrLn.fst.f.(++")").

Haskell, 211 bytes

m(')':r)=([],r)
m s|(a,u)<-e s,(b,v)<-m u=(a:b,v)
f s|(a,r)<-m s=(concat$tail(a>>"`"):a,r)
e('\\':v:_:s)|(a,r)<-f s=('\\':v:a,')':r)
e('(':s)=f s
e(v:s)=(v:"",s)
t s|(a,_)<-f(s++")")=a
main=getLine>>=putStrLn.t

The last line makes it a complete program, if that is really wanted. It also avoids the need to type and read double backslashes. t is the functon that does the transformation. The other functions follow the (simplified) pure functional parsing pattern of returning a parsing result and the unparsed rest of the string. f parses a list of expressions (possibly a multiple application) . It uses a helper function m that also parses a list of expressions and has a list of parses as result. e handles the remaining cases: lambdas, parenthesized expressions, and variables. Instead of generating a parsing tree, we directly generate the desired output.

Compile with ghc -o lam lam.hs and give input to stdin:

$ ./lam
\f.(\x.xx)\x.f(xx)
\f`\x`xx\x`f`xx

Haskell, 211 197 bytes

m(')':r)=([],r)
m s|(a,u)<-e s,(b,v)<-m u=(a:b,v)
f s|(a,r)<-m s=(tail(a>>"`"):a>>=id,r)
e('\\':v:_:s)|(a,r)<-f s=('\\':v:a,')':r)
e('(':s)=f s
e(v:s)=([v],s)
main=getLine>>=putStrLn.fst.f.(++")")

Initially I had a function to perform the whole transformation, but now that is inlined into the main function which makes it a complete program. Using getLine and putStrLn also avoids the need to type and read double backslashes. The other functions follow the (simplified) pure functional parsing pattern of returning a parsing result and the unparsed rest of the string. f parses a list of expressions (possibly a multiple application) . It uses a helper function m that also parses a list of expressions and has a list of parses as result. e handles the remaining cases: lambdas, parenthesized expressions, and variables. Instead of generating a parsing tree, we directly generate the desired output.

Compile with ghc -o lam lam.hs and give input to stdin:

$ ./lam
\f.(\x.xx)\x.f(xx)
\f`\x`xx\x`f`xx