Inverse iteration for Sylvester operators

We generalize the inverse iteration for matrices to the (generalized) Sylvester operator S(X)≡AXB^⊤−CXD^⊤, computing the null space or the homogeneous solution to S(X)=0, or the eigen-spaces for the intersecting subspectrum Λ(A,C)∩Λ(D,B). Cases with two small matrix pencils in (A,C) and (D,B), a large and a small pencils, and two large pencils, as well as the special cases for the Sylvester and Lyapunov equations, and the linear equation with tensor structures, are considered. When the solution process for the corresponding Sylvester equation is robust and efficient, the generalized inverse iteration converges in one or two iterations, especially for cases of small dimensions or with semi-simple intersecting eigenvalues. For large examples, especially with derogatory intersecting eigenvalues, the approach performs less well. Illustrative numerical experiments are presented.