Learning Fréchet differentiable operators via prespecified neural operators

Neural operators, built on neural networks, have emerged as a crucial tool in deep learning for approximating nonlinear operators. The present work develops an approximation and generalization theory for neural operators with prespecified encoder-decoder structures, extending previous work by considering target operators that are Fréchet differentiable. To utilize the Fréchet differentiability, we expand the target operator by the Taylor formula and apply a re-discretizing technique. This enables us to derive novel upper bounds on approximation and generalization errors. We further apply these bounds to two commonly used network architectures, functional neural network (FNN) and PCA-Net, and derive improved convergence rates compared with those for Lipschitz continuous operators. These results also quantitatively demonstrate how the reconstruction errors of infinite dimensional spaces and the order of Fréchet differentiability of target operators influence learning performances. Notably, for PCA-Net, we conduct error analysis over the entire input space endowed with a sub-Gaussian measure, broadening the applicability of the theory.