Enhancing supervised surface reconstruction through implicit weight regularization

Regularization plays a crucial role in stabilizing neural network training and improving generalization. In this work, we introduce the Square Highway (SqrHw) network, a novel variant of the Highway network that implicitly regularizes weight updates via a modified affine transformation. We analyze the effectiveness of the proposed regularization technique in supervised function approximation for surface construction tasks. These tasks involve large, structured datasets and clear evaluation metrics, which allow for a clearer observation of the method's performance. Our results show that SqrHw achieves superior convergence and improved stability in weight updates, outperforming both plain MLPs and cutting edge benchmark weight normalization techniques. Additionally, we conduct principal component analysis and gradient analysis to gain deeper insights into how the proposed network enhances robustness. The proposed architecture has broad applicability beyond supervised surface reconstruction, extending to fields where MLPs are commonly employed. The implementation is available at: https://github.com/AmirNoori68/SqrHw.git