Auxin response factors (ARFs) have long been a research focus and represent a class of key regulators of plant growth and development. Previous studies focusing genes from limited number of species were unable to uncover the evolutionary trajectory of this family. Here, more than 3,500 ARFs collected from plant genomes and transcriptomes covering major streptophyte lineages were used to reconstruct the broad-scale family phylogeny, where the early origin and diversification of ARF in charophytes was delineated. Based on the family phylogeny, we proposed a unified six-group classification system for angiosperm ARFs. Phylogenomic synteny network analyses revealed the deeply conserved genomic syntenies within each of the six ARF groups and the interlocking syntenic relationships connecting distinct groups. Recurrent duplication events, such as those that occurred in seed plant, angiosperms, core eudicots and grasses contributed the expansion of ARF genes which facilitated functional diversification. Ancestral transposition activities in important plant families, including crucifers, legumes and grasses, were unveiled by synteny network analyses. Ancestral gene duplications along with transpositions have profound evolutionary significance which may have accelerated the functional diversification process of paralogues. Our study provides insights into the evolution of ARFs which will enhance our current understandings for this important transcription factor family.