In constrained optimization problems, evolutionary algorithms often utilize a penalty function to deal with constraints, which is, however, difficult to control the penalty parameters. This paper therefore presents a new constraint handling scheme. It adaptively defines an extended-feasible region that includes not only all feasible solutions, but some infeasible solutions near the boundary of the feasible region. Furthermore, we construct a new fitness function based on stochastic ranking, and meanwhile propose a new crossover operator that can produce more good individuals in general. Accordingly, a new evolutionary algorithm for constrained optimization problems is proposed. The simulations show the efficiency of the proposed algorithm on four benchmark problems.