We shall examine the consequences of considering that the Yukawa couplings depend on the Higgs field through operators of mass dimension greater than 4, focusing mainly on the lepton sector. The motivation for this approach is to reproduce the hierarchy of the fermion masses, which in the Standard Model remains unexplained. Due to the smallness especially of the lepton Yukawa couplings, these operators can have a significant contribution to the lepton masses. In such a case, the lepton mass matrix and the matrix of the Higgs-lepton couplings can be misaligned leading to Lepton Flavor Violation (LFV) mediated by the Standard Model Higgs boson. We shall show model-independent bounds on the Higgs LFV couplings and quantify LFV in radiative decays of leptons and electric dipole moments for a class of lepton-Higgs operators contributing to lepton masses.