In the present work, employing the nonlinear field equations of a hot dusty plasma in the presence of nonthermal electrons and trapped ions, we studied the amplitude modulation of nonlinear waves in such a plasma medium by use of the reductive perturbation method and obtained the modified nonlinear Schrödinger equation. The modulational instability (MI) was investigated and the effects of the proportion of the fast electrons (α), the trapping parameter (b) and the plasma parameters such as the dust-ion temperature ratio (σd), the partial unperturbed electron to dust density (δ), and the ion-electron temperature ratio (σi) on it was discussed. For the investigation of modulational instability problems three parameters P/Q, Kmax and Γmax play the central role. The variations of these parameters with the wave number k and the other physical parameters are discussed and the possibility of occurence of modulational instability is indicated.
