Improving the adversarial attack transferability, or the ability of an adversarial example crafted on a known model to also fool unknown models, has recently received much attention due to their practicality in real-world scenarios. However, existing methods that try to improve such attack transferability craft perturbations in a deterministic manner. Thus, adversarial examples crafted in this manner often fail to fully explore the loss surface and fall into a poor local optimum, suffering from low transferability. To solve this problem, we propose Attentive-Diversity Attack (ADA), which disrupts diverse salient features in a stochastic manner to improve transferability. We first disrupt the image attention to perturb universal features shared by different models. We also disturb these features in a stochastic manner to explore the search space of transferable perturbations more exhaustively and thus to avoid poor local optima. To this end, we use a generator to produce adversarial perturbations that each disturbs features in different ways depending on an input latent code. Extensive experimental evaluations demonstrate the effectiveness of our method, outperforming the transferability of state-of-the-art methods.