Drug promiscuity is one of the key issues in current drug development. Many famous drugs have turned out to behave unexpectedly due to their propensity to bind to multiple targets. One of the primary reasons for this promiscuity is that drugs bind to multiple distinctive target environments, a feature that we call multi-modal binding. Accordingly, investigations into whether multi-modal binding propensities can be predicted, and if so, whether the features determining this behavior can be found, would be an important advance. In this study, we have developed a structure-based classifier that predicts whether small molecules will bind to multiple distinct binding sites. The binding sites for all ligands in the Protein Data Bank (PDB) were clustered by binding site similarity, and the ligands that bind to many dissimilar binding sites were identified as multi-modal binding ligands. The mono-binding ligands were also collected, and the classifiers were built using various machine-learning algorithms. A 10-fold cross-validation procedure showed 70-85% accuracy depending on the choice of machine-learning algorithm, and the different definitions used to identify multi-modal binding ligands. In addition, a quantified importance measurement for global and local descriptors was also provided, which suggests that the local features are more likely to have an effect on multi-modal binding than the global ones. The interpretable global and local descriptors were also ranked by their importance. To test the classifier on real examples, several test sets including well-known promiscuous drugs were collected by a literature and database search. Despite the difficulty in constructing appropriate testable sets, the classifier showed reasonable results that were consistent with existing information on drug behavior. Finally, a test on natural enzyme substrates and artificial drugs suggests that the natural compounds tend to exhibit a broader range of multi-modal binding than the drugs.