Si-nanocluster (nc-Si) luminescence and Er$^{3+}$ luminescence in silicon-rich silicon nitride (SRSN), which consists of nc-Si embedded inside a silicon nitride matrix are investigated. First, we report on the origin of visible light emission by comparing the photoluminescence (PL) properties of SRSN with those of similarly prepared silicon-rich silicon oxide (SRSO), which consists of nc-Si embedded inside a SiO$_2$ matrix. Based on the dependence of PL intensity and peak position on the film compostion, anneal temperature and hydrogenation, we identify that visible luminescence originates from nc-Si. The results indicate that nitride passivation of nc-Si leads to increased PL energy over SRSO. Second, we further investigate the role of nitrogen passivation of nc-Si by continuously tuning the passivating dielectric from pure silicon dioxide through oxynitride to pure silicon nitride while keeping the size of nc-Si and the fabrication procedure constant. We find that the observed nonlinear variation of the nc-Si luminescence peak energy agrees well with theoretical predictions about the effect of oxygen coverage on nc-Si bandgap. Such results support that strain relief by nitride passivation results in suppressing the nc-Si bandgap reduction. Finally, we report on Er$^{3+}$ luminescence in SRSN. We find that nc-Si in Er doped SRSN can act as efficient sensitizers for Er$^{3+}$, allowing off-resonant broadband pumping of Er$^{3+}$. Comparison with a similarly prepared Er doped SRSO shows that nitride offers both more effective nc-Si sensitization and higher optical activation of Er than oxide.