In this paper, a study of the effect of a transparent dielectric medium (soda-lime glass placed after an objective lens) on femtosecond pulse filamentation in Corning Eagle(2000) glass is presented. The dielectric medium placed after the objective lens increases the effective numerical aperture of the focusing lens and tightly focuses the beam to elevate the power density in a tiny focal volume. A single-shot narrow filamentary void of diameter 3.8 mu m is observed to propagate as long as 700 mu m in Corning Eagle(2000) glass. Elevated optical intensity confined in the small focal volume and low ionization losses per unit length in the narrow plasma column collectively enhance the filamentary propagation of the ultra-short pulse in bulk glass. The applied technique also offers an approach to control the length and width of a filamentary void structure in a transparent dielectric medium.