Mixed-matrix carbon molecular sieve membranes containing conventional and hierarchically structured 5A were synthesized for application in oxygen (O2)/nitrogen (N2) separation. In general, incorporating 5A fillers into porous carbon matrices dramatically increased the permeability of the membrane with a marginal decrease in selectivity, resulting in very attractive O2/N2 separation performances. Hierarchical zeolite 5A, which contains both microporous and mesoporous domains, improved the separation performance further, indicating that the mesopores in the zeolite can serve as an additional path for rapid gas diffusion without sacrificing O2/N2 selectivity substantially. This facile strategy successfully and cost-effectively pushed the performance close to the Robeson upper bound. It produced high performance membranes based on Matrimid® 5218 polyimide and zeolite 5A, which are inexpensive commercial products.