We consider the problem of providing inter-access-point (AP) fairness guarantee in dense AP deployments where starvation can occur. In particular, we develop a framework for providing robust minimum throughput guarantee for each AP under the uncertainty of user distributions. Our framework consists of an AP throughput provisioning scheme and a distributed CSMA algorithm. The throughput provisioning scheme computes a robust feasible minimum AP throughput vector based on a random AP-level conflict graph and chance-constrained optimization. By incorporating the minimum throughput vector, we develop a distributed CSMA algorithm that fulfills the minimum requirement for each AP and is compatible with the IEEE 802.11 standard. We show through extensive simulations that our framework addresses the AP starvation problem by guaranteeing minimum throughput for each AP.