Linear classifier is the most popular and widely used classifier for most classification problems, due to its simplicity and efficiency. The decision is made based on the value of linear combination of highdimensional feature vector, where the combination coefficients are pre-trained with training data. For good classification performance, the feature dimension should be large. This causes high test-time computational complexity, although the linear classifier is known as fast classifier. In this paper, we propose an efficient feature re-ordering algorithm for optimizing the linear classifier, so that when given a test sample, the resulting linear classifier can shortcut the summation and make fast decision without further wasteful summation operation. Experimental results on UCI machine learning dataset validates the efficiency of our method