In task-oriented service computing, a user's computing goal is modeled and represented as a task, which is composed of activities that are performed by accessing service instances in a local environment. The abstract service requirements specified in an activity of a task are resolved and bound to service instances dynamically in runtime. When there are many candidate services that provide similar capabilities for a task, it is essential to consider quality of service (QoS) such as response time, latency, and availability to determine which service instances to use. Finding a service composition that meets the optimal level of quality is a well-known NP-hard problem-the time complexity for task-level (global) optimization increases exponentially as the number of services and the number of quality attributes increase. Although it is possible to use a heuristic approach that shows a reasonable response time with a certain level of service quality, this strategy often fails when there are hard QoS constraints that need to be considered in the task level. In this paper, to overcome this limitation, we propose an adaptive method of selecting services based on the hardness of QoS constraints. The basic idea is to sample services that represent a specific quality-value range. The quality-value range of candidate services is divided into smaller sub-ranges in which representative services are sampled and evaluated. At this time, the size of the QoS sub-ranges is determined adaptably based on the hardness of the QoS constraints. In a QoS sub-range, candidate services may have a similar QoS value for a quality attribute. We calculate the utility of candidate services in a QoS sub-range and sample the highest utility service. This process of sampling services and evaluating their utility value is repeated until it makes a composite service that has the highest level of global utility for a task. Our experiment results show that the proposed approach effectively improves the success rate of service composition while achieving a certain level of global optimality and maintaining a reasonable level of performance. Our approach shows up to 80 percent improvement in success rate in comparison to the existing heuristic approaches.