Spin-orbit torques (SOTs) in ferromagnet (FM)/Ta/CoFeB trilayers are investigated as a function of Ta thickness. When the Ta is thinner than 1.5 nm, the sign of the SOT exerting on the top perpendicularly magnetized CoFeB depends on the bottom FM layer; it is positive for NiFe and negative for CoFeB. As the Ta thickness increases, the sign becomes negative irrespective of the bottom FM, indicating that SOTs are dominated by Ta, which has a negative spin Hall angle. SOT-induced switching without an in-plane magnetic field is observed in the thickness ranges where the bottom FM or FM/Ta interface-generated SOT is dominant. The results herein demonstrate that proper design of an FM/heavy metal combination and the material thickness can lead to an enhancement of the SOT efficiency and allow for field-free SOT switching.