A telescope control system relies on a pointing model to determine the gimbal angles that aim the telescope toward a desired target. High-accuracy telescope pointing models include parameters that describe the mount/telescope orientation as well as common mechanical effects. For professional telescopes, calibrating the pointing model requires careful initial alignment around a nominal orientation (e.g., leveling) followed by sightings of dozens to hundreds of stars to fit the model parameters. While this approach is effective for observatories, applications such as transportable optical ground stations for communications, space situational awareness, or astronomy using low-cost telescope networks can benefit from a more rapid calibration approach. We formulate a quaternion-based pointing model that utilizes measurements from an externally mounted star camera to compromise between calibration speed and accuracy. A key aspect of this formulation is that it is completely agnostic to the orientation of the telescope/mount so that no manual prealignment is required. We derive angle and rate commands for telescope pointing and tracking based on the model. We present results from a 15-min calibration procedure on a very low-cost telescope that demonstrated pointing to an accuracy of 53 arc sec RMS in azimuth and 66 arc sec RMS between 20-deg and 70-deg altitude.