The extreme ceiling height in that room probably has much to do with the discomfort, especially in winter because you're heating all that unoccupied space up there, and the heat rising to that space causes cool air to be drawn in from any of several possible sources, thus possibly creating the drafts you feel.
Much also depends on how the space is heated and cooled: location of grilles and diffusers, air temperatures and velocities. lots of factors, even the location of thermostats.
Attic insulation is vital, and the goal should be to get as much up there as you possibly can fit and afford. Then, spend some time and money making certain that the house is well-sealed by caulking and sealing around all windows and doors, installing storm windows whereever there is single-glazed windows, and paying close attention to things like electrical outlets in exterior walls as a source of drafts.
If you have access in the basement or crawl space, as the case may be, be sure the floor above is insulated, and that there is insulation along the rim joists, and be sure that any vapor retarder is UP against the subfloor and not visible in the space below. Experienced insulation installers often get this wrong.
Architect (NY) and Home Designer (PA)