Upgrade an Phone Line to an Apple Watch

Yes.  Of course.  

If you have other phones on the account, you can upgrade the phone line to a watch line for the $10 a month plus tax, plus the cost of the device itself. If this is going to be the only device on your account, the minimum plan is the starter plan, and you still need an iPhone in order to set up the device as a standalone

When I tried to upgrade the phone line in the att website there is no apple watch among the choices. A while ago I also called the customer service and they said that only phones can be upgraded for the ready to upgrade phone lines. Do I have to go to an physical At&t store to do this upgrade?

You would not be able to upgrade it online. It's going to insist on staying a phone line. 

You probably will have to go to a store or call.  Either with a watch that you have purchased,  have the line converted to a watch line and then purchase a watch.

If you want to keep the phone number that is now associated with a phone and have it used on the watch, please insist that they not cancel the line, and that they just convert the line to a watch line