Replacing current Apple Watch service with an Apple Watch purchased from Apple

It's done automatically when you set up the new watch it's really easy and nice.

If it was me I would start the setup of the new watch with your iPhone and follow the prompts until it starts asking who your cellular carrier is.

Your iPhone and att might recognize that it is a new watch

and might ask you if you are replacing your series 4.

If it doesn’t and it just seems like  it’s just setting up another watch with another additional service account with LTE access then cancel the setup.

Or heaven forbid call att and have them Remove your old device from the watch account

they might even be able to help you manually add the new watch to the watch account with the one or two of the serial or ID numbers on your series 6 box.

If the first option works please remember to be patient and wait for the setup to complete. I know it takes a while when setting up a new watch with LTE access.

And make sure your phone is connected WiFi for a strong connection between your phone and the incoming and outgoing info. 
I hope this helps in some way.

Yes you can replace the 4 with the 6.

   If you no longer have the E SIM card for your series 4, you may need to acquire a new e SIM card from AT&T. I have no idea whether they're going to charge you if you have to replace the card.  

Liz apple watches don't use esim cards.

Thanks.  Could you chime in on how service is transferred? 

Thanks.  Wow Android needs work.  Smh