A driver may use a Bluetooth device provided the phone is not being held by the driver, and the use of the phone does not require the driver at any time to press anything on the body of the phone, or to otherwise manipulate any part of the body of the phone.
In essence, you can drive with your smart watch connected to your phone, provided that you are not also breaching any of the Australian Road Rules under the Road Traffic Act 1961.
This includes touching, holding or resting the phone on one’s body. The driver can touch the Bluetooth device while driving if necessary (but not the phone itself).
Rule 300(1) Australian Road Rules under Road Traffic Act 1961.