Invulnerable units can be attacked and this is easy to detect, though the attacks deal no damage, which means that you need to deal the damage by SetUnitState instead, and if the new life value is below 0.405 (or 0), then just remove the invulnerability and deal the actual damage from its source. Thus the solution consists of three parts:
A) Detecting when an invulnerable unit is attacked - this can be done through checking for example by buff (such as Divine Shield) if there's specific source of invulnerability you want to be affected, or by BlzIsUnitInvulnerable() function.
B) Dealing with the attack's damage by calling the Unit is about to take damage -event and change the target unit's life value by SetUnitState() function. If the new life value is below 0.405, then you'd want the damage source to kill the unit for real to allow bounty and experience gain (along the other factors).
C) Enabling units to attack invulnerable targets - this can be done through the SetUnitWeaponIntegerField() function. To find the correct integer value to enable the attacks against invulnerable units, I'd recommend using the call GetUnitWeaponIntegerField() function on a unit type that can by default target invulnerable units - for example a melee unit that can attack invulnerable units has integer value of 3146042 in the mentioned integer field.