Rotations are expressed as quaternions, 4d vectors that represent points on a hyper sphere. That being said, you don't really need to know how quaternions work since they are 4D, which means they are not really human- readable. If you want to create rotations, you can use some angle converter, like
this one. Note that the quaternions in the converter are not in the same order as quaternions in rotations of Warcraft 3 nodes. Quaternions from the converter have to be rearranged in this order: { q2, q3, q4, q1 }
If you experience scaling, shearing or all sorts of weird effects, it means the quaternion isn't normalized, or in other words, its length isn't equal to 1.
I don't recommend animating entire models with War3 Model Editor; I only use it to make simple linear rotations, since animating is much easier with other programs (e.g. MdlVis) that allow you to animate with a graphical interface.
Some examples of linear rotation:
Around the X axis, clockwise
0: { 0, 0, 0, 1 }
2000: { 0.707107, 0, 0, -0.707107 }
4000: { -1, 0, 0, 0 }
6000: { 0.707107, 0, 0, 0.707107 }
8000: { 0, 0, 0, -1 }
Around the Y axis, clockwise
0: { 0, 0, 0, 1 }
2000: { 0, 0.707107, 0, -0.707107 }
4000: { 0, -1, 0, 0 }
6000: { 0, 0.707107, 0, 0.707107 }
8000: { 0, 0, 0, -1 }
Around the z axis, clockwise
0: { 0, 0, 0, 1 }
2000: { 0, 0, 0.707107, -0.707107 }
4000: { 0, 0, -1, 0 }
6000: { 0, 0, 0.707107, 0.707107 }
8000: { 0, 0, 0, -1 }
Rotations in the examples above look like this:
0 - 0 degrees
2000 - 90 degrees
4000 - 180 degrees
6000 - 270 degrees
8000 - 360/0 degrees
As you can see, rotations need to have a key every 90 degrees or less.