In Blizzard's consistent style, two variables of the same type are usually fine when you set them like A = B. A stays A, and B stays B. If you turn A into C afterwards, B doesn't turn into C, it stays as B. If you destroy A, B is still there.
For Unit Groups and Player Groups however (and i'm guessing other things), when you set a logic like A = B, they both become contingent on each other. In some sense, they essentially become A = A (or B = B) and if you modify one of them, the other is also affected. In your case, if you destroy it, you destroy both of them. Even if you cleared your temp group instead of destroying it, you would also inadvertently clear SideHumans. If you added another player to TempGroup, that player would also be added to SideHumans.
I wonder if anyone knows why it might work like that
What you are referring to is basically the difference between value semantics and reference semantics. If you know a language that has pointer/reference types and value types, like C++, then the parallel is easy to see.
If you think of a JASS variable as being like a piece of paper, we can make up some analogies which make the difference apparent.
With value semantics, we might be working with a value type like integer. If you start with two blank pieces of paper and write 5 on one of them, then one of the papers would have 5 and the other would be blank. If you did the equivalent of "set blank_paper = paper_with_5_on_it", that would be like you just writing 5 on the blank paper. Now both papers would have 5 on them. You could even do arithmetic with one of the papers and change its value to something else, like 13. Then you would have a paper with 13 and a paper with 5. It is apparent that changing one paper does not affect the other, just as you would expect.
With reference semantics the situation can change. Bring up two new pieces of paper, and now we will write something that sounds more reference-like, such as the address to your house. Write the address of your house on one paper and you will end up with a blank paper and a paper with your house's address on it. Copy the address to the other paper and you end up with two papers that have your house's address on it. So far this is like the above scenario with numbers, modifying one paper does not modify the other, just as you would expect. The situation becomes different if you give one of your papers with your address on it to your local friendly house demolishers guild, however. Both papers still have your house's address on it, but the difference is that they both now refer to a destroyed house. Acting on the house referred to on one paper can mean changing the same house another paper refers to, sounds reasonable right?
I imagine the situation in this post would be more apparent if it happened to a tangible type, such as a unit. If you had one variable that tracked your unit, then assigned another variable to it called temp_unit, then decided to do some operations on temp_unit and then clean it up by destroying the unit, it would more readily be apparent what is happening. Assigning temp_unit to your other variable did not make a copy of your unit. The unit goes away whether you called for its untimely demise with temp_unit or the other variable. As a matter of fact, most variable types in JASS have this same kind of behavior. There are only a few exceptions (like integer, real, and boolean, which you do not need to clean up).
You basically should never assume something new was introduced that you need to clean up with a Remove/Destroy call, if all you did was assign one variable to another variable. Assigning a unit variable to 10,000 other unit variables still would be just one RemoveUnit call when it comes down to it.