Buildings and other static objects like doodads (gates, archers, etc.) have something called "pathing maps" which are specific files that determine how units collide with them.
A good example of a pathing map is the arch doodad. You can walk under it, but not into either of the pillars. That is a pathing map. It's more complex.
Normal units are the ones that simply use a circle of area. If you want to fix this, I believe you'll have to change your building to a unit, or find a much larger pathing map.