Text of any kind is what increases loading times the most(It's an enormous difference, as seen in my benchmarks).
For buffs, I use a combination of indexing(every unit has a doubly linked list; available indexes are in a stack) and actual abilities.
How do I determine stacks? Level of ability.
How do I determine when to remove the visual effect? Level of ability.
Debuffing? Each buff type has a trigger for it that can be called like a function.
Duration and actual effects are handled by whoever uses the system.
If you want to use this system, then I can provide you all the requirements. For me the thinking required is too much atm though, so I don't use it myself. When I figure out the full picture I'll probably rewrite it to be more comfortable to use.