SHBlade has the correct formula. This is also how the bard talent and Yarr bonus works.
Also, remember that the random dungeon does drop red souls from treasure chests and bosses.
The chance for getting a red soul drop from chests and bosses rises exponentially with every new floor in Blackfire Deep.
And also remember: the chance to spawn a chest or boss also rises the deeper you go, so overall, you will eventually hit a point at which you almost get a guaranteed red soul drop on every floor.
The chance to get a red soul drop from a treasure chest in Blackfire Deep is roughly 1.5% on the first floor and rises up to a whopping 33% on floor 50.
This is why getting a Keystone is key (no pun intended) and probably much better than farming foothills bosses all day long. The blue quality keystone will grant you instant access to floor 20. At this floor, the chance to get a treasure chest is twice as high as on the first floor - and the chance to get a red soul drop from these chests is also already at ~4%.
As you can take souls outside no matter if you die or not, it's pretty much a no-risk scenario, if all you care about is getting red souls.