I agree on that, but depending on how much money is in the ad revenue, you might end up burning your bridges. Patreon is nice with its monthly payment system (and many users use it, so thats great too!), but ad revenue is more consistent.
Tbh, I'd try out Patreon once and see how well it goes before removing the ads. You can make it a funding tier. But remember that Patreon takes 10% of your funds, so calculate that in.