I think this goes in line with what Dr Super Good said.
You can just make them trainable units, and for each city, keep track of the number of upgrades it has and each type. You could have an array holding some representation of a city, and each member has fields for each type of upgrade and the number of each researched so far.
When a player trains a unit, i.e. tries to make a 4th upgrade when already has done 4, detect this and simply refund the player, with an error message as Dr Super Good suggested.
This doesn't gray them out, however. Instead if you wanted a physical indication, you could change the city's look through various triggers. If you went really good, each upgrade would add on the bank, hospital, etc. visibly placed on the unit.