One possible cause for this issue in newer versions of Retera Model Studio is that the newer camera manager system handles the "FarZ" clipping of models differently, so in certain cases it is possible for the default view for a model to be too far away so that you cannot see the model. If this happens, you can hold down CTRL and then Scroll Down on the Mouse Wheel and it will zoom the "FarZ" further away, possibly making it easier to see models that might not have been showing in the view.
Other than that issue, I usually use "File -> Open" without much issue, so I haven't figured yet what kind of problem you might be hitting.
If the CTRL+Scroll Down solution does not work for you, try saving a normal WC3 unit model that you can open to some arbitrary file, then close Retera Model Studio and open it again, then use "File -> Open" on the unit you saved. If that much still works, the issue might be with the specific custom model you are interested in, in some way.