Find AoE target abilities that is the same as the one you like to create [like rain of fire, blizzard etc] and use them as base, or base your spell on the hero ability "Channel" which will allow you to make an AoE selector but you will need to trigger whatever your ability should be doing...
and yes the editor is weird... Blizz even have time to put (NYI) [not yet implemented] tags into some data on the gameplay constants which are not functional when they could have just not added those options on the first place... they could have used the time adding those NYI things to fix some stuff instead... there's also some spells that bug when used as base for custom abilities (like cloud of fog which only accepts the data of the original ability)...