This all could be done. If you were to give players the option of doing quests as solo or budding up in teams of 2, 3 or more, you'd need to put measures in place to ensure certain quests instances don't interrupt with other quest instances.
Say you have a group of 2 doing a quest, a group of 1 doing the same quest and a group of 4. That's 3 quests instances running simultaneously. For some quests this could be doable, but for others, you'd need to prevent multiple instances of the same quest happening.
Simultaneous Quest: There could be a lumberjack quest. You simple hit a tree 10 times for some timber then return it to a shop. This quest could have multiple instances at once, because the actions of someone else quest wouldn't interupt a different quest instance.
Restricted Quests: Some quests wouldn't beable to have multiple instances (well not practically anyway). I can't imagine you placing 2 boss battles in the same room with 2 different teams and somehow manage to make each fight independent of one another. You would need to make a system that only allows a player (or one team) to do this quest at a time. E.g. everyone would have to wait for the team currently doing the quest to finish before they can begin.
I don't have the time to write the system and I'm not sure where you can find a premade one. However, it would involves a great use of local variables. Locals can be used to perform actions on certain players. Careful using locals, they are prone to desyncs. Here is a tutorial on how to properly use locals in various scenarios:
http://www.hiveworkshop.com/forums/world-editor-help-zone-98/getlocalplayer-faq-224876/
Perhaps with some research, you can create your own system?