Well, this sugguestion comes from the stock behavior of phpbb... In PHPBB when you setup a group, you can assign a group owner, this owner (or owners) has (have) the ability to add users into their group; pretty simple.
To up it an antie, users can also request to join groups, whereas the group owner would have the ability to accept or deny the request.