When creating an identity source, there is a small difference in the stepped config for identity sources vs. most other configuration wizards in the administration console. The identity source details page has "Save and Next Step" instead of just "Next Step", like other similar pages have. There's a reason for this. In order for us to correctly capture the attributes needed on the next page of the config, we need to save the identity source. This means when you cancel on the "User Attributes" page, you won't save the changes you made on that page, but you have already saved the identity source. If you really want that identity source gone, no problem. Just delete it from the identity source list page.