After adding a retention report to a Dashboard, that retention report is now always visible in the "Add Report" modal for a Dashboard.
There does not appear to be any way to delete retention reports, or even change the name of the retention reports displayed in the "Add Report" modal. This has resulted in a long list of redundant reports with similar names and no way to clean them up.
Removing a retention report from the Dashboard does not solve this problem.