What is the difference between Unsubscribe and Remove on my subscriber list?

Both stop your campaigns from reaching that person. On Marketing → Newsletter Subscribers:

• Unsubscribe keeps the address in your list with the status Unsubscribed. Campaigns always skip it, and you keep a record that they opted out. You can click Resubscribe if they ask to come back.
• Remove deletes the address from your list completely after you confirm.

For someone who asked to stop receiving emails, Unsubscribe is the better choice.