Can I find a customer by their phone number in the admin?

Yes. Go to Customers → Manage Customers, type the number into Search customers… and click Search.

The search looks at the customer's name, email and phone number, and matches part of the text, so a few digits are enough. Type the number without the country code, because the code is stored separately from the phone number.

Click the customer's name to open their account, where the Orders tab has an Open orders → button listing everything they have bought.