If I change a product's price or delete the product, do my past orders change?

No. When an order is placed, each line saves its own copy of the product name, SKU, quantity and price. Editing the product later, including its price, doesn't change orders already placed, their totals or their invoices.

Deleting a product doesn't remove it from past orders either. In Orders → Manage Orders, the line still shows the saved name, SKU, quantity and price.

One small difference: sales reports look up the product's current name, so once a product is permanently deleted from Trash, its past sales may show under a short product ID instead of its name.

If you only want to stop selling something, consider setting it to Draft instead of deleting it.