[TboneS54]

Removing swirls (polishing paint) is mostly about looks. A more perfect surface will better accept protection thus better resisting contamination, but I think it'd be minor.

By never polishing the paint, you'd have more paint left to sacrifice, if need be. The paint being marred, however, wouldn't show how deep the clear is - you'd need to polish to show that off.

Finishing products (wax, polymer, ceramic) can only highlight the current condition of the paint. The deep gloss comes from polishing, not so much the topping product.

There are no issues with what you propose, other than unsightly marring that only some of us really sweat over.