While it's good that the pollen is spread to other plants for pollination, it is not good for it to spread to the petals, your hands or your tablecloth. To prevent the pollen from staining everything it touches orange, remove the stamens. Watch for closed blooms to open, and grab the stamens as soon as possible.
Lilium longiflorum flower – 1. Stigma, 2. Style, 3. Stamens, 4. Filament, 5. Tepal
If you find that the pollen has already dropped onto the petals, there is one trick that will remove most of the pollen, and leave very little staining. Do not use a wet paper towel! The best tool is a pipecleaner. Fold the pipecleaner in half. Using the folded end, gently drag it toward the outside, from the center, over the pollen. Voila!