You probably didn’t purposefully pack it when you set off on your trek, but, guess what? You have a clothes drying rack with you.
If you are on the trail and have some wet gear, whether it is sunny and dry out, or even overcast (as long as it is not currently precipitating), hang your wet clothes off of your pack (socks, shirt, bandana, other layers), and let them dry in the breeze.
If you stuff them back in your pack, they will not have a chance to dry out, especially if you stuff them inside a plastic bag or other waterproof container – there is no way for the moisture to escape the wet fabric.
Then when you reach camp, toss them in the bottom of your sleeping bag overnight, and your body heat will help the drying process.
Of course, you can always use the heat from your campfire to speed up the drying process, that is, if you have a fire going.