[rkbrown419]

You could apply a percentage of the long term fatigue suffered as a penalty to Will or IQ. I'd also suggest being agressive with the penalties for things like poor sleep from being in a leaky tent in the rain or poor nutrition because the food you're carrying or foraging for isn’t a ballanced diet.
You might also penalize rolls for natural or physician assisted healing while on the trail. It's harder to keep wounds clean when camping as opposed to staying in a permanent structure.
There is also a social dynamic to consider. People are likely to be more suspicious of strangers camping at the edge of town than of travelers who stay at the local inn. The former are likely to be regarded as homeless bums, unless they are part of an army (which can lead to a whole different set of issues). The latter will be regarded as more legitimate travelers passing through.