Flintlocks & Fireballs
While in theory the Empire could control any land in the South if it desired, some stretches simply aren’t important enough for it to waste its valuable time taming. These are the Wilds: barren wastelands so desolate that nobody in their right minds could possibly want to possess them.
Of course, that doesn’t mean that nobody lives there.
There are plenty of reasons why people would want to live outside the Empire’s grasp. Privacy is one; love of freedom is another; the belief that one should be allowed to marry one’s cousins a third.
Living somewhere as inhospitable as the Wilds is a high price to pay for this. The Wilds can barely support sustenance farming: its inhabitants mainly live as nomadic villages of hunter-gatherers, or deranged hermits hiding in shacks in the mountains.
While the absence of Imperial authority in the Wilds is attractive to the usual crowd of smugglers and outlaws — not to mention various nonhumans, the Wilds are only unoccupied because the Empire doesn’t care about them. If it were to learn of any sizable settlement there, the Imperial Legions would under normal circumstances waste no time in marching in and slaughtering everybody.
Under normal circumstances.
As it is, the Imperial Legions are severely stretched fighting the rebellion. Which means that the sorts who would normally be lurking in caverns and trying not to be noticed are now rather freer to get out and stretch their legs, wings or tentacles.