Sci-Fi Writers Have No Sense Of Scale: Okay, well, first, disclaimer. I am bound to screw something up here, eventually. Despite my best intentions and dedication to at least trying to run the numbers before committing myself to paper, to err is human, and indeed, to err is probably sophont. Mea maxima culpa, and all that, but c’est la vie. Point it out at the time and I’ll fix it.
But I have, in general, at least tried to avert this, or at least provide explanations rooted in my Minovsky Physics. Space is bloody huge. The only thing that makes interstellar travel practical in the first place is the combination of extremely long lifespans, fusion and antimatter torches (with all the ensuing consequences as to how much power said ships have available, and what that means for interstellar warfare), and the availability of exotic materials (muon metals) to let you crank your lighthugger up to high c-fractions without lethally irradiating yourself; and secondarily, the existence of wormholes which first have to be towed into position by the aforementioned lighthuggers. This makes expansion c-bound, and slow.
(Which is to say that all the mighty congery of civilizations that is the Associated Worlds and its Wonder of the Known Galaxy, the stargate plexus, nonetheless occupy an insignificantly small fraction of one galactic arm; and the Elsewhere Society‘s headline project aside, other galaxies are right out. This is also why I handwave a relatively life-rich universe into being; if life was not common in the universe, no-one would have found any yet given these parameters.)
And even then, space within solar systems is still bloody huge. In-system travel by starship takes days to weeks (relatively low-thrust by SF standards – although high-thrust by present-day Earth standards – reaction drives or vector-control drives, and even if they weren’t, no-one is crazed enough to try to hit even low c-fractional speeds in-system), and interstellar travel even by wormhole incurs these delays for every system through which you pass. (And even planets are huge; granted, it’s hard to get lost on a raw planet because even on a “friendly” garden world, it’s unlikely to be so Edenic that survival doesn’t require power, which shows up very handily from space. But on any civilized planet… well, if you’re looking for someone who doesn’t want to be found and you don’t already have their address, have fun hiring detectives. Lots of them.)
(As a military side-note, “engagement range” starts at millions of miles out for spinal mounts and AKV duels, and goes down to thousands of miles. Formations use hundred- or thousand- mile spacings. Anything under twenty thousand miles is “knife fight” range, and unless you’re engaged in a fixed-point battle for a stargate, planetary orbit, or suchlike, should be avoided at all costs because engaging at that range with anything but absolute surprise is almost always a matter of mutual annihilation for ships. It often is for fixed installations, too, but another reason why you don’t want to close with them is that they can mount lots more generator, screen, and gun when they don’t have to move ’em about.)
To avoid spending pages Showing My non-distance Work In Detail, just a couple of points:
Time-wise, while the Empire does have a multi-millennial history, lots of things happened over those millennia, and I like to think I’ve been fairly good at staying out of unexplained stretches of historical deep time. There is, of course, the Fermi Paradox/simultaneity problem, but… not sure there’s a way to play in Space Opera without that one.
To comment on one major material problem: neither of the canonically-detailed Dyson Spheres are the impractical solid type. The Esilmúr facility is a fabric bubble supported by light pressure and the solar wind; that at Corícal Ailek is only semi-complete, but is intended to be a dynamic/modular structure that’s never intended to be rigid under stress.
And, yes, I know exactly how much energy powering things like lighthuggers up to 0.9c would take. That’s why facilities like Esilmúr exist; to manufacture antimatter in quantities that have a lot of zeroes in them. They have a good idea how dangerous this is, too.