So, out in Den, Colorado, the slave trade is absolutely booming. Slaves from nearby tribal villages and towns like Boulder and Five Families are always coming in and out, and honorable entrepreneurs have decided to start working in this business. To protect slavers from harsh treatment by local villages, a Slavers Union has formed. But, the SU needs an HQ, and they are too lazy to build one themselves. So they decide to use one of the pre-war buildings, but it has to fit these criteria.
Must have infrastructure capable of keeping people restrained and stop them from escaping.
Must have the ability to survive for about 5 decades unmaintained.
Must be easy to protect, with good defenses to stop unwanted people or creatures from getting in.
I currently work in corrections. The Denver Jail would suffice. Jails that are decades old are often still in use today. A county lockup nearby where I live was built in the 60's, aside from the occasional new coat of paint and some new doors installed it is still in full use housing about 50 inmates. Jails are already built like fortresses, and wouldn't be looted since jails contain very little in the way of anything valuable, and are also pretty hard to get into. Best part, the entire layout and infrastructure of the building is totally designed for holding unwilling captives and deterring nosy outsiders.
I suggest the Denver department of corrections prison building. Its big enough to hold all the merchandise but also fortress-like enough for your aspiring indentured servitude merchants to protect themselves and their profits. A prison after the apocalypse would basically be a free castle for whomever has the manpower and gumption to take it and use it as such.
While the jail may be the best choice, there is a possibility that either the Duke of Denver isn't willing to give it up as his fortress, or it was destroyed or partially destroyed in the Apocalypse.
There are some other possibilities that come to mind. The most likely is to find a records repository. These buildings are fortresslike by design, and house the records of governments or corporations. Insurance companies, in particular, have or at least had massive repositories for their records and contract paperwork. The building may have few windows or doors, was originally climate controlled, and built out of concrete in order to be fireproof or at least fire resistant, which gives it a good possibility of surviving many types of apocalypse. New residents will need to get rid of all the paper inside. The Colorado State Archives in Denver is located at 1313 Sherman St, Denver CO 80203.
Another structure which is likely to be overlooked is old telephone switch exchanges. They were built to house the electromechanical relays for the old rotary dial phones (the series of clicks you hear when dealing a rotary phone was these switching elements turning into place). Large urban centres like Denver would have had large buildings devoted to this purpose, and they would also be strongly built both to protect the equipment, provide a stable base for the machinery, and house the climate control equipment and things like emergency generators. Modern phone switching is done by computer, which takes much less space, but you can often see these mostly empty buildings still in use, with the racks of old switches long removed. The Telephone Building at 931 14th Street in Denver is a likely candidate.
A bank of two motion switches. A telephone switch exchange could have huge racks of these filling the building
The Telephone Building
Finally, and more speculatively is the tunnel complex under the Denver airport. Dug during the construction to house the various services and especially the complex (and ultimately unsuccessful) baggage handling system. Although there are plenty of people who believe the tunnels are used to house other things, including the secret headquarters of the New World Order, after the apocalypse the most likely issue with using the tunnels is they will be filled with vermin, mould or perhaps flooded. Any surviving members of the New World Order likely escaped the apocalypse with Elon Musk and now live in tunnels under Olympus Mons on Mars....
Cheyenne Mountain Complexhttps://en.wikipedia.org/wiki/Cheyenne_Mountain_Complex
This is an extreme superfort; a giant subterranean bunker and command center deep enough to survive nuclear attacks. It has its own water. It is about an hour south of modern Denver, outside Colorado Springs. As far as a slaver castle, one could use the entirety of the subterranean component for slave pens and use the above ground pieces as defensive works. The setup with the mountain road as the only approach reminds me of medieval castles. Good luck with any frontal assault on Cheyenne Mountain.