Always leave a small margin of tolerance when manually measuring. For example if a hole measures 40mm x 20mm, I’d CAD it at ~40.3mm x 20.3mm.
If you can find data sheets for component panel measurements, then I CAD them exactly to these measurements, and manually measure the actual component for a double check, don’t blindly go straight from the data sheets without checking. Make a dummy cardboard panel, cut the holes out with a scalpel and mount your components on this for a triple check, screws and all.