At the heart of any atomic clock is a quartz crystal clock that is steered to match the resonance dip of the atom. It is that oscillator that actually outputs the 10Mhz, gets divided to 1PPS, etc.

Perhaps it would be possible to have a pair of identical crystals tied together but electrically opposite in phase so that external vibrations tended to cancel. (Like a differential signaling pair in a cable)