We studied the interface changes of niobium (Nb) Josephson junction structures (Nb/Al/Nb and Nb/AlOx‐Al/Nb) using secondary ion mass spectrometry (SIMS). Using a cesium (Cs+) primary beam, CsM+ ions (M=H, O, OH, Nb, and Al) were monitored for cesium bombardment. In the SIMS depth profile, the CsH+ and CsO+ intensity indicated accumulation at junction interfaces, where these intensities were larger for the Al/Nb base electrode interface than for the Nb counterelectrode/Al interface. We think a layer of absorbed water vapor with a thickness a few nm, is formed on the junction interface, and is attributed to chamber outgassing. In the Nb/Al/Nb structure, the annealing changes for the Al/Nb base electrode interface were smaller than those for the Nb counterelectrode/Al interface. We confirmed that the absorbed water vapor layer plays an important role in the junction structure, that is, it acts as a barrier for grain boundary diffusion between Nb and Al.