Damage alarming and safety evaluation using long-term monitoring data is an area of significant research activity for long-span bridges. In order to extend the research in this field, the damage alarming technique for bridge expansion joints based on long-term monitoring data was developed. The effects of environmental factors on the expansion joint displacement were analyzed. Multiple linear regression models were obtained to describe the correlation between displacements and the dominant environmental factors. The damage alarming index was defined based on the multiple regression models. At last, the X-bar control chart was utilized to detect the abnormal change of the displacements. Analysis results reveal that temperature and traffic condition are the dominant environmental factors to influence the displacement. When the confidence level of X-bar control chart is set to be 0.003, the false-positive indications of damage can be avoided. The damage sensitivity analysis shows that the proper X-bar control chart can detect 0.1 cm damage-induced change of the expansion joint displacement. It is reasonably believed that the proposed technique is robust against false-positive indication of damage and suitable to alarm the possible future damage of the expansion joints.