Learning From the Past to Guard the Future