The conventional radio-frequency identification (RFID)-based access control system cannot prevent unauthorized personnel from following authorized personnel into a restricted area. Thus, security and safety of personnel cannot be guaranteed. In this paper, we propose an intelligent access control system to address this problem. The proposed system consists of an ultrahigh-frequency (UHF) RFID subsystem and a computer vision subsystem. The UHF RFID subsystem is responsible for the identification and authentication of personnel while the computer vision subsystem captures the images of personnel and accurately calculates their number. By comparing the number of personnel determined by the two subsystems, the access control problem of tailgating can be effectively solved. The proposed system has competitive advantages in terms of high detection accuracy and user convenience of hands-free access control. It can be used in many organizations that have security requirements.