Loss of FPS when program cant detect any face.

[JoyKaihatu]

Hello, after my testing. There are alot of FPS drop after i turn my face facing left and right. The FPS drop can be from 30 FPS constant down to 15 or sometime 12 FPS. I have try to fix the issue myself, and disabling the recognize thread seem to solve the issue and no more FPS drop when i turn my head or not facing the camera anymore.

Are there any permanent fix to this? Or any help on why its happening would help. Thanks

Note : Changing the main function of recognize.py
def main():
    """Main function to start face tracking and recognition threads."""
    file_name = "./face_tracking/config/config_tracking.yaml"
    config_tracking = load_config(file_name)

    # Start tracking thread
    thread_track = threading.Thread(
        target=tracking,
        args=(
            detector,
            config_tracking,
        ),
    )
    thread_track.start()

    # Disabling the Recognize Thread
    # Start recognition thread
    #thread_recognize = threading.Thread(target=recognize)
    #thread_recognize.start()  

[vectornguyen76]

@JoyKaihatu The reason may come from too much loop can not detect bbox, you can add sleep to reduce it. So far, it is just a demo version, if you want bring it to production, contact me.

import time
def recognize():
    """Face recognition in a separate thread."""
    while True:
        raw_image = data_mapping["raw_image"]
        detection_landmarks = data_mapping["detection_landmarks"]
        detection_bboxes = data_mapping["detection_bboxes"]
        tracking_ids = data_mapping["tracking_ids"]
        tracking_bboxes = data_mapping["tracking_bboxes"]

        for i in range(len(tracking_bboxes)):
            for j in range(len(detection_bboxes)):
                mapping_score = mapping_bbox(box1=tracking_bboxes[i], box2=detection_bboxes[j])
                if mapping_score > 0.9:
                    face_alignment = norm_crop(img=raw_image, landmark=detection_landmarks[j])

                    score, name = recognition(face_image=face_alignment)
                    if name is not None:
                        if score < 0.25:
                            caption = "UN_KNOWN"
                        else:
                            caption = f"{name}:{score:.2f}"

                    id_face_mapping[tracking_ids[i]] = caption

                    detection_bboxes = np.delete(detection_bboxes, j, axis=0)
                    detection_landmarks = np.delete(detection_landmarks, j, axis=0)

                    break

        if tracking_bboxes == []:
            time.sleep(0.001)
            print("Waiting for a person...")

[JoyKaihatu]

Hello, thanks for the reply. But the issue still presist and it got even worse where the FPS just drop even lower the bounding box will not disappear right away but instead slide to the edge of the screen untill it go away slowly out of the screen while maintaining lower FPS. I havent test it any further on how long it will stay like that or is the FPS will get better after the Recognize Thread finish processing the frame.

But from my observation, it seems that there are a different speed of process where your recognize will just process the frame and emit frame slower when the tracking_boxes == [] and it will process the frame faster when tracking boxes is not None. Which not make any sense to me.

It will be a great help if you can help me on this issue further. Since im now working on making a face recognition application on web using flask and js. Where the program will receive the image from client camera and process it, then show it back to the user. So far, the only prpoblem i encounter when testing is this. But other than that, the program can run smooth up to 45 - 60 FPS.

I can always reproduce the eror presistently, just let me know if you need information any further about the eror like a video or picture of when the erorr occured. Let me know and i will share any information you need to fix this.

[vectornguyen76]

I have a production version optimized using triton, vector database and redis. For this version I will fix this error later (maybe tonight).

[JoyKaihatu]

Will the production version be release under the same License as the demo here also in Github? ( I can already guess that it will not. But just asking ). Also, thanks for having the time to fix the demo version also :D.

[vectornguyen76]

It's just a private version