中文版 | English
FaceControl is an innovative Android open-source application that leverages computer vision to enable users to control their devices through simple facial gestures such as blinking, nodding, and shaking.
This project aims to provide a new way of interaction for people with disabilities, or to make device operation accessible when manual handling is inconvenient (e.g., washing hands, cooking).
- Real-time Face Tracking: Powered by Google MediaPipe Face Mesh, supporting 468 3D facial landmarks detection in real-time.
- Continuous Background Monitoring: Combines CameraX with a Foreground Service, ensuring the app responds even when in the background or when the screen is locked.
- Global Gesture Simulation: Utilizes the Accessibility Service API to simulate swipe and click gestures across any third-party apps (e.g., TikTok, Browser) without requiring Root access.
- Anti-Mistouch Logic: Built-in EAR (Eye Aspect Ratio) state machine with support for "Double Blink" triggering to distinguish between natural blinks and control commands.
- Language: Kotlin
- UI Framework: Jetpack Compose
- Vision Computing: MediaPipe Tasks Vision
- Camera Support: CameraX (Camera2)
- System Interface: Android Accessibility Service API
- Background Architecture: Lifecycle-aware Foreground Service
| Gesture | Logic | Simulated Action | Use Case |
|---|---|---|---|
| Double Blink | Two valid blinks within 800ms | Swipe Up | Scrolling TikTok / Web pages |
| Head Shake | Nose horizontal displacement | Swipe Left | Page turning / Dismissing notifications |
| Head Nod | Pitch angle detection (Reserved) | Screen Click | Confirm / Like |
- Android Device (Running Android 8.0+ / API 26+)
- Physical device for testing (Emulators do not support hardware camera acceleration well)
git clone https://github.com/your-id/FaceControl-Android.gitDue to size limits, the MediaPipe model file is not included in the repository. Please follow these steps:
- Download face_landmarker.task.
- Place the file into the
app/src/main/assets/folder of your project.
After running the app, please enable:
- Camera Permission (For face recognition)
- Overlay Permission (For foreground service operation)
- Accessibility Service (Find
FaceControl Servicein System Settings and turn it on)
com.example.camera
├── FaceAnalyzer.kt # Vision Core (EAR calculation, gesture detection)
├── FaceAccessibilityService.kt # Execution Engine (Gesture simulation)
├── FaceControlForegroundService.kt # Background Orchestrator
├── MainActivity.kt # User Guidance & Permission UI
└── ui/theme/ # Compose UI Styles
- Blink detection based on EAR algorithm
- Support for "Double Blink" to avoid accidental triggers
- Head shake detection logic
- Add Mouth Open trigger
- User-customizable [Gesture -> Command] mapping
- Sensitivity adjustment and haptic feedback
Privacy is our priority:
- All vision analysis is performed locally on-device. No images or videos are uploaded to any server.
- Camera access is used solely for gesture recognition and is not recorded or stored.
- Accessibility Service is used only to simulate predefined gesture commands.
This project is licensed under the MIT License.
If you like this project, please give it a Star ⭐️! Contributions via Issues or PRs are welcome.