diff --git a/README.md b/README.md index 7c87adc..400f12c 100644 --- a/README.md +++ b/README.md @@ -193,6 +193,7 @@ Ensemble provides a browser-based IDE, [Ensemble Studio](https://studio.ensemble - [Action: onViewGroupUpdate](#action-onviewgroupupdate) - [openAppSettings](#openappsettings) - [openCamera](#opencamera) + - [openFaceCamera](#openfacecamera) - [openPlaidLink](#openplaidlink) - [openUrl](#openurl) - [pauseAudio](#pauseaudio) @@ -233,6 +234,7 @@ Ensemble provides a browser-based IDE, [Ensemble Studio](https://studio.ensemble - [navigateScreen](#navigatescreen) - [notification](#notification) - [openCamera](#opencamera) + - [openFaceCamera](#openfacecamera) - [pickFiles](#pickfiles) - [rateApp](#rateapp) - [requestNotificationAccess](#requestnotificationaccess) @@ -11871,6 +11873,7 @@ When an event is triggered (e.g. button is tapped), you can perform actions such | Property | Description | | :---------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [openCamera](open-camera.md) | openCamera action allows users to access their device's camera within the app for capturing images and videos. | +| [openFaceCamera](open-face-camera.md) | openFaceCamera action provides face-aware capture with alignment guidance and optional auto-capture for selfie and verification flows. | | [getLocation](get-location.md) | getLocation action retrieves the device's current location, enabling location-based functionalities within the app. | | [requestNotificationAccess](request-notification-access.md) | requestNotificationAccess action prompts users to grant permission for the app to send notifications to their device. | | [showNotification](show-notification.md) | showNotification action displays local notifications within the app, notifying users of important events or information. | @@ -14153,6 +14156,253 @@ To learn more about openCamera functionalities, test it out here in [Ensemble Ki --- +# openFaceCamera + +## Overview + +The `ensemble_face_camera` module adds real-time face detection to the Ensemble framework. +It is extracted from the original `ensemble_camera` package and focused on face-based capture workflows across Mobile and Web. + +## What It Does + +- Real-time face detection on Mobile and Web +- Auto-capture when alignment conditions are met +- Configurable detection options (yaw, tilt, thresholds, performance mode) +- Seamless integration with Ensemble via `openFaceCamera` +- Direct Flutter widget support + +## Installation + +Add this to `pubspec.yaml`: + +```yaml +dependencies: + ensemble_face_camera: + git: + url: https://github.com/EnsembleUI/ensemble.git + ref: main + path: modules/face_camera +``` + +## Required Setup + +Register the manager during app initialization: + +```dart +import 'package:ensemble_face_camera/ensemble_face_camera.dart'; + +GetIt.I.registerSingleton( + FaceCameraManagerImpl(), +); +``` + +Without this registration, the `openFaceCamera` action will not work. + +## Action + +Use `openFaceCamera` in YAML. + +### Properties + +| Property | Type | Description | +| :------------ | :----- | :-------------------------------------------------------------------------------- | +| id | string | Camera ID used to bind captured results (for example `${myFaceCamera.files}`). | +| onCapture | action | Executes when an image is captured. Captured files are available via `event.data.files`. | +| onError | action | Executes on error. Error message is available via `event.error`. | +| options | object | Additional configuration for camera and detection. | + +### options + +| Property | Type | Description | +| :------------------------ | :------ | :------------------------------------------------------------------------------------------------ | +| initialCamera | string | Which lens to start with: `front` (default), `back` | +| message | string | Message shown above camera preview | +| messageStyle | object | Text style for `message` | +| showControls | boolean | Show/hide all camera controls. Default `true` | +| showCaptureControl | boolean | Show/hide capture button. Default `true` | +| showFlashControl | boolean | Show/hide flash control. Default `true` | +| showCameraLensControl | boolean | Show/hide camera switch control. Default `true` | +| showStatusMessage | boolean | Show/hide face detection status text. Default `true` | +| indicatorShape | string | Face indicator shape: `circle`, `square` | +| autoDisableCaptureControl | boolean | Disable capture while no valid face is detected. Default `false` | +| autoCapture | boolean | Automatically capture when face qualifies. Default `false` | +| imageResolution | string | Capture resolution: `low`, `medium`, `high` (default), `veryHigh`, `ultraHigh`, `max` | +| defaultFlashMode | string | Initial flash mode: `off` (default), `auto`, `always` | +| orientation | string | Camera orientation: `portraitUp` (default), `portraitDown`, `landscapeLeft`, `landscapeRight` | +| performanceMode | string | Face detector mode: `fast` (default), `accurate` | +| accuracyConfig | object | Fine-grained face detection thresholds. Web only. See `options.accuracyConfig` below. | + +### options.accuracyConfig + +> Note: `accuracyConfig` is supported only on web. + +The following properties are verified in code as parsed and used by web face detection: + +| Property | Type | Default | Description | +| :------------------------- | :----- | :------ | :--------------------------------------------------------------------------- | +| detectionThreshold | number | 0.6 | Minimum confidence score for a valid face detection | +| intersectionRatioThreshold | number | 0.9 | Minimum overlap ratio between detected face and expected region | +| extraHeightFactor | number | 0.3 | Additional height factor for face bounding box | +| inputSize | number | 224 | Input image size used by detector | +| landmarkRatio | number | 0.95 | Minimum landmark alignment/visibility ratio | +| frameMargin | number | 0.05 | Margin ratio to ensure face is not too close to frame edges | +| tiltAngleThreshold | number | 6 | Maximum allowed tilt angle in degrees | +| horizontalCenterTolerance | number | 0.08 | Allowed horizontal centering tolerance | +| earThreshold | number | 0.25 | Eye aspect ratio threshold for open-eye validation | +| minFaceWidthRatio | number | 0.18 | Minimum face width ratio relative to frame | +| maxFaceWidthRatio | number | 0.82 | Maximum face width ratio relative to frame | +| qualityPassThreshold | number | 0.8 | Minimum quality score required to pass | +| yawLowerThreshold | number | 0.85 | Lower yaw ratio bound | +| yawUpperThreshold | number | 1.15 | Upper yaw ratio bound | + +Alias keys also supported by parser: + +- `threshold` -> `detectionThreshold` +- `yaw` -> `yawUpperThreshold` +- `tilt` -> `tiltAngleThreshold` +- `minFaceSize` -> `minFaceWidthRatio` + +## Usage Examples + +### 1. Basic capture and preview + +Open the face camera, bind the result to an ID, and preview the captured image. + +```yaml +View: + header: + title: "Action: openFaceCamera" + styles: + scrollableView: true + body: + Column: + styles: { gap: 16, padding: 24 } + children: + - Button: + label: Open Face Camera + onTap: + openFaceCamera: + id: myFaceCamera + + - Conditional: + conditions: + - if: ${myFaceCamera.files.length > 0} + Image: + source: ${myFaceCamera.files[0].path} +``` + +### 2. Capture with common options + +Configure the most common face camera options and log the result when a capture succeeds. + +```yaml +- Button: + label: Open Face Camera with Options + onTap: + openFaceCamera: + id: myFaceCamera + options: + initialCamera: front + performanceMode: fast + imageResolution: high + defaultFlashMode: off + orientation: portraitUp + message: "Align your face" + messageStyle: + color: 0xFFFFFFFF + indicatorShape: circle + showStatusMessage: true + showControls: true + showCaptureControl: true + showFlashControl: true + showCameraLensControl: true + onCapture: + executeCode: + body: | + console.log('Face camera captured image with id: ' + myFaceCamera.files); + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +### 3. Auto-capture and upload + +Automatically capture once the face is valid, then upload the captured file. + +```yaml +- Button: + label: Auto Capture and Upload + onTap: + openFaceCamera: + id: captureMedia + options: + autoCapture: true + performanceMode: accurate + message: "Hold still for capture" + onCapture: + uploadFiles: + id: uploader + files: ${captureMedia.files[0]} + uploadApi: fileUploadApi + fieldName: file + +- Markdown: + text: ${uploader.body} + +API: + fileUploadApi: + inputs: + - url + uri: ${url} + method: POST +``` + +### 4. Web accuracy config + +Use strict face detection thresholds for web capture. + +```yaml +- Button: + label: Open Camera (web, strict) + onTap: + openFaceCamera: + id: cameraWithFaceDetection + options: + initialCamera: front + autoCapture: false + performanceMode: accurate + accuracyConfig: + detectionThreshold: 0.5 + intersectionRatioThreshold: 0.9 + extraHeightFactor: 0.6 + inputSize: 224 + landmarkRatio: 0.95 + frameMargin: 0.05 + tiltAngleThreshold: 6 + horizontalCenterTolerance: 0.08 + earThreshold: 0.25 + minFaceWidthRatio: 0.18 + maxFaceWidthRatio: 0.82 + qualityPassThreshold: 0.8 + yawLowerThreshold: 0.85 + yawUpperThreshold: 1.15 + message: "Align your face in the square" + messageStyle: + color: "#FF0000" + fontSize: 20 + onCapture: + uploadFiles: + id: uploader + files: ${cameraWithFaceDetection.files[0]} + uploadApi: fileUploadApi + fieldName: file + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +--- + # openPlaidLink openPlaidLink action allows users to open external URLs or web links directly from the app, facilitating seamless integration with external content and enhancing the user's browsing experience within the application. @@ -17567,6 +17817,90 @@ To learn more about openCamera functionalities, test it out here in [Ensemble Ki --- +# openFaceCamera + +Use `openFaceCamera` when you need face-aware capture (for example selfie verification, identity checks, or attendance) instead of generic photo capture. + +`openFaceCamera` provides guided framing, face validation, and optional auto-capture once the face qualifies. + +## When to use this instead of openCamera + +- You need face alignment guidance before capture. +- You need automatic capture when a valid face is detected. +- You need stricter face quality checks, especially on web via `accuracyConfig`. + +For generic photo/video capture or gallery-first flows, continue using [`openCamera`](#opencamera). + +## Properties + +| Property | Type | Description | +| :-------- | :----- | :------------------------------------------------------------------------------ | +| id | string | Camera ID used to bind captured results (for example `${myFaceCamera.files}`). | +| onCapture | action | Executes when an image is captured. Files are available in `event.data.files`. | +| onError | action | Executes on error. Error message is available via `event.error`. | +| options | object | Face camera and detection configuration. | + +## options + +| Property | Type | Description | +| :------------------------ | :------ | :------------------------------------------------------------------------------ | +| initialCamera | string | Which lens to start with: `front` (default), `back` | +| message | string | Message shown above camera preview | +| messageStyle | object | Text style for `message` | +| showControls | boolean | Show/hide all camera controls. Default `true` | +| showCaptureControl | boolean | Show/hide capture button. Default `true` | +| showFlashControl | boolean | Show/hide flash control. Default `true` | +| showCameraLensControl | boolean | Show/hide camera switch control. Default `true` | +| showStatusMessage | boolean | Show/hide face detection status text. Default `true` | +| indicatorShape | string | Face indicator shape: `circle`, `square` | +| autoDisableCaptureControl | boolean | Disable capture while no valid face is detected. Default `false` | +| autoCapture | boolean | Automatically capture when face qualifies. Default `false` | +| imageResolution | string | Capture resolution: `low`, `medium`, `high`, `veryHigh`, `ultraHigh`, `max` | +| defaultFlashMode | string | Initial flash mode: `off` (default), `auto`, `always` | +| orientation | string | Camera orientation: `portraitUp`, `portraitDown`, `landscapeLeft`, `landscapeRight` | +| performanceMode | string | Face detector mode: `fast` (default), `accurate` | +| accuracyConfig | object | Fine-grained face detection thresholds. Web only. | + +## Usage + +### Basic + +```yaml +- Button: + label: Open Face Camera + onTap: + openFaceCamera: + id: myFaceCamera +``` + +### Auto-capture + +```yaml +- Button: + label: Capture verified selfie + onTap: + openFaceCamera: + id: profilePhoto + options: + initialCamera: front + autoCapture: true + performanceMode: accurate + message: "Align your face" + onCapture: + executeCode: + body: | + console.log('Captured face image: ' + profilePhoto.files[0].path); + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +## Full Reference + +For complete module setup, advanced `accuracyConfig` fields, and more examples, see [`openFaceCamera` action docs](#openfacecamera). + +--- + # pickFiles pickFiles action enables users to select files from their device for further processing or usage within the app, offering a seamless way to choose and work with files, enhancing user interactions and content management capabilities. diff --git a/content/actions/_meta.json b/content/actions/_meta.json index 88198c4..ae5758e 100644 --- a/content/actions/_meta.json +++ b/content/actions/_meta.json @@ -98,6 +98,9 @@ "open-camera": { "title": "Open Camera" }, + "open-face-camera": { + "title": "Open Face Camera" + }, "open-plaid-link": { "title": "Open Plaid Link" }, diff --git a/content/actions/directory.md b/content/actions/directory.md index cf6a02a..41db6ba 100644 --- a/content/actions/directory.md +++ b/content/actions/directory.md @@ -28,6 +28,7 @@ When an event is triggered (e.g. button is tapped), you can perform actions such | Property | Description | | :---------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [openCamera](open-camera.md) | openCamera action allows users to access their device's camera within the app for capturing images and videos. | +| [openFaceCamera](open-face-camera.md) | openFaceCamera action provides a camera plugin that detects face in real-time. flows. | | [getLocation](get-location.md) | getLocation action retrieves the device's current location, enabling location-based functionalities within the app. | | [requestNotificationAccess](request-notification-access.md) | requestNotificationAccess action prompts users to grant permission for the app to send notifications to their device. | | [showNotification](show-notification.md) | showNotification action displays local notifications within the app, notifying users of important events or information. | diff --git a/content/actions/open-camera.md b/content/actions/open-camera.md index 23461f7..aff4e2f 100644 --- a/content/actions/open-camera.md +++ b/content/actions/open-camera.md @@ -37,7 +37,6 @@ | assistSpeed | object | Show assist message whenever camera is moving faster than maxSpeed. [see properties](#values-for-optionsassistspeed) | | autoCaptureInterval | integer | If set any number n, on each n interval camera will capture | | captureOverlay | boolean | If set picture will be cropped according to overlay widget | -| faceDetection | object | Enable face detection. [see properties](#values-for-optionsfacedetection) | ##### Values for options.assistAngle @@ -54,52 +53,8 @@ | maxSpeed | number | Maximum speed in km/hr. | | assistSpeedMessage | number | Custom message to show when condition is hit. | -##### Values for options.faceDetection - -| Property | Type | Description | -| :------------------------ | :------ | :------------------------------------------------------------------------------------------------------------------------------------------------ | -| enabled | boolean | Enable face detection. | -| message | string | use this pass a message above the camera | -| messageStyle | object | style applied to the message widget | -| showControls | boolean | set false to hide all controls | -| showCaptureControl | boolean | set false to hide capture control icon | -| showFlashControl | boolean | set false to hide flash control control icon | -| showCameraLensControl | boolean | set false to hide camera lens control icon | -| indicatorShape | string | use this to change the shape of the face indicator `circle`, `square` | -| autoDisableCaptureControl | boolean | set true to disable capture control widget when no face is detected | -| autoCapture | boolean | set true to capture image on face detected | -| imageResolution | string | use this to set image resolution `low`, `medium`, `high` | -| defaultFlashMode | string | use this to set initial flash mode `off`, `auto`, `always` | -| performanceMode | string | Use this to set your preferred performance mode. `accurate`, `fast` | -| accuracyConfig | object | Use this to set accuracy config for face detection. Accuracy config is only supported on web. [see properties](#values-for-optionsfacedetectionaccuracyconfig) | - -##### Values for options.faceDetection.accuracyConfig - -> [Note] -> Accuracy config is only supported on web. - -| Property | Type | Description | -| :------------------------- | :----- | :--------------------------------------------------------------------------- | -| detectionThreshold | number | Minimum confidence score required to consider a face detection valid. | -| intersectionRatioThreshold | number | Minimum allowed overlap ratio between the detected face and expected region. | -| extraHeightFactor | number | Additional height factor added to the face bounding box. | -| inputSize | number | Size of the input image used for face detection. | -| landmarkRatio | number | Minimum acceptable alignment accuracy for facial landmarks. | -| frameMargin | number | Margin ratio to ensure face is not too close to frame edges. | -| tiltAngleThreshold | number | Maximum allowed tilt angle of the detected face (in degrees). | -| horizontalCenterTolerance | number | Allowed tolerance for how centered the face must be horizontally. | -| earThreshold | number | Minimum Eye Aspect Ratio (EAR) to detect open eyes. | -| minFaceWidthRatio | number | Minimum ratio of face width relative to the frame. | -| maxFaceWidthRatio | number | Maximum ratio of face width relative to the frame. | -| qualityPassThreshold | number | Minimum quality score required for a face to pass detection. | -| yawLowerThreshold | number | Lower bound of acceptable yaw (left-right head rotation) ratio. | -| yawUpperThreshold | number | Upper bound of acceptable yaw (left-right head rotation) ratio. | - - **Usage Examples** - - ```yaml View: header: @@ -273,55 +228,4 @@ You can clear previous camera results while recapturing using `cameraId.clear()` id: captureLatest ``` -Capture image on face detected - -```yaml -- Button: - label: Open Camera - onTap: - openCamera: - id: cameraWithFaceDetection - options: - initialCamera: front - faceDetection: - enabled: true - autoCapture: false - performanceMode: accurate - accuracyConfig: # accuracyConfig is only supported on web - detectionThreshold: 0.5 - intersectionRatioThreshold: 0.9 - extraHeightFactor: 0.6 - inputSize: 224 - landmarkRatio: 0.95 - frameMargin: 0.05 - tiltAngleThreshold: 6 - horizontalCenterTolerance: 0.08 - earThreshold: 0.25 - minFaceWidthRatio: 0.18 - maxFaceWidthRatio: 0.82 - qualityPassThreshold: 0.8 - yawLowerThreshold: 0.85 - yawUpperThreshold: 1.15 - message: "Align your face in the square" - messageStyle: - color: "#FF0000" - fontSize: 20 - onCapture: - uploadFiles: - id: uploader - files: ${cameraWithFaceDetection.files[0]} - uploadApi: fileUploadApi - fieldName: file - onComplete: - showDialog: - body: - Column: - children: - - TextInput: - value: ${cameraWithFaceDetection.files[0]} - - Image: - source: ${cameraWithFaceDetection.files[0].path} -``` - - To learn more about openCamera functionalities, test it out here in [Ensemble Kitchen Sink](https://studio.ensembleui.com/app/e24402cb-75e2-404c-866c-29e6c3dd7992/screen/USuOaOZApSgzE2uVrqlv) app. diff --git a/content/actions/open-face-camera.md b/content/actions/open-face-camera.md new file mode 100644 index 0000000..bcf1c4a --- /dev/null +++ b/content/actions/open-face-camera.md @@ -0,0 +1,213 @@ +# openFaceCamera + +`openFaceCamera` action opens the device camera with real-time face detection. It automatically captures an image when the face is properly aligned (position, angle, etc.), making it ideal for selfies, profile photos, or any face-detection task on mobile and web. Use the `id` property to bind the captured image result (e.g. to a variable or data binding). + +### Setup in Ensemble Studio + +1. Go to https://studio.ensembleui.com/ +2. Open your app +3. From the left sidebar, click **Build & Deploy** +4. In the **Build Settings** tab, enable the **Face Camera** module/feature and update. +5. If building for iOS: switch to the **iOS** tab and add the required camera usage description (e.g. "We use the camera to capture your face for verification") and any face-related privacy descriptions + + +### Properties + +| Property | Type | Description | +| :------------ | :----- | :-------------------------------------------------------------------------------- | +| id | string | Camera ID used to bind captured results (for example `${myFaceCamera.files}`). | +| onCapture | action | Executes when an image is captured. Captured files are available via `event.data.files`. | +| onError | action | Executes on error. Error message is available via `event.error`. | +| options | object | Additional configuration for camera and detection. | + +### options + +| Property | Type | Description | +| :------------------------ | :------ | :------------------------------------------------------------------------------------------------ | +| initialCamera | string | Which lens to start with: `front` (default), `back` | +| message | string | Message shown above camera preview | +| messageStyle | object | Text style for `message` | +| showControls | boolean | Show/hide all camera controls. Default `true` | +| showCaptureControl | boolean | Show/hide capture button. Default `true` | +| showFlashControl | boolean | Show/hide flash control. Default `true` | +| showCameraLensControl | boolean | Show/hide camera switch control. Default `true` | +| showStatusMessage | boolean | Show/hide face detection status text. Default `true` | +| indicatorShape | string | Face indicator shape: `circle`, `square` | +| autoDisableCaptureControl | boolean | Disable capture while no valid face is detected. Default `false` | +| autoCapture | boolean | Automatically capture when face qualifies. Default `false` | +| imageResolution | string | Capture resolution: `low`, `medium`, `high` (default), `veryHigh`, `ultraHigh`, `max` | +| defaultFlashMode | string | Initial flash mode: `off` (default), `auto`, `always` | +| orientation | string | Camera orientation: `portraitUp` (default), `portraitDown`, `landscapeLeft`, `landscapeRight` | +| performanceMode | string | Face detector mode: `fast` (default), `accurate` | +| accuracyConfig | object | Fine-grained face detection thresholds. Web only. See `options.accuracyConfig` below. | + +### options.accuracyConfig + +> Note: `accuracyConfig` is supported only on web. + +The following properties are verified in code as parsed and used by web face detection: + +| Property | Type | Default | Description | +| :------------------------- | :----- | :------ | :--------------------------------------------------------------------------- | +| detectionThreshold | number | 0.6 | Minimum confidence score for a valid face detection | +| intersectionRatioThreshold | number | 0.9 | Minimum overlap ratio between detected face and expected region | +| extraHeightFactor | number | 0.3 | Additional height factor for face bounding box | +| inputSize | number | 224 | Input image size used by detector | +| landmarkRatio | number | 0.95 | Minimum landmark alignment/visibility ratio | +| frameMargin | number | 0.05 | Margin ratio to ensure face is not too close to frame edges | +| tiltAngleThreshold | number | 6 | Maximum allowed tilt angle in degrees | +| horizontalCenterTolerance | number | 0.08 | Allowed horizontal centering tolerance | +| earThreshold | number | 0.25 | Eye aspect ratio threshold for open-eye validation | +| minFaceWidthRatio | number | 0.18 | Minimum face width ratio relative to frame | +| maxFaceWidthRatio | number | 0.82 | Maximum face width ratio relative to frame | +| qualityPassThreshold | number | 0.8 | Minimum quality score required to pass | +| yawLowerThreshold | number | 0.85 | Lower yaw ratio bound | +| yawUpperThreshold | number | 1.15 | Upper yaw ratio bound | + +Alias keys also supported by parser: + +- `threshold` -> `detectionThreshold` +- `yaw` -> `yawUpperThreshold` +- `tilt` -> `tiltAngleThreshold` +- `minFaceSize` -> `minFaceWidthRatio` + +## Usage Examples + +### 1. Basic capture and preview + +Open the face camera, bind the result to an ID, and preview the captured image. + +```yaml +View: + header: + title: "Action: openFaceCamera" + styles: + scrollableView: true + body: + Column: + styles: { gap: 16, padding: 24 } + children: + - Button: + label: Open Face Camera + onTap: + openFaceCamera: + id: myFaceCamera + + - Conditional: + conditions: + - if: ${myFaceCamera.files.length > 0} + Image: + source: ${myFaceCamera.files[0].path} +``` + +### 2. Capture with common options + +Configure the most common face camera options and log the result when a capture succeeds. + +```yaml +- Button: + label: Open Face Camera with Options + onTap: + openFaceCamera: + id: myFaceCamera + options: + initialCamera: front + performanceMode: fast + imageResolution: high + defaultFlashMode: off + orientation: portraitUp + message: "Align your face" + messageStyle: + color: 0xFFFFFFFF + indicatorShape: circle + showStatusMessage: true + showControls: true + showCaptureControl: true + showFlashControl: true + showCameraLensControl: true + onCapture: + executeCode: + body: | + console.log('Face camera captured image with id: ' + myFaceCamera.files); + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +### 3. Auto-capture and upload + +Automatically capture once the face is valid, then upload the captured file. + +```yaml +- Button: + label: Auto Capture and Upload + onTap: + openFaceCamera: + id: captureMedia + options: + autoCapture: true + performanceMode: accurate + message: "Hold still for capture" + onCapture: + uploadFiles: + id: uploader + files: ${captureMedia.files[0]} + uploadApi: fileUploadApi + fieldName: file + +- Markdown: + text: ${uploader.body} + +API: + fileUploadApi: + inputs: + - url + uri: ${url} + method: POST +``` + +### 4. Web accuracy config + +Use strict face detection thresholds for web capture. + +```yaml +- Button: + label: Open Camera (web, strict) + onTap: + openFaceCamera: + id: cameraWithFaceDetection + options: + initialCamera: front + autoCapture: false + performanceMode: accurate + accuracyConfig: + detectionThreshold: 0.5 + intersectionRatioThreshold: 0.9 + extraHeightFactor: 0.6 + inputSize: 224 + landmarkRatio: 0.95 + frameMargin: 0.05 + tiltAngleThreshold: 6 + horizontalCenterTolerance: 0.08 + earThreshold: 0.25 + minFaceWidthRatio: 0.18 + maxFaceWidthRatio: 0.82 + qualityPassThreshold: 0.8 + yawLowerThreshold: 0.85 + yawUpperThreshold: 1.15 + message: "Align your face in the square" + messageStyle: + color: "#FF0000" + fontSize: 20 + onCapture: + uploadFiles: + id: uploader + files: ${cameraWithFaceDetection.files[0]} + uploadApi: fileUploadApi + fieldName: file + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +To learn more about openCamera functionalities, test it out here in [Ensemble Kitchen Sink](https://studio.ensembleui.com/app/e24402cb-75e2-404c-866c-29e6c3dd7992/screen/iLJkHUPgX3ii9EdQ1D8K) app. diff --git a/content/device-capabilities/_meta.json b/content/device-capabilities/_meta.json index 399ef72..14196ff 100644 --- a/content/device-capabilities/_meta.json +++ b/content/device-capabilities/_meta.json @@ -29,6 +29,9 @@ "open-camera": { "title": "Open Camera" }, + "open-face-camera": { + "title": "Open Face Camera" + }, "pick-files": { "title": "Pick Files" }, diff --git a/content/device-capabilities/open-face-camera.md b/content/device-capabilities/open-face-camera.md new file mode 100644 index 0000000..d1a5cf0 --- /dev/null +++ b/content/device-capabilities/open-face-camera.md @@ -0,0 +1,54 @@ +# openFaceCamera + +Use `openFaceCamera` when you need **real-time** face detection — such as selfies, profile photos etc — instead of plain camera access. + +It adds face detection, alignment guidance (framing overlay), status messages, and optional **auto-capture** when the face meets quality criteria. + +### When to use this instead of `openCamera` + +- You want visual guidance to center and align the face before capture +- You need automatic capture once a valid face is detected +- You want stricter face quality rules (especially on web with `accuracyConfig`) + +### Key Properties + +| Property | Type | Description | +|------------|--------|-----------------------------------------------------------------------------| +| id | string | Bind captured result (access via `${yourId.files}`) | +| onCapture | action | Runs after successful capture (`event.data.files` available) | +| onError | action | Handles errors (`event.error` available) | +| options | object | Controls camera lens, UI, auto-capture, resolution, flash, and detection | + +### Quick Examples + +**Basic face capture** + +```yaml +- Button: + label: Open Face Camera + onTap: + openFaceCamera: + id: myFace +``` +**Auto-capture verified selfie** + +```yaml +- Button: + label: Capture Verified Selfie + onTap: + openFaceCamera: + id: profilePhoto + options: + initialCamera: front + autoCapture: true + message: "Align your face in the frame" + performanceMode: accurate + onCapture: + showToast: + message: "Selfie captured!" + onError: + showToast: + message: "Capture failed: ${event.error}" +``` + +To learn more about openCamera functionalities, test it out here in [Ensemble Kitchen Sink](https://studio.ensembleui.com/app/e24402cb-75e2-404c-866c-29e6c3dd7992/screen/iLJkHUPgX3ii9EdQ1D8K) app. diff --git a/public/llms-full.txt b/public/llms-full.txt index 7c87adc..400f12c 100644 --- a/public/llms-full.txt +++ b/public/llms-full.txt @@ -193,6 +193,7 @@ Ensemble provides a browser-based IDE, [Ensemble Studio](https://studio.ensemble - [Action: onViewGroupUpdate](#action-onviewgroupupdate) - [openAppSettings](#openappsettings) - [openCamera](#opencamera) + - [openFaceCamera](#openfacecamera) - [openPlaidLink](#openplaidlink) - [openUrl](#openurl) - [pauseAudio](#pauseaudio) @@ -233,6 +234,7 @@ Ensemble provides a browser-based IDE, [Ensemble Studio](https://studio.ensemble - [navigateScreen](#navigatescreen) - [notification](#notification) - [openCamera](#opencamera) + - [openFaceCamera](#openfacecamera) - [pickFiles](#pickfiles) - [rateApp](#rateapp) - [requestNotificationAccess](#requestnotificationaccess) @@ -11871,6 +11873,7 @@ When an event is triggered (e.g. button is tapped), you can perform actions such | Property | Description | | :---------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [openCamera](open-camera.md) | openCamera action allows users to access their device's camera within the app for capturing images and videos. | +| [openFaceCamera](open-face-camera.md) | openFaceCamera action provides face-aware capture with alignment guidance and optional auto-capture for selfie and verification flows. | | [getLocation](get-location.md) | getLocation action retrieves the device's current location, enabling location-based functionalities within the app. | | [requestNotificationAccess](request-notification-access.md) | requestNotificationAccess action prompts users to grant permission for the app to send notifications to their device. | | [showNotification](show-notification.md) | showNotification action displays local notifications within the app, notifying users of important events or information. | @@ -14153,6 +14156,253 @@ To learn more about openCamera functionalities, test it out here in [Ensemble Ki --- +# openFaceCamera + +## Overview + +The `ensemble_face_camera` module adds real-time face detection to the Ensemble framework. +It is extracted from the original `ensemble_camera` package and focused on face-based capture workflows across Mobile and Web. + +## What It Does + +- Real-time face detection on Mobile and Web +- Auto-capture when alignment conditions are met +- Configurable detection options (yaw, tilt, thresholds, performance mode) +- Seamless integration with Ensemble via `openFaceCamera` +- Direct Flutter widget support + +## Installation + +Add this to `pubspec.yaml`: + +```yaml +dependencies: + ensemble_face_camera: + git: + url: https://github.com/EnsembleUI/ensemble.git + ref: main + path: modules/face_camera +``` + +## Required Setup + +Register the manager during app initialization: + +```dart +import 'package:ensemble_face_camera/ensemble_face_camera.dart'; + +GetIt.I.registerSingleton( + FaceCameraManagerImpl(), +); +``` + +Without this registration, the `openFaceCamera` action will not work. + +## Action + +Use `openFaceCamera` in YAML. + +### Properties + +| Property | Type | Description | +| :------------ | :----- | :-------------------------------------------------------------------------------- | +| id | string | Camera ID used to bind captured results (for example `${myFaceCamera.files}`). | +| onCapture | action | Executes when an image is captured. Captured files are available via `event.data.files`. | +| onError | action | Executes on error. Error message is available via `event.error`. | +| options | object | Additional configuration for camera and detection. | + +### options + +| Property | Type | Description | +| :------------------------ | :------ | :------------------------------------------------------------------------------------------------ | +| initialCamera | string | Which lens to start with: `front` (default), `back` | +| message | string | Message shown above camera preview | +| messageStyle | object | Text style for `message` | +| showControls | boolean | Show/hide all camera controls. Default `true` | +| showCaptureControl | boolean | Show/hide capture button. Default `true` | +| showFlashControl | boolean | Show/hide flash control. Default `true` | +| showCameraLensControl | boolean | Show/hide camera switch control. Default `true` | +| showStatusMessage | boolean | Show/hide face detection status text. Default `true` | +| indicatorShape | string | Face indicator shape: `circle`, `square` | +| autoDisableCaptureControl | boolean | Disable capture while no valid face is detected. Default `false` | +| autoCapture | boolean | Automatically capture when face qualifies. Default `false` | +| imageResolution | string | Capture resolution: `low`, `medium`, `high` (default), `veryHigh`, `ultraHigh`, `max` | +| defaultFlashMode | string | Initial flash mode: `off` (default), `auto`, `always` | +| orientation | string | Camera orientation: `portraitUp` (default), `portraitDown`, `landscapeLeft`, `landscapeRight` | +| performanceMode | string | Face detector mode: `fast` (default), `accurate` | +| accuracyConfig | object | Fine-grained face detection thresholds. Web only. See `options.accuracyConfig` below. | + +### options.accuracyConfig + +> Note: `accuracyConfig` is supported only on web. + +The following properties are verified in code as parsed and used by web face detection: + +| Property | Type | Default | Description | +| :------------------------- | :----- | :------ | :--------------------------------------------------------------------------- | +| detectionThreshold | number | 0.6 | Minimum confidence score for a valid face detection | +| intersectionRatioThreshold | number | 0.9 | Minimum overlap ratio between detected face and expected region | +| extraHeightFactor | number | 0.3 | Additional height factor for face bounding box | +| inputSize | number | 224 | Input image size used by detector | +| landmarkRatio | number | 0.95 | Minimum landmark alignment/visibility ratio | +| frameMargin | number | 0.05 | Margin ratio to ensure face is not too close to frame edges | +| tiltAngleThreshold | number | 6 | Maximum allowed tilt angle in degrees | +| horizontalCenterTolerance | number | 0.08 | Allowed horizontal centering tolerance | +| earThreshold | number | 0.25 | Eye aspect ratio threshold for open-eye validation | +| minFaceWidthRatio | number | 0.18 | Minimum face width ratio relative to frame | +| maxFaceWidthRatio | number | 0.82 | Maximum face width ratio relative to frame | +| qualityPassThreshold | number | 0.8 | Minimum quality score required to pass | +| yawLowerThreshold | number | 0.85 | Lower yaw ratio bound | +| yawUpperThreshold | number | 1.15 | Upper yaw ratio bound | + +Alias keys also supported by parser: + +- `threshold` -> `detectionThreshold` +- `yaw` -> `yawUpperThreshold` +- `tilt` -> `tiltAngleThreshold` +- `minFaceSize` -> `minFaceWidthRatio` + +## Usage Examples + +### 1. Basic capture and preview + +Open the face camera, bind the result to an ID, and preview the captured image. + +```yaml +View: + header: + title: "Action: openFaceCamera" + styles: + scrollableView: true + body: + Column: + styles: { gap: 16, padding: 24 } + children: + - Button: + label: Open Face Camera + onTap: + openFaceCamera: + id: myFaceCamera + + - Conditional: + conditions: + - if: ${myFaceCamera.files.length > 0} + Image: + source: ${myFaceCamera.files[0].path} +``` + +### 2. Capture with common options + +Configure the most common face camera options and log the result when a capture succeeds. + +```yaml +- Button: + label: Open Face Camera with Options + onTap: + openFaceCamera: + id: myFaceCamera + options: + initialCamera: front + performanceMode: fast + imageResolution: high + defaultFlashMode: off + orientation: portraitUp + message: "Align your face" + messageStyle: + color: 0xFFFFFFFF + indicatorShape: circle + showStatusMessage: true + showControls: true + showCaptureControl: true + showFlashControl: true + showCameraLensControl: true + onCapture: + executeCode: + body: | + console.log('Face camera captured image with id: ' + myFaceCamera.files); + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +### 3. Auto-capture and upload + +Automatically capture once the face is valid, then upload the captured file. + +```yaml +- Button: + label: Auto Capture and Upload + onTap: + openFaceCamera: + id: captureMedia + options: + autoCapture: true + performanceMode: accurate + message: "Hold still for capture" + onCapture: + uploadFiles: + id: uploader + files: ${captureMedia.files[0]} + uploadApi: fileUploadApi + fieldName: file + +- Markdown: + text: ${uploader.body} + +API: + fileUploadApi: + inputs: + - url + uri: ${url} + method: POST +``` + +### 4. Web accuracy config + +Use strict face detection thresholds for web capture. + +```yaml +- Button: + label: Open Camera (web, strict) + onTap: + openFaceCamera: + id: cameraWithFaceDetection + options: + initialCamera: front + autoCapture: false + performanceMode: accurate + accuracyConfig: + detectionThreshold: 0.5 + intersectionRatioThreshold: 0.9 + extraHeightFactor: 0.6 + inputSize: 224 + landmarkRatio: 0.95 + frameMargin: 0.05 + tiltAngleThreshold: 6 + horizontalCenterTolerance: 0.08 + earThreshold: 0.25 + minFaceWidthRatio: 0.18 + maxFaceWidthRatio: 0.82 + qualityPassThreshold: 0.8 + yawLowerThreshold: 0.85 + yawUpperThreshold: 1.15 + message: "Align your face in the square" + messageStyle: + color: "#FF0000" + fontSize: 20 + onCapture: + uploadFiles: + id: uploader + files: ${cameraWithFaceDetection.files[0]} + uploadApi: fileUploadApi + fieldName: file + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +--- + # openPlaidLink openPlaidLink action allows users to open external URLs or web links directly from the app, facilitating seamless integration with external content and enhancing the user's browsing experience within the application. @@ -17567,6 +17817,90 @@ To learn more about openCamera functionalities, test it out here in [Ensemble Ki --- +# openFaceCamera + +Use `openFaceCamera` when you need face-aware capture (for example selfie verification, identity checks, or attendance) instead of generic photo capture. + +`openFaceCamera` provides guided framing, face validation, and optional auto-capture once the face qualifies. + +## When to use this instead of openCamera + +- You need face alignment guidance before capture. +- You need automatic capture when a valid face is detected. +- You need stricter face quality checks, especially on web via `accuracyConfig`. + +For generic photo/video capture or gallery-first flows, continue using [`openCamera`](#opencamera). + +## Properties + +| Property | Type | Description | +| :-------- | :----- | :------------------------------------------------------------------------------ | +| id | string | Camera ID used to bind captured results (for example `${myFaceCamera.files}`). | +| onCapture | action | Executes when an image is captured. Files are available in `event.data.files`. | +| onError | action | Executes on error. Error message is available via `event.error`. | +| options | object | Face camera and detection configuration. | + +## options + +| Property | Type | Description | +| :------------------------ | :------ | :------------------------------------------------------------------------------ | +| initialCamera | string | Which lens to start with: `front` (default), `back` | +| message | string | Message shown above camera preview | +| messageStyle | object | Text style for `message` | +| showControls | boolean | Show/hide all camera controls. Default `true` | +| showCaptureControl | boolean | Show/hide capture button. Default `true` | +| showFlashControl | boolean | Show/hide flash control. Default `true` | +| showCameraLensControl | boolean | Show/hide camera switch control. Default `true` | +| showStatusMessage | boolean | Show/hide face detection status text. Default `true` | +| indicatorShape | string | Face indicator shape: `circle`, `square` | +| autoDisableCaptureControl | boolean | Disable capture while no valid face is detected. Default `false` | +| autoCapture | boolean | Automatically capture when face qualifies. Default `false` | +| imageResolution | string | Capture resolution: `low`, `medium`, `high`, `veryHigh`, `ultraHigh`, `max` | +| defaultFlashMode | string | Initial flash mode: `off` (default), `auto`, `always` | +| orientation | string | Camera orientation: `portraitUp`, `portraitDown`, `landscapeLeft`, `landscapeRight` | +| performanceMode | string | Face detector mode: `fast` (default), `accurate` | +| accuracyConfig | object | Fine-grained face detection thresholds. Web only. | + +## Usage + +### Basic + +```yaml +- Button: + label: Open Face Camera + onTap: + openFaceCamera: + id: myFaceCamera +``` + +### Auto-capture + +```yaml +- Button: + label: Capture verified selfie + onTap: + openFaceCamera: + id: profilePhoto + options: + initialCamera: front + autoCapture: true + performanceMode: accurate + message: "Align your face" + onCapture: + executeCode: + body: | + console.log('Captured face image: ' + profilePhoto.files[0].path); + onError: + showToast: + message: "Error capturing image: ${event.error}" +``` + +## Full Reference + +For complete module setup, advanced `accuracyConfig` fields, and more examples, see [`openFaceCamera` action docs](#openfacecamera). + +--- + # pickFiles pickFiles action enables users to select files from their device for further processing or usage within the app, offering a seamless way to choose and work with files, enhancing user interactions and content management capabilities. diff --git a/public/llms.txt b/public/llms.txt index acb8cf4..9972e8d 100644 --- a/public/llms.txt +++ b/public/llms.txt @@ -156,6 +156,7 @@ Ensemble is a platform where you can build, publish, and iterate native and web - [Action: onViewGroupUpdate](https://docs.ensembleui.com/actions/on-view-group-update) - [openAppSettings](https://docs.ensembleui.com/actions/open-app-settings) - [openCamera](https://docs.ensembleui.com/actions/open-camera) +- [openFaceCamera](https://docs.ensembleui.com/actions/open-face-camera) - [openPlaidLink](https://docs.ensembleui.com/actions/open-plaid-link) - [openUrl](https://docs.ensembleui.com/actions/open-url) - [pauseAudio](https://docs.ensembleui.com/actions/pause-audio) @@ -194,6 +195,7 @@ Ensemble is a platform where you can build, publish, and iterate native and web - [navigateScreen](https://docs.ensembleui.com/device-capabilities/navigate-screen) - [notification](https://docs.ensembleui.com/device-capabilities/notification) - [openCamera](https://docs.ensembleui.com/device-capabilities/open-camera) +- [openFaceCamera](https://docs.ensembleui.com/device-capabilities/open-face-camera) - [pickFiles](https://docs.ensembleui.com/device-capabilities/pick-files) - [rateApp](https://docs.ensembleui.com/device-capabilities/rate-app) - [requestNotificationAccess](https://docs.ensembleui.com/device-capabilities/request-notification-access)