Skip to content

Docs/document sensor analytics#879

Open
naman-ranka wants to merge 46 commits intointel-retail:mainfrom
naman-ranka:docs/document-sensor-analytics
Open

Docs/document sensor analytics#879
naman-ranka wants to merge 46 commits intointel-retail:mainfrom
naman-ranka:docs/document-sensor-analytics

Conversation

@naman-ranka
Copy link
Copy Markdown
Contributor

PR Checklist

  • Added label to the Pull Request for easier discoverability and search
  • Commit Message meets guidelines as indicated in the URL https://github.com/intel-retail/automated-self-checkout/blob/main/CONTRIBUTING.md
  • Every commit is a single defect fix and does not mix feature addition or changes
  • Unit Tests have been added for new changes
  • Updated Documentation as relevant to the changes
  • All commented code has been removed
  • If you've added a dependency, you've ensured license is compatible with repository license and clearly outlined the added dependency.
  • PR change contains code related to security
  • PR introduces changes that breaks compatibility with other modules (If YES, please provide details below)

What are you changing?

Issue this PR will close

close: #issue_number

Anything the reviewer should know when reviewing this PR?

Test Instructions if applicable

If the there are associated PRs in other repositories, please link them here (i.e. intel-retail/automated-self-checkout )

naman-ranka and others added 30 commits January 28, 2025 01:55
…Kafka/HTTP publishing

- Implement Python microservice to ingest real or mock Lidar data
- Publish sensor data to MQTT, Kafka, or HTTP (controlled by environment vars)
- Add logging, retry logic, and graceful shutdown handling
- Integrate with existing Makefile and Docker Compose

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
…DAR and weight data

- Subscribes to LiDAR and weight sensor MQTT topics
- Performs analytics to calculate item density (weight per LiDAR point)
- Publishes analytics results to a dedicated MQTT topic ()
- Includes robust logging and error handling

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Introduced Grafana container configuration in docker-compose.yml
- Configured Grafana to depend on mqtt-broker_1

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Added required copyright header to newly created Python files.
- Ensures compliance with repository licensing guidelines.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
…blisher logic

- Consolidated sensor-specific directories into a single common-sources directory
- Implemented a common publisher.py to streamline all types of publishing (MQTT, Kafka, HTTP) across sensor types
- Added support for multiple sensors of the same type to improve scalability
- Introduced a custom MQTT Docker configuration to resolve previous network configuration issues
- Removed redundant directories to simplify the codebase

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Integrated a Grafana dashboard to visualize and analyze data from LiDAR and weight sensors.
- Configured dashboard panels to display time series, sensor counts, and analytics metrics.
- Dashboard pulls data from our existing MQTT data source for real-time insights.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Implemented Grafana datasource provisioning via YAML to auto-load the MQTT datasource.
- Updated dashboard.json to improve data visualization and remove manual refresh requirement.
- Note: Datasource URL still needs to be manually set due to Grafana's provisioning limitations.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Added  to test HTTP publishing inside .
- Updated  to expose port 5000 for internal HTTP testing.
- Verified HTTP publishing via Webhook.site without modifying .

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
…tion

- Updated Kafka Docker configuration for improved stability and compatibility.
- Added  to verify message consumption from any user-defined topic.
- Supports command-line arguments for dynamic topic selection and timeout configuration.
- Ensures Kafka messages are received correctly and provides meaningful test results.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Added a README file inside the common-service directory.
- Includes details on setup, environment variables, and testing procedures.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Replaced COPY commands with volumes in docker-compose.yml
Allows dynamic updates to dashboards and datasources without rebuilding the image

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
… image

Replaced custom-built Mosquitto image with the official eclipse-mosquitto:2.0.18.
Mounted mosquitto.conf as a volume to allow dynamic updates without rebuilding.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Co-authored-by: NeethuES <neethu.elizabeth.simon@intel.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Co-authored-by: ejlee-intel <elizabeth.j.lee@intel.com>
…Kafka/HTTP publishing

- Implement Python microservice to ingest real or mock Lidar data
- Publish sensor data to MQTT, Kafka, or HTTP (controlled by environment vars)
- Add logging, retry logic, and graceful shutdown handling
- Integrate with existing Makefile and Docker Compose

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
…DAR and weight data

- Subscribes to LiDAR and weight sensor MQTT topics
- Performs analytics to calculate item density (weight per LiDAR point)
- Publishes analytics results to a dedicated MQTT topic ()
- Includes robust logging and error handling

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Introduced Grafana container configuration in docker-compose.yml
- Configured Grafana to depend on mqtt-broker_1

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Added required copyright header to newly created Python files.
- Ensures compliance with repository licensing guidelines.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
…blisher logic

- Consolidated sensor-specific directories into a single common-sources directory
- Implemented a common publisher.py to streamline all types of publishing (MQTT, Kafka, HTTP) across sensor types
- Added support for multiple sensors of the same type to improve scalability
- Introduced a custom MQTT Docker configuration to resolve previous network configuration issues
- Removed redundant directories to simplify the codebase

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Integrated a Grafana dashboard to visualize and analyze data from LiDAR and weight sensors.
- Configured dashboard panels to display time series, sensor counts, and analytics metrics.
- Dashboard pulls data from our existing MQTT data source for real-time insights.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
naman-ranka and others added 16 commits February 14, 2025 00:49
- Implemented Grafana datasource provisioning via YAML to auto-load the MQTT datasource.
- Updated dashboard.json to improve data visualization and remove manual refresh requirement.
- Note: Datasource URL still needs to be manually set due to Grafana's provisioning limitations.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Added  to test HTTP publishing inside .
- Updated  to expose port 5000 for internal HTTP testing.
- Verified HTTP publishing via Webhook.site without modifying .

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
…tion

- Updated Kafka Docker configuration for improved stability and compatibility.
- Added  to verify message consumption from any user-defined topic.
- Supports command-line arguments for dynamic topic selection and timeout configuration.
- Ensures Kafka messages are received correctly and provides meaningful test results.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
- Added a README file inside the common-service directory.
- Includes details on setup, environment variables, and testing procedures.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Replaced COPY commands with volumes in docker-compose.yml
Allows dynamic updates to dashboards and datasources without rebuilding the image

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
… image

Replaced custom-built Mosquitto image with the official eclipse-mosquitto:2.0.18.
Mounted mosquitto.conf as a volume to allow dynamic updates without rebuilding.

Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Co-authored-by: NeethuES <neethu.elizabeth.simon@intel.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
Co-authored-by: ejlee-intel <elizabeth.j.lee@intel.com>
Signed-off-by: Naman Yeshwanth Kumar <namanranka14@gmail.com>
This change adds a new "Sensor Analytics and Visualization" section to the `README.md` file to document the features from PR intel-retail#655.

The new section includes:
- An overview of the `common-service` for LiDAR and Weight sensors.
- Information on data publishing via MQTT, Kafka, and HTTP.
- Instructions on how to access the Grafana dashboard for real-time visualization.

This will help users understand and utilize the new sensor analytics and visualization capabilities of the project.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant