Use normalization to compute echo range scaling by ctuguinay · Pull Request #1463 · OSOceanAcoustics/echopype

ctuguinay · 2025-02-18T00:03:50Z

Enforces normalization of beam direction z if normalization of the beam direction vector is not already present. If vector is all zeros, return NaN.

codecov-commenter · 2025-02-25T00:57:43Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.14%. Comparing base (9f56124) to head (a993c55).
⚠️ Report is 387 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1463       +/-   ##
===========================================
+ Coverage   83.52%   95.14%   +11.62%     
===========================================
  Files          64        5       -59     
  Lines        5686      309     -5377     
===========================================
- Hits         4749      294     -4455     
+ Misses        937       15      -922

Flag	Coverage Δ
unittests	`95.14% <100.00%> (+11.62%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ctuguinay · 2025-02-25T01:06:53Z

@leewujung This should be ready for review

leewujung · 2025-04-03T19:59:41Z

+    # For channels with near-zero norm, we return NaN. Otherwise, we return the normalized
+    # z component.


This statement is reversed, right?

I don't think so since xr.where(norm < tolerance, np.nan, beam_direction_z / norm) will return NaN if norm is small

oh i know what I read wrong! I read "near-zero" as "non-zero" 😵
This is resolved then!

leewujung

@ctuguinay : Two minor comments, but otherwise this looks great. Feel free to merge once those are addressed!

Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>

for more information, see https://pre-commit.ci

…nAcoustics#1455)

… not fail (OSOceanAcoustics#1460)

updates: - [github.com/PyCQA/flake8: 7.1.1 → 7.1.2](PyCQA/flake8@7.1.1...7.1.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…s ci] (OSOceanAcoustics#1451) * check if zarr stores > 0 * add a test for echodata delete * fix path * add comments for readability --------- Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>

* dims to sizes * dims to sizes when using ds.dims * dims to sizes for test echodata combine

…oustics#1454) * chunks as dictionaries in _get_auto_chunk * when zarr encoding get chunks as list-like and rename _get_auto_chunk to _get_dask_auto_chunk to show that there is a difference between Dask and Zarr chunking

…xarray versions (OSOceanAcoustics#1462)

…ci] (OSOceanAcoustics#1450) * fix deprecation warning for truth value of an empty array * change test outcome from True to False when param is missing * found logic error in checking parameter validity before loading XML --------- Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>

…stics#1458) * fill in nan for missing filter coeffs from all channels * update test_convert_ek80_no_fil_coeff * remove commented out section * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * cast numpy float array to int * add compute Sv test * remove test mark --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ctuguinay <cmtuguinay2000@gmail.com>

…ll tests ci] (OSOceanAcoustics#1447) * EP-1420 resolved path issue with ek60_missing_channel_power * EP-1420 fixed path to use string concatenation * formatting * adding conditionals for checking platform nmea and exchange from time1 to time_nmea * working datatree for v0.9.0 w o calibration * Drop Ping Time Duplicates (OSOceanAcoustics#1382) * init commit * revert change to fix merge conflict * test only one file * use other file * move test duplicate to test convert ek * add extra line * move test back to ek80 convert * pin zarr and add check unique ping time duplicates and tests * fix test message * test remove zarr pin * add back zarr pin * Fix tests that fail from new Xarray variable and attribute assignment updates (OSOceanAcoustics#1433) * add .values to change water level scalar value * add .values to fillna * fix assign attributes * remove test1 * update echodata properly * remove print statement * Assemble AD2CP timestamp with nanosecond precision (OSOceanAcoustics#1436) * assemble ad2cp timestamp at ns instead of us * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add noqa --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Use `import_resources.files` instead of the legacy `open_text` (OSOceanAcoustics#1434) * use import_resources.files to open yaml files * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * import from importlib.resources * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * bump codespell version, add exceptions (OSOceanAcoustics#1438) * Pin `zarr` and `netcdf4` temporarily [all tests ci] (OSOceanAcoustics#1429) * pin zarr and netcdf4 in requirements.txt * change np.alltrue to np.all * use np.argmin(da.data) instead of da.argmin() * use S instead s for MVBS ping_time_bin * extract single element in computing nsamples and L in tapered_chirp * change from S to s in testing.py * Add `type-extensions` to requirements.txt (OSOceanAcoustics#1440) * add type-extensions * sort requirements.txt * add manual trigger to pypi workflow (OSOceanAcoustics#1442) * update cff with ices paper data (OSOceanAcoustics#1443) * add assign actual range utility function (OSOceanAcoustics#1435) * chore(deps): bump actions/setup-python from 5.3.0 to 5.4.0 (OSOceanAcoustics#1445) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.3.0 to 5.4.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5.3.0...v5.4.0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [pre-commit.ci] pre-commit autoupdate (OSOceanAcoustics#1446) * [pre-commit.ci] pre-commit autoupdate updates: - [github.com/PyCQA/isort: 5.13.2 → 6.0.0](PyCQA/isort@5.13.2...6.0.0) - [github.com/psf/black: 24.10.0 → 25.1.0](psf/black@24.10.0...25.1.0) - [github.com/codespell-project/codespell: v2.4.0 → v2.4.1](codespell-project/codespell@v2.4.0...v2.4.1) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * added pooch and bound the test registry to the echodata test * EP-1420 added legacy test files >=v0.8.4, created pooch registry, parameterized test * EP-1420 checked tests * EP-1420 debugging TestEchoData test failures, test_nbytes, test_setattr, & test_setitem * EP-1447 troubleshooting bug still * EP-1420 removed pooch and added normal test path resources * EP-1420 fixed test_nbytes, commented out test_setattr * EP-1420 changed "time_nmea" to "nmea_time", added ek80 legacy_datatree tests, fixed readme * EP-1420 removed import * trying to add sonar channel_all * EP-1420 checked sonar for channel and renamed to channel_all * check legacy data differently * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove v0.9.2 zarr test files * remove test_setattr that is no longer needed, see OSOceanAcoustics#1457 that removes EchoData.__setattr__ * Update echopype/tests/echodata/test_echodata.py --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Caesar Tuguinay <87830138+ctuguinay@users.noreply.github.com> Co-authored-by: Wu-Jung Lee <leewujung@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add v0.9.1 and v0.10.0 whats new * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…SOceanAcoustics#1466) * update to use python 3.12 and ubuntu 22.04 * update v0.10.0 release notes

…es` [all tests ci] (OSOceanAcoustics#1468) * replace pkg_resources with importlib.resources * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused pkg_resources import * update __name__ to __package__ and use read_text * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove unused pkg_resources again * update whats new --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…SOceanAcoustics#1430) Bumps [pypa/gh-action-pypi-publish](https://github.com/pypa/gh-action-pypi-publish) from 1.12.3 to 1.12.4. - [Release notes](https://github.com/pypa/gh-action-pypi-publish/releases) - [Commits](pypa/gh-action-pypi-publish@v1.12.3...v1.12.4) --- updated-dependencies: - dependency-name: pypa/gh-action-pypi-publish dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…#1469) Bumps [actions/cache](https://github.com/actions/cache) from 4.2.0 to 4.2.1. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4.2.0...v4.2.1) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

updates: - [github.com/PyCQA/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…#1473) Bumps [actions/cache](https://github.com/actions/cache) from 4.2.1 to 4.2.2. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4.2.1...v4.2.2) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…oustics#1481) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.4.0 to 5.5.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@v5.4.0...v5.5.0) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 5.5.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

updates: - [github.com/PyCQA/flake8: 7.1.2 → 7.2.0](PyCQA/flake8@7.1.2...7.2.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

…#1479) Bumps [actions/cache](https://github.com/actions/cache) from 4.2.2 to 4.2.3. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v4.2.2...v4.2.3) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…SOceanAcoustics#1483)

* add range var max and reindex to compute_MVBS parameters and drop flox maximum version * add tests for compute MVBS changes * add todo for compute NASC * add np.nan fill value * Update echopype/commongrid/api.py Co-authored-by: Wu-Jung Lee <leewujung@gmail.com> --------- Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>

review-notebook-app · 2025-04-03T21:12:04Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

use align vectors instead of from matrix for a channel-wise rotation

2883f9d

ctuguinay self-assigned this Feb 18, 2025

ctuguinay added the bug Something isn't working label Feb 18, 2025

ctuguinay added this to the v0.9.2 milestone Feb 18, 2025

ctuguinay changed the title ~~Use align vectors instead of from matrix for a channel-wise rotation~~ Use align vectors for channel-wise rotation Feb 18, 2025

leewujung modified the milestones: v0.9.2, v0.10.0, v0.10.1 Feb 18, 2025

use normalization logic, add warnings, and add tests

5b37e39

ctuguinay changed the title ~~Use align vectors for channel-wise rotation~~ Use normalization to compute echo range scaling Feb 25, 2025

ctuguinay added 2 commits February 25, 2025 01:00

simplify test

3734591

small word change

96c2ea3

ctuguinay requested a review from leewujung February 25, 2025 01:07

ctuguinay marked this pull request as ready for review February 25, 2025 01:07

leewujung reviewed Apr 3, 2025

View reviewed changes

Comment thread echopype/consolidate/ek_depth_utils.py Outdated

leewujung previously approved these changes Apr 3, 2025

View reviewed changes

ctuguinay dismissed leewujung’s stale review via 50e29c5 April 3, 2025 21:11

ctuguinay and others added 9 commits April 3, 2025 14:11

Update echopype/consolidate/ek_depth_utils.py

50e29c5

Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

e06d9cb

for more information, see https://pre-commit.ci

replace previous APA7 citation with recent ICES APA7 citation (OSOcea…

5faf016

…nAcoustics#1455)

pin scipy to temporarily ensure that rotation matrix calculation does…

6186616

… not fail (OSOceanAcoustics#1460)

[pre-commit.ci] pre-commit autoupdate (OSOceanAcoustics#1461)

095d54f

updates: - [github.com/PyCQA/flake8: 7.1.1 → 7.1.2](PyCQA/flake8@7.1.1...7.1.2) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Check if there exist any swap files before cleaning them up [all test…

62a78b0

…s ci] (OSOceanAcoustics#1451) * check if zarr stores > 0 * add a test for echodata delete * fix path * add comments for readability --------- Co-authored-by: Wu-Jung Lee <leewujung@gmail.com>

remove __setattr__ from EchoData (OSOceanAcoustics#1457)

9f1a1f7

compute dask array before np array equal (OSOceanAcoustics#1452)

79cf828

Dataset.dims to Dataset.sizes [all tests ci] (OSOceanAcoustics#1453)

a01785e

* dims to sizes * dims to sizes when using ds.dims * dims to sizes for test echodata combine

ctuguinay and others added 18 commits April 3, 2025 14:11

set decode timedelta to False since it will default to this in later …

2c430fd

…xarray versions (OSOceanAcoustics#1462)

Update workflows to use python 3.12 and ubuntu 22.04 [all tests ci] (O…

e98e213

…SOceanAcoustics#1466) * update to use python 3.12 and ubuntu 22.04 * update v0.10.0 release notes

not -> no (OSOceanAcoustics#1471)

1c1d18f

[pre-commit.ci] pre-commit autoupdate (OSOceanAcoustics#1472)

e0fa5a4

updates: - [github.com/PyCQA/isort: 6.0.0 → 6.0.1](PyCQA/isort@6.0.0...6.0.1) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

[pre-commit.ci] pre-commit autoupdate (OSOceanAcoustics#1480)

f9d30f5

updates: - [github.com/PyCQA/flake8: 7.1.2 → 7.2.0](PyCQA/flake8@7.1.2...7.2.0) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

expand ping time preemptively and add corresponding chunking tests (O…

6bd05d6

…SOceanAcoustics#1483)

ctuguinay added 2 commits April 3, 2025 14:16

merge main

ff3cfaa

indent line of code

a993c55

ctuguinay merged commit b20995b into OSOceanAcoustics:main Apr 3, 2025
5 checks passed

github-project-automation Bot moved this to Done in Echopype <2025 Apr 3, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use normalization to compute echo range scaling#1463

Use normalization to compute echo range scaling#1463
ctuguinay merged 33 commits intoOSOceanAcoustics:mainfrom
ctuguinay:investigate_rotation_matrix_upgrade

ctuguinay commented Feb 18, 2025 •

edited

Loading

Uh oh!

codecov-commenter commented Feb 25, 2025 •

edited

Loading

Uh oh!

ctuguinay commented Feb 25, 2025

Uh oh!

leewujung Apr 3, 2025

Uh oh!

ctuguinay Apr 3, 2025

Uh oh!

leewujung Apr 3, 2025

Uh oh!

Uh oh!

leewujung left a comment

Uh oh!

review-notebook-app Bot commented Apr 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

		# For channels with near-zero norm, we return NaN. Otherwise, we return the normalized
		# z component.

Conversation

ctuguinay commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ctuguinay commented Feb 25, 2025

Uh oh!

leewujung Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

ctuguinay Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

leewujung Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

leewujung left a comment

Choose a reason for hiding this comment

Uh oh!

review-notebook-app Bot commented Apr 3, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ctuguinay commented Feb 18, 2025 •

edited

Loading

codecov-commenter commented Feb 25, 2025 •

edited

Loading