Skip to content

Saving and reopening in Zarr shifts interval from Start to Center #1061

@felixcremer

Description

@felixcremer

When I am saving a dataset to Zarr and reopen it, the Intervals are shifted from Start to Center. I just saw this in a rasterize result and therefore I used this as a MWE, but maybe we could make this even more minimal by setting the dimensions directly.
This also shifts the data from an interval into a explicit representation as a list of floating point values, but this might be a restriction of the zarr model.

julia> points = GeoInterface.Point.(rand(100),rand(100))
100-element Vector{GeoInterface.Wrappers.Point{false, false, Tuple{Float64, Float64}, Nothing}}:
 Point{false, false}((0.34936438645970813, 0.22830783823783907))
 Point{false, false}((0.291604895925729, 0.6838662764562373))
 Point{false, false}((0.26306199948175846, 0.5675371228170234))
 Point{false, false}((0.5917137241194883, 0.07065864730004334))
 Point{false, false}((0.46951306798715253, 0.5490487849797747))
 Point{false, false}((0.9462307675941996, 0.8777301721763594))
 Point{false, false}((0.24375937660104852, 0.5268215625711652))
 Point{false, false}((0.7721677811295526, 0.45611952497040487))
 
 Point{false, false}((0.4632182002710399, 0.8084084988743806))
 Point{false, false}((0.22655224719511935, 0.8955394026808583))
 Point{false, false}((0.9209939423252101, 0.136380658645339))
 Point{false, false}((0.6771003743460674, 0.10433183712336536))
 Point{false, false}((0.18690016364665052, 0.5720007420808291))
 Point{false, false}((0.5754865898050198, 0.6143429403563927))
 Point{false, false}((0.8783819470213879, 0.12229676310128401))
 Point{false, false}((0.15123906160740552, 0.2571979744776246))

julia> ras = rasterize(count, points, res=0.01, to=Extent(X=(0,1),Y=(0,1)))
┌ 101×101 Raster{Int64, 2} count ┐
├────────────────────────────────┴────────────────────────────────────── dims ┐
   X Sampled{Float64} 0.0:0.01:1.0 ForwardOrdered Regular Intervals{Start},
   Y Sampled{Float64} 0.0:0.01:1.0 ForwardOrdered Regular Intervals{Start}
├───────────────────────────────────────────────────────────────────── raster ┤
  missingval: 0
  extent: Extent(X = (0.0, 1.01), Y = (0.0, 1.01))
└─────────────────────────────────────────────────────────────────────────────┘
     0.0  0.01  0.02  0.03  0.04  0.05  0.06    0.93  0.94  0.95  0.96  0.97  0.98  0.99  1.0
 0.0   0    0     0     0     0     0     0        0     0     0     0     0     0     0     0
 0.01  0    0     0     0     0     0     0        0     0     0     0     0     0     0     0
                                                                                        
 0.98  0    0     0     0     0     0     0        0     0     0     0     0     0     0     0
 0.99  0    0     0     0     0     0     0       0     0     0     0     0     0     0     0
 1.0   0    0     0     0     0     0     0        0     0     0     0     0     0     0     0

julia> filename = tempname() * ".zarr"
"/tmp/jl_B2Bi9Gtsvq.zarr"

julia> write(filename, ras, force=true)
"/tmp/jl_B2Bi9Gtsvq.zarr"

julia> rasdisk = Raster(filename) # This gets shifted to Intervals Center101×101 Raster{Union{Missing, Int64}, 2} count ┐
├────────────────────────────────────────────────┴─────────────────────────── dims ┐
   X Mapped{Float64} [0.005, , 1.005] ForwardOrdered Regular Intervals{Center},
   Y Mapped{Float64} [0.005, , 1.005] ForwardOrdered Regular Intervals{Center}
├──────────────────────────────────────────────────────────────────────── metadata ┤
  Metadata{Rasters.Zarrsource} of Dict{String, Any} with 1 entry:
  "_FillValue" => 0
├────────────────────────────────────────────────────────────────────────── raster ┤
  missingval: missing
  extent: Extent(X = (0.0, 1.0099999999999998), Y = (0.0, 1.0099999999999998))
└──────────────────────────────────────────────────────────────────────────────────┘
      0.005     0.015     0.025     0.035     0.045       0.975     0.985     0.995     1.005
                                                                                       
 1.005   missing   missing   missing   missing   missing      missing   missing   missing   missing

julia> ras #This is the on disk data which has Start101×101 Raster{Int64, 2} count ┐
├────────────────────────────────┴────────────────────────────────────── dims ┐
   X Sampled{Float64} 0.0:0.01:1.0 ForwardOrdered Regular Intervals{Start},
   Y Sampled{Float64} 0.0:0.01:1.0 ForwardOrdered Regular Intervals{Start}
├───────────────────────────────────────────────────────────────────── raster ┤
  missingval: 0
  extent: Extent(X = (0.0, 1.01), Y = (0.0, 1.01))
└─────────────────────────────────────────────────────────────────────────────┘
     0.0  0.01  0.02  0.03  0.04  0.05  0.06    0.93  0.94  0.95  0.96  0.97  0.98  0.99  1.0
 0.0   0    0     0     0     0     0     0        0     0     0     0     0     0     0     0
 0.01  0    0     0     0     0     0     0        0     0     0     0     0     0     0     0
                                                                                        
 0.98  0    0     0     0     0     0     0        0     0     0     0     0     0     0     0
 0.99  0    0     0     0     0     0     0       0     0     0     0     0     0     0     0
 1.0   0    0     0     0     0     0     0        0     0     0     0     0     0     0     0

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions