With about 250 lines of code (not counting libdeflate for ZLIB compression...), one can generate PNG files significantly faster than with libpng, and smaller!
Okay - further to my early rants about CDSE data, it aint as bad as I thought it also prompted me to properly sort out my approach to scaling/offsets which had been driving me mad! So if anyone cares for another way to download data from CDSE, with #rspatial / #gdal I made a gist:
https://gist.github.com/h-a-graham/86cd3403445cf163ce958efa2d29c621
There are still some improvements to be made for sure.
FYI @Micha_Silver
okay #GDAL fans - how the hell am I supposed to work with jpeg2000 files and why on God's earth is the Copernicus Data Space Ecosystem using this format? I would much rather make requests to a european server but it looks like I'm expected to just download entire files and nothing is cloud optimised - really hoping I'm wrong...
I've been quiet about {vrtility} recently but I've made some nice steady progress with improved asynch read/write of raster files. A colleague wanted some help getting at the amazing Digital Earth Africa GeoMAD landsat catalogs so I put togther this gist - sharing incase it's useful to anyone else! with an already beautifully processed annual image composite - this is a classic and simple use case for VRT . #rstats #rspatial #gdal https://gist.github.com/h-a-graham/46153e63e25905ba337d93ebe99607c4
Found a cool feature of ogr2ogr: you can use SQL like 'SELECT * EXCEPT (col1, col2) FROM table' with the -sql parameter to exclude a few fields from the conversion process. Handy when you've got lots of fields and don't want to list them all out in the SELECT.
Now I just wish Postgres had this functionality (DuckDB does!)
See https://gdal.org/en/stable/user/ogr_sql_dialect.html#list-operators for details, and I found it from https://github.com/OSGeo/gdal/issues/854
#GDAL is there a way to specify an empty raster source entirely as a text string (compactly)?
I can use MEM and generate a datapointer description from an array in memory, but what about the equivalent of an empty cog? something like
"MEM:::EMPTY_COG,87,61,1,Float64,0/1/0/61/0/-1"
(I certainly don't recommend experimenting with arbitrary values for DATAPOINTER)
Ooh CNN-based cloud masking baked right into a VRT file. This should be fun! #rstats #rspatial #gdal #python
This is using https://github.com/DPIRD-DMA/OmniCloudMask and appears to be a massive improvement on the standard Sentinel 2A SCL band!
@MichaelTBacon @eliocamp here it seems #Rspatial is innocent, #GDAL error is helpful here.
GDAL modernization webinar and FAQ: The #GDAL Project Steering Committee recently hosted a webinar highlighting the major #modernization of the GDAL command-line interface, featuring the unification of commands and improved parameter consistency. You can now catch up on all the...
https://spatialists.ch/posts/2025/06-07-gdal-modernization-webinar-and-faq/ #GIS #GISchat #geospatial #SwissGIS
Upcoming #GDAL cli modernization webinar, free registration (it's 2pm UTC, in 4.3 hours from now)
https://discourse.pangeo.io/t/webinar-about-gdal-3-11-cli-refactor-june-3rd-at-1400-utc/5159
@EvenRouault Look out #numpy, #GDAL is coming for you!
Quite excited by the new artihmetic #GDAL band on-the-fly capabilities of https://github.com/OSGeo/gdal/pull/12507
```python
with gdal.Open("rgb.tif") as ds:
R = ds.GetRasterBand(1)
G = ds.GetRasterBand(2)
B = ds.GetRasterBand(3)
gray = (0.299 * R + 0.587 * G + 0.114 * B).as_type(gdal.GDT_Byte)
gdal.GetDriverByName("GTiff").CreateCopy("gray.tif", gray)
```
Fun #GDAL contest. Craft the smallest file (in terms of byte size) that is recognized (opened by GDAL, using any official driver of your choice) with 2147483647 x 2147483647 raster size. My current record is 27 bytes. Any improvement over that?
With 'gdal raster calc' we can perform pixel-wise calculations on raster data. I am wondering if we could somehow also perform calculations on neighboring pixels?
To calc e.g. slope, focal min, ...
@EvenRouault #gdal
One for the geo format nerds https://element84.com/software-engineering/is-zarr-the-new-cog/
#rspatial #gdal
If you need the latest dev versions of #gdal and #gdalraster for #rstats #rspatial development I made a docker file for that :
Kicked off a HPC job, 64 CPUs, converting global 11 variable netcdf to single var cog, from and to the local object storage.
There's 120K files to be created, and 20 minutes or so in it's up to 37K done, each is 2880x1440, Int32 (zstd, predictor 2)
Blows my mind how fast stuff is when obstacles and complications are removed
...thanks to #GDAL
and #rstats {gdalraster}, and {mirai} via {purrr}