GRASS GIS 8: from Desktop to Big Data Cubes

Verónica Andreo
& Vaclav Petras, Martin Landa, Anna Petrasova, Guido Riembauer, Māris Nartišs, Moritz Lennert, Markus Metz, Stefan Blumentrath, Huidae Cho, Markus Neteler

Open Data Science Europe Workshop - Wageningen, 2021

veroandreo.github.io/grass-gis-talks

About the presenter...


https://veroandreo.gitlab.io/

Which GRASS?

  • GRASS GIS (Geographic Resources Analysis Support System), a FOSS suite used for geospatial data management and analysis, image processing, spatial modeling, and visualization.
  • Originally developed by the U.S. Army CERL for land management and environmental planning (1982-1995).
  • 38 years of continuous geospatial development

A bit of (geek) GRASS GIS history...




In case you didn't notice, the video is narrated by William Shatner


GRASS GIS general features

Interfaces

Interfaces

All-in-one software suite

  • All matured tools available right away
  • Download of experimental tools possible
  • Network analysis, hydrology, remote sensing, OBIA, time series, …

Cool stuff that has been there for a while...

64-bit and large file support in all Operating Systems

392 billion cells!

Time series support

by Sören Gebbert, Luca Delucchi, Anna Petrasova, …

FUTURES

r.futures - set of tools for urban growth modeling

Improved scalability, new validation modules

by NC State Center for Geospatial Analytics

PoPS

r.pops.spread - Pest or Pathogen Spread simulation

by NC State Center for Geospatial Analytics

Tangible Landscape

Geospatial tangible user interface

by NC State Center for Geospatial Analytics

Community & Shared Improvements

  • Provides software to the public.
  • Shares its source code.
  • Includes latest additions.

Credit goes to the whole community, not just this talk's authors

Real Global Community

  • Open community of users and developers
  • Contributors to code, documentation, translations, …
  • Project Steering Committee

Scientific Foundations

  • references to related scientific papers
  • references to papers associated with tools or libraries
  • tools used in open science


GRASS GIS google scholar profile

Innovation with stability

  • Graphical user interface changes as needed.
  • Python API added in v6, extended in v7.
  • Basic things continue to work in the same way.


Raster algebra expression from 2002 (v5) in 2021 (v8):

depressions = if((elevation - fill) < 0, 1, 0)

Support


GRASS GIS logo

GRASS GIS 8

  • Roadmap:
    • 7.8.6 Fall 2021
      • Old but still good.
    • 8.0.0 Fall 2021
      • First release in a new era.
    • 8.0.1 Winter 2021
      • What did we miss?
    • 8.2.0 Spring 2022
      • Enjoy the GSoC 2021 results!

Exciting new features in GRASS 8

Dark Theme Support

Interface respects system dark theme.

by Anna Petrasova, Nicklas Larsson

Improved First-time User Experience

Initial project sets up automatically. Guidance provided for next steps.

by Linda Kladivova & rest of the community (many reviews, calls, user surveys, …)

Improved First-time User Experience

Centralized Data Management

All data manipulation centered around revamped Data tab.

by Linda Kladivova, Anna Petrasova, Vaclav Petras

Raster band references and portable signature files

  • assign a band reference (semantic label) to any raster map
  • extended metadata for band reference
  • band references stored in classification signature files
  • signature files can be used for classification if band references match (raster values are semantically equal)

by Māris Nartišs and Martin Landa

Faster External Data Links

r.external links (opens) external raster data (GeoTiffs, …) faster.
(2-5× faster, or almost no time for some workflows)

by Markus Metz

Great for workflows when only portion of the data is processed in GRASS GIS.

LiDAR data import with PDAL

  • import of any point dimension (including user defined ones e.g. from PDAL filters)
  • support of 19 binning methods (including eigenvalues)
  • point filtering by values in any dimension

by Vaclav Petras and Māris Nartišs

Skeletons and Centerlines

v.voronoi tool can now create area skeletons and centerlines.

by Markus Metz

Latest PROJ and GDAL

  • Support for PROJ >=6 added.
  • Support for GDAL >=3 added.
  • WKT2 can be used for CRS definitions.
  • High-accuracy coordinate transformations supported.

by Markus Metz

Improved and robust code quality checks

  • Python code formatted by Black.
  • Python code checked against Flake8.
  • Most of warnings in C code removed and now checked.
  • Code tested with CodeQL.
  • Flake8 and Black checks also for Addons.

by Nicklas Larsson, Vaclav Petras, Anna Petrasova, Carmen Tawalika, ...

Google Summer of Code 2021

OpenMP parallelization by Aaron Saw Min Sern

  • r.series
  • r.univar
  • r.neighbors
  • r.patch
  • r.resamp.interp
  • r.resamp.filter
  • r.mfilter
  • r.slope.aspect

+ benchmarking library

Coming in 8.2

Google Summer of Code 2021

Single-Window GUI by Linda Kladivova

Code refactoring to enable Single-Window GUI with dockable widgets.

Coming in 8.2

Google Summer of Code 2021

Integration of GRASS GIS and Jupyter Notebooks by Caitlin Haedrich

New Python library that simplifies the launch of GRASS GIS in Jupyter and the interactive display of raster/vector data.

Many new addons contributed by the community


Spatial Query of Projections

g.projpicker queries projections spatially using user-drawn geometries and set-theoretic logical operators. It requires ProjPicker.

by Huidae Cho

Hydrologic Parameters Using a Flow Direction Raster

r.accumulate calculates weighted flow accumulation, subwatersheds, stream networks, and longest flow paths using a flow direction map.

by Huidae Cho

Creating a Gabor filter bank

i.gabor creates directional filters for image segmentation using i.segment. It requires NumPy and SciPy.

by Owen Smith

Ecological applications

r.suitability.regions allows to identify suitable regions, e.g., for species in danger, starting from suitability maps.

by Paulo van Breugel

Weighted layers for dasymetric mapping

r.area.createweight creates a weighting layer for dasymetric mapping using a random forest regression model.

by Charlotte Flasse, Tais Grippa and Safa Fennia. DOI.

Can you see that landslide??

r.survey allows to assess whether objects of certain size could be detected by an observer moving along roads or sitting on a flying object

by Ivan Marchesini

Tap directly into FAIR data warehouses

GRASS GIS understands netCDF data that follows the CF-convention

  • m.crawl.thredds: Lists URLs for netCDF datasets on Thredds servers
  • t.rast.import.netcdf: Makes Spatio-temporal data in netCDF format directly available for analysis in GRASS STRDS (also without downloading)

Enjoy data delivered right into your GRASS GIS database from e.g.:

by Stefan Blumentrath (Norwegian Institute for Nature Research - NINA) …

GRASS loves Remote Sensing


Dedicated tools for image processing and analysis

  • 50+ core modules
  • 60+ addons, e.g., OBIA chain

Sentinel, MODIS, Landsat, NED, NAIP

Automated download and import of common datasets i.sentinel, i.modis, i.landsat, r.in.usgs, r.in.nasadem, …

by Luca Delucchi (Fondazione Edmund Mach), Martin Landa (OpenGeoLabs), Anika Weinmann (mundialis), Guido Riembauer (mundialis), Roberta Fagandini (GSoC), Zechariah Krautwurst (GSoC), Anna Petrasova (NC State University), Vaclav Petras (NC State University), Veronica Andreo (CONICET), …

GRASS Sentinel Addons


Access & Downloading → Importing → Preprocessing → Analysis


Contains modified Copernicus Sentinel data (2021)

Access & Downloading

  • i.sentinel.coverage
  • i.sentinel.download
    • ESA Copernicus Open Access Hub
    • USGS Earth Explorer
    • Google Cloud Storage

Importing

  • i.sentinel.import
    • selection of individual bands
    • import of metadata, band reference, cloud masks
  • t.sentinel.import
  • r.s1.grd.import

Importing

  • Importing into a space-time raster dataset with t.sentinel.import

Preprocessing

  • i.sentinel.mask
  • t.sentinel.mask
  • i.sentinel.preproc
  • i.sentinel-2.sen2cor
  • t.rast.mosaic
  • i.sar.speckle

Preprocessing: cloud masking with i.sentinel.mask


Preprocessing: temporal mosaicking with t.rast.mosaic


Preprocessing: Sentinel-1 data

  • r.s1.grd.import
    • requires ESA's SNAP software
    • complete SAR preprocessing chain

GRASS is ready for the cloud

GRASS in docker containers


  • Different flavours
  • Built and updated automatically
  • Can be deployed anywhere you need them

See https://grass.osgeo.org/download/docker/

actinia: The GRASS GIS REST API

by Markus Neteler, Carmen Tawalika, Anika Weinmann, …

Application examples

GRASS for remote sensing analysis in the cloud


Daily LST space-time reconstruction done with GRASS - Metz et al. 2017.

Germany land-cover mapping based on Sentinel-2 data

GRASS modules in use (excerpt):
  • i.vi
  • t.rast.series
  • r.learn.ml2
  • r.change.info


For details, follow this blog post.

Forest monitoring

GRASS modules in use (excerpt):
  • r.mapcalc
  • r.learn.ml2
  • r.texture
  • r.resamp.interp

Register and try the HERMOSA demo at https://hermosa.earth

Flood Mapping in Ecuador based on Sentinel-1 data

GRASS modules in use (excerpt):
  • r.mapcalc
  • i.segment
  • t.rast.algebra
  • t.rast.series

For details, follow this blog post.

Plenty of resources

Access to Source Code

Documentation links to the source code.

by Luca Delucchi

Get involved! All contributions are welcome!

Code contributions


GRASS GIS development is GitHub-centered: core, addons, website

  • File bug reports or feature requests
  • All issues and PR's are publicly visible
  • Ask, comment, suggest also in Github Discussions
  • "Fork me on GitHub" and suggest changes or fix bugs via pull requests
  • Create your own addon! See this nice workshop for a guideline: How to write a Python tool for GRASS

Other contributions are relevant too!

  • Translation: we use Transifex
  • Documentation: start by fixing typos in manual pages, add examples where missing, create cool screenshots, write tutorials in the wiki, etc.
  • Contribute material for our social media
  • Write a blog post for our website
  • Bring your own ideas!

Sponsoring


  • Money donations are very important for GRASS GIS development. They allow us to:
    • organize face-to-face coding sessions (sprints)
    • finance infrastucture needs (web site, etc)
    • pay developers to implement new features and fix specific important bugs

Sponsoring



"One of the greatest benefits of GRASS GIS is that its environments gives us a plethora of options for manipulating data and testing/designing our automation/workflow processes."


https://opencollective.com/grass

Meet the GRASS team at FOSS4G by the end of the month!


Have a look at the amazing list of workshops and talks offered and register here!


You can also join us at the code sprint!

Thanks for your attention!!


GRASS GIS logo

grass.osgeo.org


VeronicaAndreo
veroandreo.gitlab.io