iiinova — where your digital self keeps evolving

Reposted by Zarr

scverse @scverse.bsky.social · 11/08/2025

Anndata now supports @zarr.dev v3, great for both performance and interoperability. Check out our guide for more info, especially sharding: anndata.readthedocs.io/en/stable/tu.... Don’t worry though, we remain fully v2 backwards compatible - upgrade fearlessly!

172

Reposted by Zarr

Al Merose @al.merose.com · 27/06/2025

I've been waiting a few years for this: Zarr v2 support was just launched in Google Earth Engine! Check out this demo that loads ARCO-ERA5 into GEE 🔥 code.earthengine.google.com/67ac1e48848d... developers.google.com/earth-engine...

The Google Earth Engine code editor displaying an evaporation map loaded directly from ARCO-ERA5! The black and white map overlooks the continental US. It's easy to make out land masses, but there are also cloud-looking visuals on the map.

1174

Reposted by Zarr

Lazaro Alonso @lazarusa.bsky.social · 24/06/2025

If you missed today's presentation at the LPS by @thejeran.bsky.social on Browzarr, a browser-based visualization toolkit for exploring and analyzing Zarr data stores, go and check out the version online, and open issues and do feature requests on GitHub. github.com/EarthyScienc...

081

Reposted by Zarr

Petra Korlević @petrathepostdoc.bsky.social · 18/06/2025

doodles of two Tree of Life department speakers @sangerinstitute.bsky.social this week, Evgeny Zakharov talking about insanely large scale barcode sequencing and Jerome Kelleher talking about large scale human genomic data analysis. that's a lot of data 🧬📄

comic doodle of Evgeny Zakharov in a suit and stripey shirt, with a small plane to the left, while DNA strands and bases are above, a camera is taking a photo of a feather and a wasp, the coloring is grey shades on a red background.

comic doodle of Jerome Kelleher in a t-shirt and stripey shirt, the background is full of small peon people with DNA strands in them and 4 files with the word ZARR written on them, the coloring is grey shades on a green background.

041

Zarr @zarr.dev · 11/06/2025

ICYMI: Check out this alternative storage approach for variation data using #Zarr. Rather than grouping all data for a given variant together, its groups all data for a given field, and store as chunked, compressed N-D arrays (tensors). 🤩 doi.org/10.1093/giga...

doi.org

Analysis-ready VCF at Biobank scale using Zarr

AbstractBackground. Variant Call Format (VCF) is the standard file format for interchanging genetic variation data and associated quality control metrics.

070

Reposted by Zarr

Rita Strack @ritastrack.bsky.social · 13/05/2025

OME-Zarr 4 life

193

Reposted by Zarr

Kit Macleod @kitmacleod.bsky.social · 19/05/2025

"The commitment from major entities like Copernicus, USGS, and Google Earth Engine signals a paradigm shift in how we will store, access, and analyze geospatial data in the cloud... #Zarr is poised to become a cornerstone of the cloud-native geospatial ecosystem, unlocking new frontiers..."

032

Reposted by Zarr

Earthmover @earthmover.io · 07/05/2025

1/ Our team just got back from an inspiring week at the @cloudnativegeo.org conference in Snowbird, Utah, and one thing is clear: 𝐙𝐚𝐫𝐫 𝐚𝐝𝐨𝐩𝐭𝐢𝐨𝐧 𝐢𝐬 𝐬𝐮𝐫𝐠𝐢𝐧𝐠 𝐚𝐜𝐫𝐨𝐬𝐬 𝐠𝐞𝐨𝐬𝐩𝐚𝐭𝐢𝐚𝐥.

193

Reposted by Zarr

Copernicus ECMWF @copernicusecmwf.bsky.social · 18/03/2025

Curious about climate patterns? The new ERA Explorer app lets you compare 85 years of climate data worldwide. Analyse climatological averages at any map point or explore month-by-month data.Dive deeper with ERA Explorer! era-explorer.climate.copernicus.eu?lat=50.86&ln... #C3S #beautifuldata

38146

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · 07/05/2025

Just back from #CNG2025, I wrote up some thoughts on the traction the @zarr.dev format is gaining in the broader geospatial industry (specifically Earth Observation). I also summarize what features and tools the user and developer communities are asking for to fully leverage Zarr. 👇👇👇

042

Reposted by Zarr

Earthmover @earthmover.io · 17/04/2025

1/ 💡 Our latest blog post in the fundamentals series, written by @tegnicholas.bsky.social, demystifies cloud-optimized scientific data formats! Read more: earthmover.io/blog/fundame...

earthmover.io

Fundamentals: What is Cloud-Optimized Scientific Data?

What cloud-optimized data really means, and how Zarr and Icechunk enable fast access to massive scientific datasets in cloud object storage.

2159

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · 16/04/2025

🚀 Big launch today: introducing Flux — a game-changing way to serve geospatial data via standard APIs in seconds. Built for scale, speed, and simplicity.

153

Reposted by Zarr

Ryan Abernathey @rabernat.bsky.social · 03/04/2025

This one was a lot of fun to work on! Why @xarray.bsky.social and @zarr.dev are faster than DuckDB and Parquet for gridded data...from first principles.

083

Reposted by Zarr

Copernicus ECMWF @copernicusecmwf.bsky.social · 20/03/2025

What's behind ERA Explorer's speed? Meet the ZARR archive! 🚀 ZARR revolutionises data processing, enabling fast, efficient access to over 80 years of climate data. Test out its speed 👉 era-explorer.climate.copernicus.eu?lat=50.86&ln... #C3S @zarr.dev #beautifuldata

1179

Reposted by Zarr

Jackson Burns @fastprop.bsky.social · 24/03/2025

Just started using Zarr - literally unbelievably good.

022

Zarr @zarr.dev · 29/03/2025

🤩 " #NASA ... leveraged #Icechunk’s virtualization capabilities to present the entire collection of over a #million files as a single analysis-ready #Zarr data cube. "

061

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · 28/03/2025

🚨 New blog post 🚨 In it, we show off our recent work deploying #icechunk on top of #NASA's existing archives of Earth observation data. The results: 100x speed up when extracting time series from existing datasets stored as netCDF.

0114

Reposted by Zarr

Al Merose @al.merose.com · 07/03/2025

Really cool new neuroscience benchmark just dropped. Data is in @zarr.dev and demos use tensorstore. zapbench-release.storage.googleapis.com/landing.html

zapbench-release.storage.googleapis.com

ZAPBench

ZAPBench evaluates how well different models can predict the activity of over 70,000 neurons in a novel larval zebrafish dataset.

052

Reposted by Zarr

Michael Morehead @michael-syglass.bsky.social · 14/01/2025

Very happy to announce our winter release of syGlass: 2.2.0! Now control syGlass with your voice. We've also upgraded the auto-segmentation, mask handling & proofreading, and the narration system. We've added basic support for @zarr.dev. Check out the changelog: www.syglass.io/blog/announc...

0147

Reposted by Zarr

Chris Holmes @opencholmes.bsky.social · 10/02/2025

You, yes you, can help the Cloud Native Geospatial Movement! Have you benefitted from Cloud-Optimized GeoTIFF’s? SpatioTemporal Asset Catalogs? Zarr, COPC or GeoParquet? And the ecosystem around them? Learn how to give back in my post: cholmes.medium.com/you-yes-you-...

cholmes.medium.com

You, yes you, can help the Cloud Native Geospatial Movement!

Have you benefitted from COG’s, STAC, Zarr, COPC or GeoParquet? Join us in Utah for the first CNG Conference and help just by attending.

174

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · 17/01/2025

If you missed the Zarr-Python 3 release party last week, no worries because today, we released zarr-python 3.0.1 and numcodecs 0.15.0. release notes 👉 zarr.readthedocs.io/en/stable/re...

zarr.readthedocs.io

Release notes — zarr 3.0.1 documentation

0102

Reposted by Zarr

Aafke Gros @aafkegros.bsky.social · 15/01/2025

Microscopy Nodes loads data from a @zarr.dev bucket, making it easy to use open data archives that host Zarr data! This is a mitotic cell from Walther et al. 2018, that's hosted publicly on the Image Data Resource 😱

1102

Zarr @zarr.dev · 10/01/2025

😍 That's what friends are for!! #opensource #scientific #python #ftw 💪

042

Zarr @zarr.dev · 09/01/2025

🎉 Zarr-Python 3 is here! 🎉 - Full support for Zarr v3 spec - Chunk-sharding for more efficient data storage - Major performance boosts with async I/O & parallel compression 💻 pip install --upgrade zarr 💻 conda install --channel conda-forge zarr Blog post: buff.ly/3C3OwYw

25520

Zarr @zarr.dev · 04/01/2025

Hi, Zarr community! 👋 🚨 Big news: Zarr-Python 3.0 is dropping next week! 🚨 If your project depends on Zarr-Python, make sure to check out the migration guide to get ready for the update. Guide: buff.ly/3W0cLxd More updates soon! 🧑🏻‍💻

buff.ly

02713

Reposted by Zarr

Kyle Harrington @kyleharrington.com · 29/12/2024

For the @napari.fosstodon.org.ap.brid.gy and large image data out there, I made some updates to the tiled loading zarr demo. It still has limitations (scale isn't respected, etc.), but this is an easy way to run it on your own code. github.com/napari/napar... More in 🧵👇

github.com

[wip] Example files for 2D and 3D progressive loading by kephale · Pull Request #6043 · napari/napari

Fixes/Closes This is the first step at addressing #5561 and https://github.com/orgs/napari/projects/24?pane=issue&itemId=29670723 Description This PR introduces: napari.experimental._progressi...

191

Reposted by Zarr

TEGNicholas.bsky.social @tegnicholas.bsky.social · 20/12/2024

More of this from all the companies that depend on the same tools the scientists use/develop please

042

Zarr @zarr.dev · 16/12/2024

Looking for a job where you get to play with #Zarr 🧊🧊 🧊🧊❔ 👇

020

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · 16/12/2024

In the talk, I made a few simple points: - Separation of storage and compute is key to unlocking the scaling potential of cloud - Cloud optimized data formats are key (example: @zarr.dev and #icechunk) - API services should be stateless/serverless and should be able to scale horizontally [0->N]

121

Zarr @zarr.dev · 11/12/2024

Hi Zarrites! 👋🏻 We have our bi-weekly community meeting today. 🗓️ These meetings are a great place to share updates, showcase your projects, raise an issue, or converse with the vibrant community. 🧑🏻‍💻  Join here: buff.ly/3SHCzNN

020

Zarr @zarr.dev · 10/12/2024

Please help the #Zarr community welcome @normanrz.bsky.social as the newest member of the Steering Council! 👏🏼 zarr.dev/blog/steerin...

zarr.dev

Steering council membership update

Please welcome the new steering council member, Norman Rzepka (normanrz)!

0104

Reposted by Zarr

TEGNicholas.bsky.social @tegnicholas.bsky.social · 10/12/2024

I'm at #AGU2024 this week! If you want to talk about anything @xarray.bsky.social , Pangeo, @zarr.dev , or @cworthy.bsky.social , message me! I have a poster on the VirtualiZarr package I wrote on Wednesday morning agu.confex.com/agu/agu24/me... but also several other projects I've worked on...

agu.confex.com

VirtualiZarr: Create Virtual Zarr Stores Using Xarray Syntax

VirtualiZarr is a tool for creating “virtual” Zarr stores. Virtual Zarr stores ...

1115

Reposted by Zarr

Al Merose @al.merose.com · 07/12/2024

I have a few ideas: - create kurchunk indexes on the corpus via @tomnicholas.bsky.social’s VirtualiZarr github.com/zarr-develop... - Use @openclimatefix.org’s hypergrib github.com/JackKelly/hy... 1/

github.com

GitHub - zarr-developers/VirtualiZarr: Create virtual Zarr stores from archival data files using xarray syntax

Create virtual Zarr stores from archival data files using xarray syntax - zarr-developers/VirtualiZarr

131

Reposted by Zarr

Development Seed @developmentseed.org · 06/12/2024

🌍 IN21D: Scaling Data Discovery w Dynamic Spatial Aggregations Using STAC Metadata (@emmapaz.com) 🌍 GC23A: Choosing Earth Foundation Models: From Research Grade to Community Maintained (Wei Ji Leong) 🌍 IN31C: VirtualiZarr: Create Virtual Zarr Stores Using Xarray Syntax (Aimee, Max & Sean) #AGU24

141

Reposted by Zarr

Daniel Rothenberg @danielrothenberg.com · 04/12/2024

Also nice coverage in the @nytimes.com, although not sure I'd call GenCast an "AI Agent" - www.nytimes.com/2024/12/04/s... ... and I guess some e-mails are warranted to clarify, but I hope to see forecasts published elsewhere than Earth Engine and BigQuery. Standard Zarr files on GCS, please?

nytimes.com

Google Introduces A.I. Agent That Aces 15-Day Weather Forecasts

GenCast, from the company’s DeepMind division, outperformed the world’s best predictions of deadly storms as well as everyday weather.

181

Reposted by Zarr

Joe Hamman @jhamman.bsky.social · 04/12/2024

Also on Thursday afternoon, I'll be giving an invited talk titled "Seamless Arrays: A Full Stack, Cloud-Native Architecture for Fast, Scalable Data Access". It combines all that we've been working on for the past year including @zarr.dev v3, #icechunk, and Xpublish. agu.confex.com/agu/agu24/me...

agu.confex.com

Seamless Arrays: A Full Stack, Cloud-Native Architecture for Fast, Scalable Data Access

Just about everyone agrees on what the ideal Earth system data service would pr...

131

Zarr @zarr.dev · 29/11/2024

Thanks to gitub.com/ldeakin the list of #zarr implementations is now shiny and new! ✨ zarr.dev/implementati... 👈 If any are missing or need updating, please let us know.

Tall, narrow screenshot of https://zarr.dev/implementations/ showing all the language implementations of Zarr and their current status.

091

Reposted by Zarr

Aafke Gros @aafkegros.bsky.social · 21/10/2024

And with OME-Zarr loading, due to the pyramidal format, you can design your video at small-scale and only replace to full-scale at the final render! (data from Granita Lokaj, @ s3.embl.de/microscopyno...)

1173