Sign in

Ritchie Vink

@ritchie46.bsky.social
228 followers 5 following 28 posts

Author and Founder of Polars

PostsRepliesMedia
Ritchie Vink @ritchie46.bsky.social · 24/08/2026
Want to know which engine a Polars expression will run on? We now have engine tags in the reference guide. The gpu tag will follow soon as well. Note that if the engine you selected isn't there, it will transitively fallback to an engine it does support.
010
Ritchie Vink @ritchie46.bsky.social · 17/08/2026
We will ship the first Polars 2.0 release candidate next week. Polars 2.0 will be great for casual Polars users as it will default to the streaming engine, meaning your code will be faster without doing anything. So the first release candidate next week. Give it a spin!
073
Ritchie Vink @ritchie46.bsky.social · 26/05/2026
Super glad to see nested common subplan elimination finally land. We might need to tune somewhat, but the potential query time savings are huge! And that's (by far) not even the biggest thing we'll launch this week. Stay tuned!
030
Ritchie Vink @ritchie46.bsky.social · 16/02/2026
We shipped a major release of Polars Cloud Live query profiles. See all data flowing through and exactly which nodes take up your compute. Aside from that it also lands: - Streaming shuffles - Defaulting to our cost based planner - Streaming and broadcasting ASOF joins github.com/pola-rs/pola...
github.com
Release Polars Cloud client 0.5.0 · pola-rs/polars-cloud-client
Highlights Launch of the Compute Dashboard This release marks the launch of the compute dashboard that is tied to the cluster directly. This allows for direct compute metrics and advanced query pr...
040
Ritchie Vink @ritchie46.bsky.social · 02/02/2026
In 1-2 weeks we land live query profiling in Polars Cloud. See exactly how many rows are consumed and produced per operation. Which operation takes most runtime, and watch the data flow through live, like water. 😍
091
Ritchie Vink @ritchie46.bsky.social · 14/01/2026
ClickBench now runs the Polars streaming engine. Polars is the fastest solution on that benchmark on Parquet file(s) 😎 The speed is there. This year, we will tackle out of core (spill to disk) and distributed to truly tackle scale. benchmark.clickhouse.com#system=-ahi|...
080
Ritchie Vink @ritchie46.bsky.social · 02/12/2025
The pre-release of Polars 1.36 is out. Please give it a try so that we can ensure a stable final release with minimal regressions. It lands a lot of goodies: - Extension types - Lazy pivots - Streaming group_by_dynamic - Float16 support - Nested .over() expressions github.com/pola-rs/pola...
github.com
Release Python Polars 1.36.0-beta.2 · pola-rs/polars
🏆 Highlights Add Extension types (#25322) 🚀 Performance improvements Reduce HuggingFace API calls (#25521) Use strong hash instead of traversal for CSPE equality (#25537) Fix panic in is_between...
071
Ritchie Vink @ritchie46.bsky.social · 03/10/2025
Polars 1.34.0 is out! Any Polars query can be turned into a generator! Aside from that Polars now properly supports decimal types, scan_iceberg is completely native, cross joins can maintain order and much more. Changelogs here: - github.com/pola-rs/pola... - github.com/pola-rs/pola...
060
Ritchie Vink @ritchie46.bsky.social · 11/08/2025
Polars 1.32 is out and it lands a lot! Let's go through a few: 1/4 Selectors are now implemented in Rust and we can finally select arbitrary nested types:
1171
Ritchie Vink @ritchie46.bsky.social · 14/07/2025
Join me the 24th in SF for a @pola.rs meetup! I will be having a talk about Polars, Polars-Cloud and the upcoming distributed engine. NVIDIA will also be doing a talk about their GPU acceleration with Polars-CuDF Hope to see you there! lu.ma/60b6wfs8
lu.ma
Polars Meetup - Polars Cloud and Acceleration · Luma
Join the second edition of our Polars Meetup with talks from Ritchie Vink (Polars) and Vyas Ramasubramani (NVIDIA) to discuss accelerating and scaling…
110
Ritchie Vink @ritchie46.bsky.social · 04/07/2025
No more `with pl.StringCache()` Soon... 🌈
000
Ritchie Vink @ritchie46.bsky.social · 10/06/2025
This Thursday I will join Lawrence Mitchell from @nvidia on the podium during the NVIDIA GTC in Paris. We'll discuss how we made Polars work on the GPU and how it will scale to multi-GPU in the future. On se voit là-bas ! vivatechnology.com/sessions/ses...
vivatechnology.com
GP1085: Scaling DataFrames With Polars - Theme: undefined
Location: NVIDIA GTC PARIS - Pavillon 7 - June 12 1:00 PM 1:45 PM - CET | Resume: Room: N03 Polars is a query engine with a DataFrame frontend designed for fast, efficient data processing. This sess...
020
Reposted by Ritchie Vink
Polars @pola.rs · 01/05/2025
Polars has gotten 4x faster than Polars! 🚀 In the last months, the team has worked incredibly hard on the new-streaming engine and the results pay off. It is incredibly fast, and beats the Polars in-memory engine by a factor of 4 on a 96vCPU machine.
4163
Reposted by Ritchie Vink
RustNL @rustnl.bsky.social · 18/04/2025
** Sponsor announcement ** Polars is a Supporter of RustWeek!  Find out more about them here: pola.rs Thank you @pola.rs for your support! 🙏 More info about RustWeek and tickets: rustweek.org #rustweek #rustlang
021
Ritchie Vink @ritchie46.bsky.social · 12/02/2025
Already got all TPC-H queries running distributed!
070
Ritchie Vink @ritchie46.bsky.social · 25/01/2025
This weeks Polars release we shipped initial Unity Catalog support. This makes integration with Databricks much smoother. Writing features are under development and will follow soon. Full release notes: github.com/pola-rs/pola...
0114
Reposted by Ritchie Vink
Richard Bamattre @bamattre.bsky.social · 23/01/2025
Learning polars has been ... actually a joy? It just makes sense to my #rstats #dplyr trained data muscles #databs #python kevinheavey.github.io/modern-polars/
kevinheavey.github.io
Modern Polars
A side-by-side comparison of the Polars and Pandas libraries.
24512
Ritchie Vink @ritchie46.bsky.social · 19/01/2025
This weeks Polars release has a huge improvement for window functions. They can be an order of magnitude faster. And we can run 20/22 TPC-H queries on the new streaming engine and all on Polars cloud. More will follow soon! ;) See the full release docs here: github.com/pola-rs/pola...
github.com
Release Python Polars 1.20.0 · pola-rs/polars
⚠️ Deprecations Make parameter of str.to_decimal keyword-only (#20570) 🚀 Performance improvements Extend functionality on BitmapBuilder and use in Growables (#20754) Specialize first/last agg fo...
0202
Reposted by Ritchie Vink
George Stagg @gws.phd · 04/01/2025
Recently I've been working on getting #polars running in #pyodide. This was a fun one, even requiring patches to LLVM's #wasm writer! Everything has now been upstreamed and earlier this week Pyodide v0.27.0 released, including a Wasm build of Polars usable in Pyodide, Shinylive and Quarto Live 🎉
A screenshot of a Pyodide REPL executing Polars code:

import polars as pl
import requests
r = requests.get("https://raw.githubusercontent.com/pola-rs/polars/refs/heads/main/examples/datasets/foods2.csv")
pl.read_csv(r.content).group_by("category").mean()A screenshot of a Quarto Live code cell executing Polars code:

import polars as pl
import requests
r = requests.get("https://raw.githubusercontent.com/pola-rs/polars/refs/heads/main/examples/datasets/foods2.csv")
pl.read_csv(r.content).group_by("category").mean()A screenshot of a Shinylive app using Polars code:

from shiny import App, render, ui
import polars as pl
from pathlib import Path

app_ui = ui.page_fluid(
    ui.input_select("cyl", "Select Cylinders", choices=["4", "6", "8"]),
    ui.output_data_frame("filtered_data")
)

def server(input, output, session):
    df = pl.read_csv(Path(__file__).parent / "mtcars.csv")
    
    @output
    @render.data_frame
    def filtered_data():
        return (df
                .filter(pl.col("cyl") == int(input.cyl()))
                .select(["mpg", "cyl", "hp"]))

app = App(app_ui, server)
0499
Reposted by Ritchie Vink
marcogorelli.bsky.social @marcogorelli.bsky.social · 20/12/2024
✨ New temporal feature in the next Polars release! ⏲️ dt.replace lets you replace components of Date / Datetime columns ⚡🦀 It's an expressified vectorised rustified version of the Python standard library datetime.replace
demo of dt.replace
091
Ritchie Vink @ritchie46.bsky.social · 18/12/2024
We removed serde from our Series struct and saw a significant drop in Polars' binary size (of all features activated). The amount of codegen is huge. 😮
050
Ritchie Vink @ritchie46.bsky.social · 12/12/2024
We finally support writing to cloud storage natively and seamlessly!
0182
Reposted by Ritchie Vink
:probabl. @probabl.ai · 10/12/2024
Join us this Friday if you're eager to see what it can be like to design a recommender while limiting ourselves to just a DataFrame API. It is somewhat unconventional, but a great excuse to show off a Polars trick or two. www.youtube.com/watch?v=U3Fi...
youtube.com
Making a recommender by just using Polars!
YouTube video by probabl
0142
Reposted by Ritchie Vink
Polars @pola.rs · 21/11/2024
How to “expand” ranges like "3-5" across new rows with the values 3, 4, 5? This comes straight from our Discord server (discord.com/invite/4UfP5...)
A diagram shows how to use str.split, list.first / list.last, cast, pl.int_ranges, and explode, all together, to turn a dataframe where a column may contain ranges like "3-5" into a similar dataframe where all ranges have been expanded, or exploded, across multiple rows.

The full code is:

range_start = pl.col("nrs").str.split("-").list.first().cast(pl.Int64)
range_end = pl.col("nrs").str.split("-").list.last().cast(pl.Int64)
df.with_columns(pl.int_ranges(range_start, range_end + 1)).explode("nrs")
072
Reposted by Ritchie Vink
Polars @pola.rs · 20/11/2024
Why is there a `struct` data type? A single expression produces a single column, so expressions like `value_counts` need to output structs to map the values to their counts. With that said, do you understand why `.struct.unnest` doesn't break the 1 expr = 1 column principle?
Diagram showing how `value_counts` produces a column with struct values, mapping column values to their counts.
We then show how to use `.struct.field` to extract a single field from the struct and how to use `.struct.unnest` to extract all fields into corresponding columns.
072