Sign in

Erfan Hesami

@hopefanhe.bsky.social
31 followers 39 following 11 posts

Data and AI Engineer Substack: pipeline2insights.substack.com

PostsRepliesMedia
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 19/09/2026
As a data engineer, you probably already: Gather requirements from people who don't speak in specs Work with systems you didn't design Build quickly when requirements are unclear Translate between business problems and technical systems Those aren't just DE skills. They're core FDE skills, too.
pipeline2insights.substack.com
From Data Engineer to Forward Deployed Engineer
What data engineers already know, what they need to learn, and how to practise it in your current job.
021
Reposted by Erfan Hesami
Yordan Ivanov @ivanovyordan.com · 16/03/2026
Stakeholders don't want data. They want to feel less wrong. Your job isn't to find the truth and present it. It's to help someone decide better than they would without you. Those are different jobs.
011
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 08/03/2026
What does a good GitHub portfolio actually look like? What kind of projects show real skills? How do you go beyond tutorials? And how should you explain what you’ve built? @ivanovyordan.com Answer these questions here. #databs #datasky #dataengineering
substack.com
The Data Engineer’s GitHub Portfolio (2026 Edition)
If you want the interview, your GitHub must prove you can design systems, handle infrastructure, and make engineering trade-offs
042
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 28/11/2025
How to Learn Data Engineering in 2026 Thinking about learning Data Engineering or transitioning into a Data Engineering role? Curious how AI advancements have changed the field, but still unsure where to start, what to focus on, or how to get going? This guide can help #databs #dataengineering
pipeline2insights.substack.com
How to Learn Data Engineering in 2026
The essential fundamentals you need to know and how to leverage AI tools to stay ahead of the curve in Data Engineering.
021
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 15/03/2025
Data Compression in SQL In this post, we’ll explore: - What is Data Compression? - Benefits of Data Compression in SQL. - Types of Compression in SQL. - Comparison of SQL Compression Techniques. - When to Use Compression and When to Avoid. #databs #datasky
pipeline2insights.substack.com
Data Compression in SQL
How to Store More and Query Faster in SQL
132
Erfan Hesami @hopefanhe.bsky.social · 11/01/2025
Explore key data modelling concepts for databases and data warehouses, including: - Normalisation vs. Denormalisation - 3NF - Dimensional Modeling - Star vs. Snowflake Schema comparisons. #databs #datasky
open.substack.com
Data Modelling Fundamentals: Normalisation, 3NF and Dimensional Modelling
Normalisation, 3NF, and dimensional modelling, with insights into Star and Snowflake schemas for efficient database and warehouse design
010
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 04/01/2025
11 Storage Formats for Data Engineers Efficient data starts with the right storage format. Explore 11 formats every data engineer should know to match workloads and scale seamlessly. Highlights: Row & Columnar, Key-Value, Document, Graph, Time-Series, Hybrid. #databs #datasky
pipeline2insights.substack.com
11 Storage Formats for Data Engineers
How to leverage storage formats for efficient and scalable data systems
053
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 17/12/2024
Week 6 of '100 Days of SQL Optimisation': Focused on DuckDB, leveraging columnar storage, sorted data, temp tables, Parquet, and optimal data types to boost efficiency. See how in-memory execution and smart structures enhance query performance! @duckdb.org #dataBS #datasky #duckdb
open.substack.com
Week #6: 100 Days of SQL Optimisation
Exploring DuckDB and Its Capabilities
141
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 08/12/2024
We are starting a 32-week Data Engineering Interview Guide program, covering everything from fundamentals to advanced topics, with sessions every Saturday. Do you think we're missing any critical topics? We're curious about your opinions😊 #dataBS #datasky
open.substack.com
Week 0/32 - A Comprehensive Data Engineering Interview Preparation Guide
Join us every Saturday on This New Journey
063
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 04/12/2024
As a Data Engineer, understanding the data storage lifecycle and data retention policies is critical for designing efficient, cost-effective, and compliant data systems. @joereis.bsky.social #dataBS #datasky substack.com/@pipeline2in...
082
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 03/12/2024
In our new post, we've covered 10 of the most popular data pipeline design patterns. We’d love to hear your thoughts. For more details, please check out the full post created by (@hgeren.bsky.social and @hopefanhe.bsky.social ): open.substack.com/pub/pipeline... #dataBS #datasky
open.substack.com
10 Pipeline Design Patterns for Data Engineers
How to leverage Design Patterns for scalable and efficient data pipelines
042
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 01/12/2024
Discover how dlt simplifies data ingestion. Learn its origins and real-world use cases. Follow a step-by-step guide to build your first pipeline and join the growing dlt community! @matthausk.bsky.social @datateam.bsky.social @hgeren.bsky.social @hopefanhe.bsky.social #dataBS #datasky
open.substack.com
Introduction to data load tool (dlt): A Python Library for Simple Data Ingestion
Discover the basics of dlt and its role in modern data engineering workflows
293
Reposted by Erfan Hesami
Pipeline To Insights @pipeline2insights.bsky.social · 26/11/2024
Storage is at the heart of Data Engineering. In this post, we explore the hierarchy of data storage from the ground up, drawing inspiration from Fundamentals of Data Engineering by @joereis.bsky.social and Matt Housley, as well as insights from the DE Professionals on Coursera. #dataBS #datasky
open.substack.com
Storage Fundamentals For Data Engineers
Why organised and durable storage is the cornerstone of Data Engineering?
3162
Erfan Hesami @hopefanhe.bsky.social · 25/11/2024
My curiosity about the data life cycle led me to explore Data Engineering.Reading Fundamentals of DE book built my foundation, while pioneers like Zach Wilson and Joe Reis inspired me. Community engagement was key to my success, enabling learning, sharing,and growth in this journey. #databs #datasky
open.substack.com
From Analytics to Data Engineering: A Career Journey
How curiosity and continuous learning drive the transition from Analytics to Data Engineering
060
Reposted by Erfan Hesami
Hasan Geren @hgeren.bsky.social · 19/11/2024
Hey #dataBS and #datasky folks, Our new post about "how understanding Big O Notation & Execution Plans can optimize SQL queries" has just been posted. Check it out if you're interested, and we'd love to hear your thoughts! @hopefanhe.bsky.social open.substack.com/pub/pipeline...
open.substack.com
SQL Behind the Curtain: How Are Queries Executed?
Explore the journey of your SQL query guided by execution plans
182
Erfan Hesami @hopefanhe.bsky.social · 12/11/2024
The last step in the data engineering lifecycle is serving data, making it accessible for analytics, machine learning, and Reverse ETL. In our latest post, we break down how data engineers play a crucial role in: - Analytics - ML - Reverse ETL #databs #datasky open.substack.com/pub/pipeline...
open.substack.com
From Data to Decisions: A guide to serving insights for maximum impact
Master the final step in data engineering, serving data, to empower stakeholders, drive smarter decisions, and transform insights into action.
040
Erfan Hesami @hopefanhe.bsky.social · 10/11/2024
Free 6-week Bootcamp with new video every day from November 15th to December 31st 2024 by @eczachly.bsky.social Watch the launch video here:https://www.youtube.com/watch?v=myhe0LXpCeo Don’t miss it Thank you Zach 🙏 #dataBS #datasky
020
Erfan Hesami @hopefanhe.bsky.social · 07/11/2024
Week 1 of "100 Days of SQL Optimisation" covered key techniques like column selection, multicolumn indexes, filtering, window functions, Rank, CTE and composite indexes with IMDb data. Check out the full post for more! @hgeren.bsky.social #dataBS #datasky
open.substack.com
Week #1: 100 Days of SQL Optimisation
How Small Tweaks Transformed Our Queries, Saving Time and Resources
061