HN Atlas
← All topics

Data engineering

Data pipelines, analytics infrastructure, data products, transformation tools, and data teams.

7,310 stories · 55 in 30 days · Updated Sep 28, 2026

The pace of conversation

Posts per month

2020-01: 162 posts2020-02: 143 posts2020-03: 132 posts2020-04: 139 posts2020-05: 135 posts2020-06: 129 posts2020-07: 117 posts2020-08: 109 posts2020-09: 108 posts2020-10: 146 posts2020-11: 118 posts2020-12: 146 posts2021-01: 106 posts2021-02: 110 posts2021-03: 103 posts2021-04: 99 posts2021-05: 102 posts2021-06: 87 posts2021-07: 85 posts2021-08: 71 posts2021-09: 85 posts2021-10: 101 posts2021-11: 86 posts2021-12: 98 posts2022-01: 97 posts2022-02: 82 posts2022-03: 89 posts2022-04: 91 posts2022-05: 88 posts2022-06: 94 posts2022-07: 84 posts2022-08: 120 posts2022-09: 75 posts2022-10: 74 posts2022-11: 102 posts2022-12: 80 posts2023-01: 90 posts2023-02: 89 posts2023-03: 91 posts2023-04: 66 posts2023-05: 95 posts2023-06: 90 posts2023-07: 89 posts2023-08: 79 posts2023-09: 86 posts2023-10: 83 posts2023-11: 71 posts2023-12: 69 posts2024-01: 75 posts2024-02: 89 posts2024-03: 117 posts2024-04: 73 posts2024-05: 75 posts2024-06: 91 posts2024-07: 90 posts2024-08: 89 posts2024-09: 72 posts2024-10: 95 posts2024-11: 73 posts2024-12: 61 posts2025-01: 83 posts2025-02: 79 posts2025-03: 88 posts2025-04: 51 posts2025-05: 69 posts2025-06: 81 posts2025-07: 69 posts2025-08: 61 posts2025-09: 70 posts2025-10: 74 posts2025-11: 76 posts2025-12: 78 posts2026-01: 74 posts2026-02: 71 posts2026-03: 80 posts2026-04: 67 posts2026-05: 47 posts2026-06: 94 posts2026-07: 83 posts2026-08: 71 posts2026-09: 53 posts
2020-012026-09

Posts per month · first and latest months may be partial.

01 / EXPLORE THROUGH TIME

Timeline

Start with the latest. Scroll back through time, or zoom in for more.

Year by yearUp to 5 stories per year
2026640 stories
Fast drilldown dashboards from a single Parquet file↗ hamiltonulmer.com · HN · 23 comments · 173 points · Aug 24, 2026
What category theory teaches us about dataframes↗ mchav.github.io · HN · 65 comments · 190 points · Mar 29, 2026
The revenge of the data scientist↗ hamel.dev · HN · 37 comments · 174 points · Mar 28, 2026
Show HN: Data Engineering Book – An open source, community-driven guide↗ github.com · HN · 30 comments · 251 points · Feb 13, 2026
How Is Data Stored?↗ makingsoftware.com · HN · 18 comments · 167 points · Feb 10, 2026
2025879 stories
Your data model is your destiny↗ notes.mtb.xyz · HN · 108 comments · 403 points · Oct 14, 2025
Databricks acquires Neon↗ databricks.com · HN · 226 comments · 389 points · May 14, 2025
Announcing the data.gov archive↗ lil.law.harvard.edu · HN · 132 comments · 382 points · Feb 07, 2025
Archivists work to save disappearing data.gov datasets↗ 404media.co · HN · 238 comments · 801 points · Jan 30, 2025
Datadog acquires Quickwit↗ quickwit.io · HN · 115 comments · 296 points · Jan 09, 2025
20241,000 stories
Get me out of data hell↗ ludic.mataroa.blog · HN · 299 comments · 698 points · Oct 31, 2024
ROOT: analyzing petabytes of data scientifically↗ root.cern · HN · 93 comments · 356 points · Jun 01, 2024
Big data is dead (2023)↗ motherduck.com · HN · 457 comments · 585 points · May 27, 2024
The hunt for the missing data type↗ hillelwayne.com · HN · 254 comments · 690 points · Mar 04, 2024
Data Structures for Data-Intensive Applications [pdf] (2023)↗ cs-people.bu.edu · HN · 26 comments · 583 points · Feb 09, 2024
2023998 stories
Understanding Parquet, Iceberg and Data Lakehouses↗ davidgomes.com · HN · 101 comments · 312 points · Dec 29, 2023
Natural Language Corpus Data: Beautiful Data (2009)↗ norvig.com · HN · 3 comments · 53 points · Dec 29, 2023
Data scientists need to learn about significant digits↗ lemire.me · HN · 65 comments · 68 points · Dec 25, 2023
Most Data Work Seems Fundamentally Worthless↗ ludic.mataroa.blog · HN · 20 comments · 57 points · Feb 27, 2023
Big Data Is Dead↗ motherduck.com · HN · 8 comments · 43 points · Feb 07, 2023
20221,076 stories
AI Data Laundering↗ waxy.org · HN · 113 comments · 304 points · Oct 17, 2022
CS109a: Introduction to Data Science – Resources↗ harvard-iacs.github.io · HN · 25 comments · 223 points · Jul 31, 2022
Data Science in Context – Peter Norvig's New Book↗ datascienceincontext.com · HN · 40 comments · 308 points · May 24, 2022
Use fast data algorithms (2021)↗ jolynch.github.io · HN · 34 comments · 221 points · May 02, 2022
Introduction to Probability for Data Science↗ probability4datascience.com · HN · 15 comments · 235 points · Jan 24, 2022
20211,133 stories
Building a data team at a mid-stage startup↗ erikbern.com · HN · 89 comments · 607 points · Jul 08, 2021
Is It Pokemon or Big Data?↗ pixelastic.github.io · HN · 67 comments · 387 points · Apr 11, 2021
Zero Data App↗ 0data.app · HN · 129 comments · 529 points · Mar 06, 2021
Show HN: I wrote a book about using data science to solve “everyday” problems↗ andrewnc.github.io · HN · 96 comments · 632 points · Feb 24, 2021
We don't need data scientists, we need data engineers↗ mihaileric.com · HN · 353 comments · 718 points · Jan 14, 2021
20201,584 stories
Personal Data Warehouses: Reclaiming Your Data↗ simonwillison.net · HN · 59 comments · 436 points · Nov 14, 2020
Emerging Architectures for Modern Data Infrastructure↗ a16z.com · HN · 105 comments · 431 points · Oct 18, 2020
Data structures and algorithms I actually used while working at tech companies↗ blog.pragmaticengineer.com · HN · 524 comments · 1025 points · Jul 15, 2020
Data Science: Reality Doesn't Meet Expectations↗ dfrieds.com · HN · 163 comments · 462 points · Apr 07, 2020
Discovering Millions of Datasets on the Web↗ blog.google · HN · 32 comments · 443 points · Jan 23, 2020
Scroll through time ↕

02 / CATCH UP

The conversation now

Popular stories, with attention fading over roughly two weeks.

The Dataflow Model Revisited↗ vldb.org · HN · 22 comments · 116 points · Sep 06, 2026
Fast drilldown dashboards from a single Parquet file↗ hamiltonulmer.com · HN · 23 comments · 173 points · Aug 24, 2026
Common Crawl Data Stored on a Hugging Face Bucket↗ commoncrawl.org · HN · 1 comments · 23 points · Sep 16, 2026
Show HN: Filament – Fast data movement engine in Go↗ github.com · HN · 6 comments · 31 points · Sep 08, 2026
Launch HN: Hebbian Robotics (YC S26) – Build scalable robotics data pipelines↗ github.com · HN · 12 comments · 46 points · Aug 31, 2026

03 / GO DEEPER

Follow your curiosity

Find a particular story, revisit a year, or see what resonated.

Highest rated · the last 30 days

The Dataflow Model Revisited↗ vldb.org · HN · 22 comments · 116 points · Sep 06, 2026
Launch HN: Hebbian Robotics (YC S26) – Build scalable robotics data pipelines↗ github.com · HN · 12 comments · 46 points · Aug 31, 2026
Show HN: Filament – Fast data movement engine in Go↗ github.com · HN · 6 comments · 31 points · Sep 08, 2026
Common Crawl Data Stored on a Hugging Face Bucket↗ commoncrawl.org · HN · 1 comments · 23 points · Sep 16, 2026
Show HN: Litelink – local-first, embedded stream capture into Iceberg tables↗ github.com · HN · 2 comments · 9 points · Sep 03, 2026
A Bitter Lesson for Data Filtering↗ arxiv.org · HN · 1 comments · 5 points · Sep 18, 2026
Data Science Weekly – Issue 669↗ datascienceweekly.substack.com · HN · 0 comments · 5 points · Sep 17, 2026
Show HN: Give every agent the right data, access and context to get work done↗ lanes.sh · HN · 0 comments · 5 points · Sep 04, 2026
Show HN: Agentic Data Kernal↗ github.com · HN · 0 comments · 5 points · Sep 02, 2026
Building an NBA stats warehouse without writing any of the code↗ jeffknupp719305.substack.com · HN · 1 comments · 4 points · Sep 20, 2026

10 stories shown.

Help shape Hacker Atlas

Useful, confusing, or missing something? I’d love to hear.

Your feedback is private. Your email is only used to reply, never to sign you up for a newsletter.