← All topics
Data engineering
Data pipelines, analytics infrastructure, data products, transformation tools, and data teams.
15,325 stories · 45 in 7 days · Updated Nov 21, 2026
01 / CATCH UP
The conversation now
Popular recent stories, shown newest first.
Monetize your company's operational data↗ mainstreetwealth.ai · HN · 0 comments · 5 points · Nov 16, 2026
The Dawn of the Data Intelligence Engineer↗ motherduck.com · HN · 78 comments · 160 points · Nov 13, 2026
Keep up with this topicA few highlights, once a week.Get the weekly email +
Weekly highlights from Data engineering, Sundays at 18:00 Stockholm time.
One email a week about this topic. Unsubscribe any time.02 / EXPLORE THROUGH TIME
Timeline
Start with the latest. Scroll back through time, or zoom in for more.
Year by yearUp to 5 stories per year
20261,723 stories
PgDog is funded and coming to a database near you↗ pgdog.dev · HN · 261 comments · 548 points · Jun 10, 2026
Atlassian enables default data collection to train AI↗ letsdatascience.com · HN · 136 comments · 604 points · Apr 20, 2026
ICE using Palantir tool that feeds on Medicaid data↗ eff.org · HN · 969 comments · 1475 points · Jan 25, 2026
Ask HN: How are you doing RAG locally?↗ news.ycombinator.com · HN · 156 comments · 413 points · Jan 14, 2026
20251,907 stories
Production RAG: what I learned from processing 5M+ documents↗ blog.abdellatif.io · HN · 114 comments · 551 points · Oct 20, 2025
Archivists work to save disappearing data.gov datasets↗ 404media.co · HN · 238 comments · 801 points · Jan 30, 2025
20242,123 stories
Data Structures for Data-Intensive Applications [pdf] (2023)↗ cs-people.bu.edu · HN · 26 comments · 583 points · Feb 09, 2024
SQL for data scientists in 100 queries↗ gvwilson.github.io · HN · 158 comments · 834 points · Feb 06, 2024
20232,123 stories
Understanding Parquet, Iceberg and Data Lakehouses↗ davidgomes.com · HN · 101 comments · 312 points · Dec 29, 2023
Natural Language Corpus Data: Beautiful Data (2009)↗ norvig.com · HN · 3 comments · 53 points · Dec 29, 2023
Data scientists need to learn about significant digits↗ lemire.me · HN · 65 comments · 68 points · Dec 25, 2023
Most Data Work Seems Fundamentally Worthless↗ ludic.mataroa.blog · HN · 20 comments · 57 points · Feb 27, 2023
DataDog asked OpenTelemetry contributor to kill pull request↗ github.com · HN · 29 comments · 143 points · Jan 26, 2023
20222,152 stories
Show HN: Open Prompts – dataset of 10M Stable Diffusion generations↗ github.com · HN · 71 comments · 279 points · Sep 22, 2022
Data Science in Context – Peter Norvig's New Book↗ datascienceincontext.com · HN · 40 comments · 308 points · May 24, 2022
Lessons learned from running Apache Airflow at scale↗ shopify.engineering · HN · 182 comments · 285 points · May 23, 2022
20212,176 stories
Building a data team at a mid-stage startup↗ erikbern.com · HN · 89 comments · 607 points · Jul 08, 2021
Show HN: I wrote a book about using data science to solve “everyday” problems↗ andrewnc.github.io · HN · 96 comments · 632 points · Feb 24, 2021
We don't need data scientists, we need data engineers↗ mihaileric.com · HN · 353 comments · 718 points · Jan 14, 2021
20203,121 stories
Simdjson: Parsing gigabytes of JSON per second↗ github.com · HN · 151 comments · 503 points · Aug 06, 2020
You Want to See My Data? I Thought We Were Friends↗ nautil.us · HN · 134 comments · 489 points · Jul 30, 2020
Data structures and algorithms I actually used while working at tech companies↗ blog.pragmaticengineer.com · HN · 524 comments · 1025 points · Jul 15, 2020
Bayesian Data Analysis, Third Edition [pdf]↗ users.aalto.fi · HN · 48 comments · 466 points · May 06, 2020
Nov 2026117 stories
The Dawn of the Data Intelligence Engineer↗ motherduck.com · HN · 78 comments · 160 points · Nov 13, 2026
The U.S. Census Bureau and "Super Intelligence"↗ simonwillison.net · HN · 54 comments · 127 points · Nov 02, 2026
Oct 2026169 stories
Keplearn: Extract Equations from Data in Seconds↗ github.com · HN · 19 comments · 108 points · Oct 29, 2026
Ontology-Driven Observability: Building an End-to-End Knowledge Graph at Netflix↗ netflixtechblog.com · HN · 32 comments · 131 points · Oct 19, 2026
Can you use autoregressive diffusion to generate market data?↗ blog.janestreet.com · HN · 57 comments · 186 points · Oct 09, 2026
Sep 2026129 stories
I've operated petabyte-scale ClickHouse clusters for 5 years↗ tinybird.co · HN · 90 comments · 246 points · Sep 07, 2026
Aug 2026159 stories
Fast drilldown dashboards from a single Parquet file↗ hamiltonulmer.com · HN · 23 comments · 173 points · Aug 24, 2026
What went wrong with data lakes? A 15-year reality check↗ arxiv.org · HN · 16 comments · 54 points · Aug 05, 2026
Jul 2026168 stories
Algorithms on billion-scale graph using 10GB RAM: I love DataFusion↗ semyonsinchenko.github.io · HN · 40 comments · 137 points · Jul 31, 2026
Benchmarking coding agents on Databricks' multi-million line codebase↗ databricks.com · HN · 69 comments · 161 points · Jul 08, 2026
Show HN: Microsoft releases Flint, a visualization language for AI agents↗ microsoft.github.io · HN · 137 comments · 350 points · Jul 08, 2026
Jun 2026187 stories
PgDog is funded and coming to a database near you↗ pgdog.dev · HN · 261 comments · 548 points · Jun 10, 2026
May 2026110 stories
High dimensional geometry is transforming the MRI industry (2017) [pdf]↗ ams.org · HN · 37 comments · 111 points · May 15, 2026
Show HN: Mljar Studio – local AI data analyst that saves analysis as notebooks↗ mljar.com · HN · 18 comments · 73 points · May 02, 2026
Apr 2026159 stories
Atlassian enables default data collection to train AI↗ letsdatascience.com · HN · 136 comments · 604 points · Apr 20, 2026
A better R programming experience thanks to Tree-sitter↗ ropensci.org · HN · 33 comments · 164 points · Apr 16, 2026
Mar 2026179 stories
What category theory teaches us about dataframes↗ mchav.github.io · HN · 65 comments · 190 points · Mar 29, 2026
Pandas Exercises for Data Analysis (Interactive)↗ machinelearningplus.com · HN · 33 comments · 126 points · Mar 13, 2026
Feb 2026169 stories
Hardwood: A New Parser for Apache Parquet↗ morling.dev · HN · 12 comments · 110 points · Feb 26, 2026
Show HN: Data Engineering Book – An open source, community-driven guide↗ github.com · HN · 30 comments · 251 points · Feb 13, 2026
Jan 2026177 stories
ICE using Palantir tool that feeds on Medicaid data↗ eff.org · HN · 969 comments · 1475 points · Jan 25, 2026
Ask HN: How are you doing RAG locally?↗ news.ycombinator.com · HN · 156 comments · 413 points · Jan 14, 2026
Dec 2025165 stories
Show HN: Marmot – Single-binary data catalog (no Kafka, no Elasticsearch)↗ github.com · HN · 21 comments · 103 points · Dec 02, 2025
Nov 2025182 stories
650GB of Data (Delta Lake on S3). Polars vs. DuckDB vs. Daft vs. Spark↗ dataengineeringcentral.substack.com · HN · 107 comments · 263 points · Nov 13, 2025
Launch HN: Hypercubic (YC F25) – AI for COBOL and Mainframes↗ news.ycombinator.com · HN · 58 comments · 91 points · Nov 10, 2025
Oct 2025178 stories
Production RAG: what I learned from processing 5M+ documents↗ blog.abdellatif.io · HN · 114 comments · 551 points · Oct 20, 2025
F3: Open-source data file format for the future [pdf]↗ db.cs.cmu.edu · HN · 125 comments · 386 points · Oct 01, 2025
Sep 2025150 stories
Bayesian Data Analysis, Third edition (2013) [pdf]↗ sites.stat.columbia.edu · HN · 68 comments · 347 points · Sep 28, 2025
How Palantir is mapping the nation’s data↗ theconversation.com · HN · 86 comments · 230 points · Sep 11, 2025
Will Amazon S3 Vectors kill vector databases or save them?↗ zilliz.com · HN · 122 comments · 280 points · Sep 08, 2025
Aug 2025119 stories
Materialized views are obviously useful↗ sophiebits.com · HN · 78 comments · 173 points · Aug 23, 2025
Why was Apache Kafka created?↗ bigdata.2minutestreaming.com · HN · 221 comments · 201 points · Aug 22, 2025
Jul 2025155 stories
A major AI training data set contains millions of examples of personal data↗ technologyreview.com · HN · 118 comments · 167 points · Jul 30, 2025
Positron – A next-generation data science IDE↗ positron.posit.co · HN · 92 comments · 220 points · Jul 21, 2025
Big Data was used to see if TCM was scientific (2023)↗ mcgill.ca · HN · 124 comments · 122 points · Jul 13, 2025
Jun 2025173 stories
Datadog's $65M/year customer mystery solved↗ blog.pragmaticengineer.com · HN · 62 comments · 152 points · Jun 30, 2025
May 2025170 stories
DuckLake is an integrated data lake and catalog format↗ ducklake.select · HN · 107 comments · 276 points · May 27, 2025
A lost decade chasing distributed architectures for data analytics?↗ duckdb.org · HN · 113 comments · 215 points · May 19, 2025
Apr 2025130 stories
Palantir's Justification for Building ICE's Master Database↗ 404media.co · HN · 17 comments · 76 points · Apr 29, 2025
Kaggle and the Wikimedia Foundation are partnering on open data↗ blog.google · HN · 38 comments · 136 points · Apr 16, 2025
DEDA – Tracking Dots Extraction, Decoding and Anonymisation Toolkit↗ github.com · HN · 99 comments · 286 points · Apr 01, 2025
Mar 2025168 stories
Euclid opens data treasure trove, offers glimpse of deep fields↗ esa.int · HN · 21 comments · 107 points · Mar 20, 2025
Apache iceberg the Hadoop of the modern-data-stack?↗ blog.det.life · HN · 65 comments · 115 points · Mar 06, 2025
Feb 2025153 stories
Announcing the data.gov archive↗ lil.law.harvard.edu · HN · 132 comments · 382 points · Feb 07, 2025
Jan 2025164 stories
Archivists work to save disappearing data.gov datasets↗ 404media.co · HN · 238 comments · 801 points · Jan 30, 2025
B-Trees: More Than I Thought I'd Want to Know (2021)↗ benjamincongdon.me · HN · 16 comments · 319 points · Jan 03, 2025
Dec 2024137 stories
Surfer: Open-Source Personal Data Warehouse↗ github.com · HN · 21 comments · 147 points · Dec 14, 2024
1,600 days of a failed hobby data science project↗ lellep.xyz · HN · 70 comments · 158 points · Dec 08, 2024
Nov 2024151 stories
Autoflow, a Graph RAG based and conversational knowledge base tool↗ github.com · HN · 36 comments · 280 points · Nov 22, 2024
Oct 2024202 stories
Drasi: Microsoft's open source data processing platform for event-driven systems↗ github.com · HN · 67 comments · 331 points · Oct 20, 2024
Sep 2024160 stories
Insights after 11 years with Datomic [video]↗ youtube.com · HN · 106 comments · 205 points · Sep 25, 2024
Show HN: Sourcetable – AI Spreadsheet and Data Platform↗ news.ycombinator.com · HN · 88 comments · 188 points · Sep 19, 2024
Data Engineering Vault: A 1000 Node Second Brain for DE Knowledge↗ ssp.sh · HN · 19 comments · 123 points · Sep 15, 2024
Aug 2024189 stories
Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data↗ news.ycombinator.com · HN · 116 comments · 234 points · Aug 13, 2024
Jul 2024167 stories
Call of Duty: Warzone Caldera Data Set for Academic Use↗ blog.activision.com · HN · 95 comments · 178 points · Jul 31, 2024
A multimodal dataset with one trillion tokens↗ github.com · HN · 52 comments · 224 points · Jul 24, 2024
Jun 2024169 stories
Lessons Learned from Scaling to Multi-Terabyte Datasets↗ v2thegreat.com · HN · 45 comments · 149 points · Jun 20, 2024
Koheesio: Nike's Python-based framework to build advanced data-pipelines↗ github.com · HN · 73 comments · 217 points · Jun 04, 2024
ROOT: analyzing petabytes of data scientifically↗ root.cern · HN · 93 comments · 356 points · Jun 01, 2024
May 2024179 stories
Show HN: Open-source BI and analytics for engineers↗ github.com · HN · 75 comments · 293 points · May 15, 2024
Apr 2024191 stories
So you want to scrape like the big boys (2021)↗ incolumitas.com · HN · 159 comments · 321 points · Apr 27, 2024
AI for Data Journalism: demonstrating what we can do with this stuff↗ simonwillison.net · HN · 32 comments · 167 points · Apr 22, 2024
Loading a trillion rows of weather data into TimescaleDB↗ aliramadhan.me · HN · 177 comments · 405 points · Apr 16, 2024
Mar 2024207 stories
How Figma's databases team lived to tell the scale↗ figma.com · HN · 223 comments · 568 points · Mar 14, 2024
The Pile is a 825 GiB diverse, open-source language modelling data set (2020)↗ pile.eleuther.ai · HN · 234 comments · 332 points · Mar 07, 2024
Feb 2024189 stories
Blazer: Business intelligence made simple↗ github.com · HN · 37 comments · 215 points · Feb 27, 2024
Data Structures for Data-Intensive Applications [pdf] (2023)↗ cs-people.bu.edu · HN · 26 comments · 583 points · Feb 09, 2024
SQL for data scientists in 100 queries↗ gvwilson.github.io · HN · 158 comments · 834 points · Feb 06, 2024
Jan 2024182 stories
Good DevEx increases productivity. Here is the data↗ github.blog · HN · 128 comments · 243 points · Jan 24, 2024
Dec 2023142 stories
Understanding Parquet, Iceberg and Data Lakehouses↗ davidgomes.com · HN · 101 comments · 312 points · Dec 29, 2023
Natural Language Corpus Data: Beautiful Data (2009)↗ norvig.com · HN · 3 comments · 53 points · Dec 29, 2023
Data scientists need to learn about significant digits↗ lemire.me · HN · 65 comments · 68 points · Dec 25, 2023
Nov 2023162 stories
Show HN: Taipy – Turns Data and AI algorithms into full web applications↗ github.com · HN · 2 comments · 22 points · Nov 30, 2023
Show HN: I've Published 28B Molecule Embeddings on AWS Open Data↗ ashvardanian.com · HN · 0 comments · 10 points · Nov 21, 2023
ELTP: Extending ELT for Modern AI and Analytics↗ airbyte.com · HN · 0 comments · 16 points · Nov 07, 2023
Oct 2023170 stories
Five Design Principles for Building a Data Pipeline Engine↗ docs.tenzir.com · HN · 2 comments · 16 points · Oct 17, 2023
Causal inference as a blind spot of data scientists↗ dzidas.com · HN · 1 comments · 17 points · Oct 15, 2023
Sep 2023189 stories
Ask HN: How do your ML teams version datasets and models?↗ news.ycombinator.com · HN · 4 comments · 9 points · Sep 28, 2023
Show HN: PipeRider – open-source Data Impact Analysis for dbt changes↗ github.com · HN · 2 comments · 11 points · Sep 06, 2023
Show HN: Specfy – Stack Intelligence Platform↗ github.com · HN · 0 comments · 16 points · Sep 05, 2023
Aug 2023165 stories
The uses and abuses of Cloud Data Warehouses↗ materialize.com · HN · 1 comments · 9 points · Aug 16, 2023
Show HN: Netdata released v1.42 and it includes 800 integrations↗ github.com · HN · 1 comments · 7 points · Aug 10, 2023
Show HN: Instant Datasets: Production data for development↗ news.ycombinator.com · HN · 0 comments · 10 points · Aug 09, 2023
Jul 2023174 stories
Whom the Gods Would Destroy, They First Give Real-Time Analytics↗ mcfunley.com · HN · 3 comments · 17 points · Jul 25, 2023
Snakemake – A framework for reproducible data analysis↗ snakemake.github.io · HN · 3 comments · 26 points · Jul 15, 2023
The Barbie-Oppenheimer Double Feature Is Happening, Data Shows↗ bnnbloomberg.ca · HN · 6 comments · 16 points · Jul 12, 2023
Jun 2023177 stories
English as the new programming language for Apache spark↗ databricks.com · HN · 4 comments · 13 points · Jun 29, 2023
Show HN: Download HuggingFace Models/Datasets easily and super fast↗ github.com · HN · 1 comments · 9 points · Jun 24, 2023
May 2023203 stories
Apr 2023168 stories
Scaling Databases at Activision [pdf]↗ static.sched.com · HN · 0 comments · 15 points · Apr 21, 2023
A Tale of Three Real-Time OLAP Databases: Apache Pinot, Apache Druid, ClickHouse↗ startree.ai · HN · 2 comments · 40 points · Apr 20, 2023
Show HN: Datree (YC W20) – End-to-End Policy Management for Kubernetes↗ news.ycombinator.com · HN · 3 comments · 24 points · Apr 04, 2023
Mar 2023210 stories
Analyzing multi-gigabyte JSON files locally↗ thenybble.de · HN · 13 comments · 19 points · Mar 18, 2023
Show HN: Ingest data from your customers (Prequel YC W21)↗ news.ycombinator.com · HN · 0 comments · 15 points · Mar 15, 2023
Feb 2023174 stories
Most Data Work Seems Fundamentally Worthless↗ ludic.mataroa.blog · HN · 20 comments · 57 points · Feb 27, 2023
Replacing a SQL Analyst with Recursive GPT Prompts – Part II, Matplotlib↗ patterns.app · HN · 1 comments · 9 points · Feb 09, 2023
Jan 2023189 stories
DataDog asked OpenTelemetry contributor to kill pull request↗ github.com · HN · 29 comments · 143 points · Jan 26, 2023
Perform What-If Analysis of Your Network Without Compromising Data Integrity↗ memgraph.com · HN · 1 comments · 11 points · Jan 13, 2023
UC Berkeley researchers set new visions for making CRDT data stores practical [pdf]↗ vldb.org · HN · 1 comments · 11 points · Jan 10, 2023
Dec 2022157 stories
Ask HN: Is the industry saturated with data scientists already?↗ news.ycombinator.com · HN · 1 comments · 13 points · Dec 22, 2022
Nov 2022197 stories
Launch HN: Activeloop (YC S18) – Data lake for deep learning↗ news.ycombinator.com · HN · 24 comments · 64 points · Nov 15, 2022
Phlare: open-source database for continuous profiling at scale↗ grafana.com · HN · 40 comments · 240 points · Nov 02, 2022
Oct 2022165 stories
Launch HN: MovingLake (YC S22) – Real-time data connectors for almost anything↗ app.movinglake.com · HN · 65 comments · 132 points · Oct 27, 2022
The evolution of the data engineer role↗ airbyte.com · HN · 117 comments · 167 points · Oct 24, 2022
Sep 2022170 stories
Launch HN: Prequel (YC W21) – Sync data to your customer’s data warehouse↗ news.ycombinator.com · HN · 40 comments · 118 points · Sep 26, 2022
Show HN: Open Prompts – dataset of 10M Stable Diffusion generations↗ github.com · HN · 71 comments · 279 points · Sep 22, 2022
Aug 2022209 stories
Physically Based: A Database of PBR Values for Real-World Materials↗ 80.lv · HN · 46 comments · 121 points · Aug 17, 2022
Launch HN: Sematic (YC S22) – Open-source framework to build ML pipelines faster↗ news.ycombinator.com · HN · 47 comments · 121 points · Aug 10, 2022
Jul 2022163 stories
CS109a: Introduction to Data Science – Resources↗ harvard-iacs.github.io · HN · 25 comments · 223 points · Jul 31, 2022
Interactive course about “everyday” data science↗ tigyog.app · HN · 52 comments · 153 points · Jul 16, 2022
Show HN: A benchmark for analytical databases (Snowflake, Druid, Redshift)↗ benchmark.clickhouse.com · HN · 58 comments · 137 points · Jul 13, 2022
Jun 2022194 stories
Show HN: Featureform – An open-source Feature Store for ML↗ github.com · HN · 27 comments · 136 points · Jun 30, 2022
Show HN: Data Diff – compare tables of any size across databases↗ news.ycombinator.com · HN · 21 comments · 127 points · Jun 22, 2022
A Data-Centric Introduction to Computing↗ dcic-world.org · HN · 15 comments · 106 points · Jun 20, 2022
May 2022178 stories
Data Science in Context – Peter Norvig's New Book↗ datascienceincontext.com · HN · 40 comments · 308 points · May 24, 2022
Lessons learned from running Apache Airflow at scale↗ shopify.engineering · HN · 182 comments · 285 points · May 23, 2022
Apr 2022186 stories
Explorer – A library to bring dataframes to Elixir↗ cigrainger.com · HN · 24 comments · 124 points · Apr 28, 2022
Meta quickly detects silent data corruptions at scale↗ engineering.fb.com · HN · 95 comments · 167 points · Apr 04, 2022
Mar 2022184 stories
Shopify's Data Science and Engineering Foundations (2020)↗ shopify.engineering · HN · 25 comments · 205 points · Mar 11, 2022
Phases of Netflix’s real-time data infrastructure↗ zhenzhongxu.com · HN · 47 comments · 164 points · Mar 10, 2022
Feb 2022164 stories
PlanetScale increases plans to include billions of reads↗ planetscale.com · HN · 68 comments · 136 points · Feb 24, 2022
Become a better data engineer on a shoestring (free resources)↗ dataengineering.academy · HN · 25 comments · 161 points · Feb 24, 2022
Jan 2022185 stories
Show HN: Electric Tables – an experiment in personal databases↗ tomcritchlow.com · HN · 37 comments · 231 points · Jan 26, 2022
Introduction to Probability for Data Science↗ probability4datascience.com · HN · 15 comments · 235 points · Jan 24, 2022
Dec 2021174 stories
The Science of Visual Data Communication: What Works↗ journals.sagepub.com · HN · 11 comments · 117 points · Dec 30, 2021
Big data may not know your name, but it knows everything else↗ wired.com · HN · 69 comments · 190 points · Dec 30, 2021
Nov 2021177 stories
Databricks response to Snowflake's accusation of lacking integrity↗ databricks.com · HN · 156 comments · 217 points · Nov 15, 2021
Spending $5k to learn how database indexes work↗ briananglin.me · HN · 199 comments · 236 points · Nov 06, 2021
Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative↗ news.ycombinator.com · HN · 110 comments · 265 points · Nov 04, 2021
Oct 2021191 stories
Tao: Facebook’s distributed data store for the social graph↗ micahlerner.com · HN · 39 comments · 170 points · Oct 30, 2021
We need new data books, so we published one↗ theinformedcompany.com · HN · 31 comments · 171 points · Oct 26, 2021
How Time Series Databases Work, and Where They Don’t↗ honeycomb.io · HN · 55 comments · 228 points · Oct 18, 2021
Sep 2021156 stories
Why I Use Nim instead of Python for Data Processing↗ benjamindlee.com · HN · 178 comments · 256 points · Sep 23, 2021
Is BI dead? – On dismantling data's ship of Theseus↗ benn.substack.com · HN · 116 comments · 174 points · Sep 18, 2021
Aug 2021145 stories
Launch HN: Evidence (YC S21) – Web framework for data analysts↗ news.ycombinator.com · HN · 91 comments · 263 points · Aug 25, 2021
B-Trees: More Than I Thought I'd Want to Know↗ benjamincongdon.me · HN · 63 comments · 239 points · Aug 18, 2021
Classical data structures that can outperform learned indexes (2018)↗ dawn.cs.stanford.edu · HN · 39 comments · 252 points · Aug 16, 2021
Jul 2021178 stories
If you want to transform IT, start with finance↗ zwischenzugs.com · HN · 150 comments · 293 points · Jul 13, 2021
Building a data team at a mid-stage startup↗ erikbern.com · HN · 89 comments · 607 points · Jul 08, 2021
The Rise and Fall of the OLAP Cube (2020)↗ holistics.io · HN · 91 comments · 211 points · Jul 05, 2021
Jun 2021174 stories
Show HN: A flow based data processing editor↗ datablocks.pro · HN · 20 comments · 188 points · Jun 10, 2021
Perl Data Language: Scientific Computing with Perl↗ pdl.perl.org · HN · 108 comments · 144 points · Jun 08, 2021
May 2021194 stories
Good data scientist, bad data scientist↗ ianwhitestone.work · HN · 91 comments · 159 points · May 11, 2021
Game theory as an engine for large-scale data analysis↗ deepmind.com · HN · 14 comments · 203 points · May 10, 2021
Apr 2021178 stories
Show HN: Personal Data Management Dashboard↗ volmarg.github.io · HN · 53 comments · 176 points · Apr 12, 2021
Mar 2021205 stories
Everyone wants to do the model work, not the data work [pdf]↗ storage.googleapis.com · HN · 112 comments · 358 points · Mar 29, 2021
Data Science in Julia for Hackers↗ datasciencejuliahackers.com · HN · 51 comments · 240 points · Mar 19, 2021
Feb 2021203 stories
Show HN: I wrote a book about using data science to solve “everyday” problems↗ andrewnc.github.io · HN · 96 comments · 632 points · Feb 24, 2021
Solid Project: All of your data, under your control↗ solidproject.org · HN · 118 comments · 350 points · Feb 01, 2021
Jan 2021201 stories
Why isn't differential dataflow more popular?↗ scattered-thoughts.net · HN · 120 comments · 228 points · Jan 22, 2021
We don't need data scientists, we need data engineers↗ mihaileric.com · HN · 353 comments · 718 points · Jan 14, 2021
How to Become a Data Engineer in 2021↗ khashtamov.com · HN · 137 comments · 264 points · Jan 11, 2021
Dec 2020247 stories
The Elements of Statistical Learning [pdf]↗ web.stanford.edu · HN · 59 comments · 282 points · Dec 31, 2020
Datasette: An open source multi-tool for exploring and publishing data↗ datasette.io · HN · 22 comments · 285 points · Dec 11, 2020
Kaitai: Describe the structure of data, not how you read or write it↗ kaitai.io · HN · 65 comments · 245 points · Dec 05, 2020
Nov 2020241 stories
Ask HN: Am I too late for the “Data Science” wave?↗ news.ycombinator.com · HN · 143 comments · 199 points · Nov 15, 2020
Personal Data Warehouses: Reclaiming Your Data↗ simonwillison.net · HN · 59 comments · 436 points · Nov 14, 2020
AWS releases Glue Databrew, a visual ETL tool↗ aws.amazon.com · HN · 70 comments · 169 points · Nov 12, 2020
Oct 2020252 stories
Launch HN: Deepnote (YC S19) – A better data science notebook↗ news.ycombinator.com · HN · 70 comments · 316 points · Oct 30, 2020
Emerging Architectures for Modern Data Infrastructure↗ a16z.com · HN · 105 comments · 431 points · Oct 18, 2020
Ask HN: What are good resources to learn system design?↗ news.ycombinator.com · HN · 65 comments · 246 points · Oct 13, 2020
Sep 2020253 stories
AresDB: Uber's GPU-Powered Open Source, Real-Time Analytics Engine↗ ubere.ng · HN · 83 comments · 191 points · Sep 13, 2020
Data science interview questions with answers↗ github.com · HN · 71 comments · 242 points · Sep 13, 2020
Aug 2020230 stories
The Splitgraph Data Delivery Network – query over 40k public datasets↗ splitgraph.com · HN · 95 comments · 297 points · Aug 21, 2020
Launch HN: Datafold (YC S20) – Diff Tool for SQL Databases↗ news.ycombinator.com · HN · 76 comments · 189 points · Aug 06, 2020
Simdjson: Parsing gigabytes of JSON per second↗ github.com · HN · 151 comments · 503 points · Aug 06, 2020
Jul 2020259 stories
You Want to See My Data? I Thought We Were Friends↗ nautil.us · HN · 134 comments · 489 points · Jul 30, 2020
Best Data Science Books According to the Experts↗ builtin.com · HN · 60 comments · 284 points · Jul 24, 2020
Data structures and algorithms I actually used while working at tech companies↗ blog.pragmaticengineer.com · HN · 524 comments · 1025 points · Jul 15, 2020
Jun 2020248 stories
Hash: A free, online platform for modeling↗ joelonsoftware.com · HN · 59 comments · 208 points · Jun 18, 2020
GoPlus – The Go+ language for data science↗ github.com · HN · 61 comments · 143 points · Jun 17, 2020
May 2020259 stories
Bayesian Data Analysis, Third Edition [pdf]↗ users.aalto.fi · HN · 48 comments · 466 points · May 06, 2020
Apr 2020295 stories
Data Science: Reality Doesn't Meet Expectations↗ dfrieds.com · HN · 163 comments · 462 points · Apr 07, 2020
Mar 2020289 stories
OpenRefine – free, open source, powerful tool for working with messy data↗ openrefine.org · HN · 23 comments · 235 points · Mar 22, 2020
Launch HN: Datasaur (YC W20) – data labeling interface for NLP↗ news.ycombinator.com · HN · 62 comments · 174 points · Mar 06, 2020
Aleph: A suite of data analysis tools for investigators↗ docs.alephdata.org · HN · 21 comments · 240 points · Mar 01, 2020
Feb 2020269 stories
Materialize: A Streaming Data Warehouse↗ materialize.io · HN · 100 comments · 262 points · Feb 18, 2020
Jan 2020279 stories
Discovering Millions of Datasets on the Web↗ blog.google · HN · 32 comments · 443 points · Jan 23, 2020
Copying data is wasteful, mutating data is dangerous↗ pythonspeed.com · HN · 153 comments · 230 points · Jan 08, 2020
Nov 16, 202637 stories
Monetize your company's operational data↗ mainstreetwealth.ai · HN · 0 comments · 5 points · Nov 16, 2026
Nov 09, 202640 stories
SP4N – Helping construction teams understand data and make better decisions↗ sp4n.ai · HN · 0 comments · 5 points · Nov 15, 2026
The Dawn of the Data Intelligence Engineer↗ motherduck.com · HN · 78 comments · 160 points · Nov 13, 2026
Nov 02, 202638 stories
USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance↗ arxiv.org · HN · 8 comments · 49 points · Nov 03, 2026
The U.S. Census Bureau and "Super Intelligence"↗ simonwillison.net · HN · 54 comments · 127 points · Nov 02, 2026
Oct 26, 202641 stories
From Spreadsheets to DuckLake: A Data Engineer's Road to DuckDB↗ duckdb.org · HN · 0 comments · 22 points · Oct 29, 2026
Keplearn: Extract Equations from Data in Seconds↗ github.com · HN · 19 comments · 108 points · Oct 29, 2026
MiniLake Is a Local Databricks API Emulator with SQL, Delta Lake and Spark↗ dmux.github.io · HN · 25 comments · 101 points · Oct 27, 2026
Oct 19, 202639 stories
Ontology-Driven Observability: Building an End-to-End Knowledge Graph at Netflix↗ netflixtechblog.com · HN · 32 comments · 131 points · Oct 19, 2026
Oct 12, 202628 stories
Show HN: Chkit – Schema, migrations and backfills for ClickHouse (like Drizzle)↗ github.com · HN · 0 comments · 9 points · Oct 12, 2026
Oct 05, 202639 stories
High-quality data is the best compute multiplier↗ datologyai.com · HN · 2 comments · 10 points · Oct 09, 2026
Can you use autoregressive diffusion to generate market data?↗ blog.janestreet.com · HN · 57 comments · 186 points · Oct 09, 2026
Show HN: Pivot – fast analytics on Iceberg, 2x+ faster than ClickHouse / DuckDB↗ pivotlake.io · HN · 4 comments · 16 points · Oct 08, 2026
Sep 28, 202636 stories
Show HN: Dataviz, ranked daily from GitHub, NPM, PyPI and CRAN↗ awesomedataviz.com · HN · 1 comments · 11 points · Oct 04, 2026
DEDA – Tracking Dots Extraction, Decoding and Anonymisation Toolkit↗ github.com · HN · 12 comments · 91 points · Oct 04, 2026
From Embedded Dashboards to Programmable Analytics↗ ag-grid.com · HN · 3 comments · 8 points · Oct 01, 2026
Sep 21, 202630 stories
Show HN: Ppgrid – Rasterising points in minutes, not hours↗ mrzk.io · HN · 0 comments · 3 points · Sep 27, 2026
How Man City overhauled their squad from a 650k-strong database↗ bbc.com · HN · 0 comments · 4 points · Sep 24, 2026
Sep 14, 202638 stories
Common Crawl Data Stored on a Hugging Face Bucket↗ commoncrawl.org · HN · 1 comments · 23 points · Sep 16, 2026
Show HN: Direct SQL Access to Your QuickBooks Data↗ instabooks.io · HN · 2 comments · 5 points · Sep 14, 2026
Sep 07, 202626 stories
Show HN: Dbmask – Discover, mask, and verify sensitive data in SQL databases↗ github.com · HN · 0 comments · 5 points · Sep 10, 2026
The Grammar of Data: From Definition to Execution↗ xorq.dev · HN · 0 comments · 5 points · Sep 09, 2026
I've operated petabyte-scale ClickHouse clusters for 5 years↗ tinybird.co · HN · 90 comments · 246 points · Sep 07, 2026
Aug 31, 202632 stories
Launch HN: Hebbian Robotics (YC S26) – Build scalable robotics data pipelines↗ github.com · HN · 12 comments · 46 points · Aug 31, 2026
Aug 24, 202644 stories
How An Atlanta Suburb Shares Flock Data with More Than 2k Organizations↗ wired.com · HN · 1 comments · 13 points · Aug 28, 2026
Fast drilldown dashboards from a single Parquet file↗ hamiltonulmer.com · HN · 23 comments · 173 points · Aug 24, 2026
Aug 17, 202630 stories
Show HN: A sample dataset of computer-use tasks on professional software↗ cua.graderlabs.com · HN · 3 comments · 16 points · Aug 18, 2026
Can Haskell Become a Great Language for Data Science?↗ youtube.com · HN · 0 comments · 5 points · Aug 18, 2026
Data Loading for AI/ML: A Comprehensive Guide↗ lancedb.com · HN · 0 comments · 7 points · Aug 17, 2026
Aug 10, 202634 stories
ICE to Pay LexisNexis Millions for Data to Feed to Palantir↗ 404media.co · HN · 5 comments · 37 points · Aug 11, 2026
Hetzner Experiments Platform: Inference API↗ experiments.hetzner.com · HN · 8 comments · 17 points · Aug 10, 2026
Aug 03, 202633 stories
What went wrong with data lakes? A 15-year reality check↗ arxiv.org · HN · 16 comments · 54 points · Aug 05, 2026
Dataset: Dead mental health startups, 2000-2026, coded on 18 fields↗ mentalium.me · HN · 24 comments · 31 points · Aug 04, 2026
Bending Spoons buying Airtable for $2.2B↗ techcrunch.com · HN · 4 comments · 18 points · Aug 04, 2026
Jul 27, 202635 stories
Indexing the Data Lake for Online Point Queries↗ engineering.atspotify.com · HN · 3 comments · 19 points · Aug 01, 2026
Algorithms on billion-scale graph using 10GB RAM: I love DataFusion↗ semyonsinchenko.github.io · HN · 40 comments · 137 points · Jul 31, 2026
Show HN: Lean4 Datalog DSL Based on Google Zanzibar for AI Projects↗ github.com · HN · 2 comments · 18 points · Jul 29, 2026
Jul 20, 202636 stories
Show HN: Fitter – declarative web-data extraction,running in the browser also↗ pxyup.github.io · HN · 3 comments · 5 points · Jul 26, 2026
Splink – Fast, accurate and scalable probabilistic data linkage↗ moj-analytical-services.github.io · HN · 0 comments · 10 points · Jul 24, 2026
Apache Spark 4.2: Making Your Data AI‑Developer Friendly↗ techstrong.it · HN · 0 comments · 7 points · Jul 21, 2026
Jul 13, 202638 stories
Guide to data tools landscape for developers↗ sinja.io · HN · 36 comments · 135 points · Jul 16, 2026
Show HN: I implemented a neural network in SQL↗ github.com · HN · 20 comments · 121 points · Jul 13, 2026
Jul 06, 202640 stories
A Beautiful Theory Falls to Ugly Data↗ marginalrevolution.com · HN · 69 comments · 81 points · Jul 12, 2026
Benchmarking coding agents on Databricks' multi-million line codebase↗ databricks.com · HN · 69 comments · 161 points · Jul 08, 2026
Show HN: Microsoft releases Flint, a visualization language for AI agents↗ microsoft.github.io · HN · 137 comments · 350 points · Jul 08, 2026
Jun 29, 202640 stories
Giving a domain a hill to climb: benchmarking as data activation↗ sparsethought.com · HN · 7 comments · 12 points · Jul 03, 2026
Launch HN: Parsewise (YC P25) – Reason Across Documents with an API↗ news.ycombinator.com · HN · 54 comments · 56 points · Jul 01, 2026
Jun 22, 202637 stories
Show HN: The TypeScript Semantic Layer for ClickHouse↗ github.com · HN · 6 comments · 8 points · Jun 27, 2026
Fixing Failures in Browser-Use Models: Why More Data Isn't Enough↗ fig.inc · HN · 1 comments · 17 points · Jun 26, 2026
Too many R packages: CRAN is inundated with submissions↗ rworks.dev · HN · 83 comments · 98 points · Jun 24, 2026
Jun 15, 202648 stories
Robotics Teams Are Rebuilding the Data Stack from Scratch↗ rerun.io · HN · 1 comments · 34 points · Jun 21, 2026
Databricks Launches LTAP: A Unified OLAP/OLTP Data Architecture↗ databricks.com · HN · 14 comments · 38 points · Jun 16, 2026
Jun 08, 202652 stories
Please Switch to Python. Or R. Or Anything. Just Not Stata, SAS, SPSS, or Matlab↗ presentofcoding.substack.com · HN · 5 comments · 14 points · Jun 12, 2026
PgDog is funded and coming to a database near you↗ pgdog.dev · HN · 261 comments · 548 points · Jun 10, 2026
Show HN: Artie – Real-time data replication to your warehouse, now self-serve↗ artie.com · HN · 6 comments · 26 points · Jun 10, 2026
Jun 01, 202638 stories
Neo4j Acquires GraphAware for an Intelligence Analysis Alternative to Palantir↗ hpcwire.com · HN · 0 comments · 6 points · Jun 05, 2026
Show HN: Data Leak Prevention inside the n8n execution graph.↗ npmjs.com · HN · 1 comments · 18 points · Jun 01, 2026
May 25, 202625 stories
Building a LangGraph pipeline for production data engineering↗ labyrinthanalyticsconsulting.com · HN · 0 comments · 13 points · May 31, 2026
May 18, 202624 stories
Execs Are Deploying Digital Twins to Do Their Work↗ wsj.com · HN · 1 comments · 6 points · May 23, 2026
Codex for Everything Exfiltrates Connected Data↗ promptarmor.com · HN · 0 comments · 4 points · May 21, 2026
May 11, 202625 stories
High dimensional geometry is transforming the MRI industry (2017) [pdf]↗ ams.org · HN · 37 comments · 111 points · May 15, 2026
Datadog scales time series foundation models to 2.5B parameters↗ huggingface.co · HN · 4 comments · 6 points · May 14, 2026
TabPFN-3: scaling foundation models for tabular data to 1M rows↗ priorlabs.ai · HN · 0 comments · 9 points · May 12, 2026
May 04, 202631 stories
Show HN: Transform Claude/Codex/Cursor sessions into high-value data assets↗ github.com · HN · 4 comments · 7 points · May 07, 2026
Apr 27, 202625 stories
Show HN: Mljar Studio – local AI data analyst that saves analysis as notebooks↗ mljar.com · HN · 18 comments · 73 points · May 02, 2026
OpenData Buffer: HA pipelines without Kafka↗ opendata.dev · HN · 1 comments · 13 points · Apr 30, 2026
Apr 20, 202641 stories
Atlassian enables default data collection to train AI↗ letsdatascience.com · HN · 136 comments · 604 points · Apr 20, 2026
Apr 13, 202638 stories
Atlassian is changing how we use customer data on August 17, 2026↗ atlassian.com · HN · 5 comments · 17 points · Apr 17, 2026
A better R programming experience thanks to Tree-sitter↗ ropensci.org · HN · 33 comments · 164 points · Apr 16, 2026
Apr 06, 202638 stories
What game engines know about data that databases forgot↗ nockawa.github.io · HN · 62 comments · 86 points · Apr 09, 2026
Show HN: I built a local data lake for AI powered data engineering and analytics↗ stream-sock-3f5.notion.site · HN · 10 comments · 14 points · Apr 08, 2026
Show HN: 500k+ events/sec transformations for ClickHouse ingestion↗ github.com · HN · 4 comments · 13 points · Apr 08, 2026
Mar 30, 202635 stories
The Hardest Document Extraction Problem in Insurance↗ furtherai.com · HN · 5 comments · 36 points · Apr 03, 2026
Show HN: Dux, distributed DuckDB-backed dataframes on the Beam↗ github.com · HN · 0 comments · 7 points · Mar 31, 2026
Mar 23, 202642 stories
What category theory teaches us about dataframes↗ mchav.github.io · HN · 65 comments · 190 points · Mar 29, 2026
Mar 16, 202638 stories
I built a free interactive platform to learn KDB/q↗ kdb-academy.web.app · HN · 2 comments · 14 points · Mar 22, 2026
Context Use – turn your data exports into portable AI memories↗ github.com · HN · 3 comments · 7 points · Mar 22, 2026
Show HN: Altimate Code – Open-Source Agentic Data Engineering Harness↗ github.com · HN · 2 comments · 20 points · Mar 19, 2026
Mar 09, 202649 stories
Pandas Exercises for Data Analysis (Interactive)↗ machinelearningplus.com · HN · 33 comments · 126 points · Mar 13, 2026
Where did you think the training data was coming from?↗ idiallo.com · HN · 14 comments · 50 points · Mar 11, 2026
Show HN: XLA-based array computing framework for R↗ github.com · HN · 1 comments · 15 points · Mar 09, 2026
Mar 02, 202633 stories
Show HN: Max – a federated data query layer for AI agents (and humans)↗ max.cloud · HN · 1 comments · 7 points · Mar 06, 2026
Why Your BI Stack Knows More About Your Processes Than You Think↗ exasol.com · HN · 0 comments · 11 points · Mar 03, 2026
Feb 23, 202652 stories
Hardwood: A New Parser for Apache Parquet↗ morling.dev · HN · 12 comments · 110 points · Feb 26, 2026
SETI@home: Data Acquisition and Front-End Processing (2025)↗ iopscience.iop.org · HN · 23 comments · 93 points · Feb 23, 2026
Feb 16, 202637 stories
Ask HN: How to measure how much data one can effectively process or understand?↗ news.ycombinator.com · HN · 7 comments · 18 points · Feb 20, 2026
Show HN: Data Studio – Open-Source Data Notebooks↗ github.com · HN · 6 comments · 28 points · Feb 17, 2026
Feb 09, 202643 stories
Show HN: Data Engineering Book – An open source, community-driven guide↗ github.com · HN · 30 comments · 251 points · Feb 13, 2026
Launch HN: Livedocs (YC W22) – An AI-native notebook for data analysis↗ livedocs.com · HN · 19 comments · 48 points · Feb 10, 2026
Feb 02, 202638 stories
A tale of two flows: Metaflow and Kubeflow↗ blog.kubeflow.org · HN · 2 comments · 24 points · Feb 05, 2026
From Data Federation to AI-Ready Analytics with Virtual Schemas↗ exasol.com · HN · 0 comments · 14 points · Feb 03, 2026
Data Brokers Can Fuel Violence Against Public Servants↗ wired.com · HN · 65 comments · 93 points · Feb 03, 2026
Jan 26, 202636 stories
Show HN: We are building Git for data↗ news.ycombinator.com · HN · 2 comments · 9 points · Jan 27, 2026
Good if make prior after data instead of before↗ lesswrong.com · HN · 9 comments · 13 points · Jan 27, 2026
Jan 19, 202640 stories
ICE using Palantir tool that feeds on Medicaid data↗ eff.org · HN · 969 comments · 1475 points · Jan 25, 2026
Self-hosted AI data workflow: DB and Ollama and SQL↗ exasol.github.io · HN · 3 comments · 11 points · Jan 21, 2026
Jan 12, 202648 stories
Ask HN: How are you doing RAG locally?↗ news.ycombinator.com · HN · 156 comments · 413 points · Jan 14, 2026
Open sourcing Dicer: Databricks's auto-sharder↗ databricks.com · HN · 11 comments · 85 points · Jan 13, 2026
Jan 05, 202648 stories
Show HN: DDL to Data – Generate realistic test data from SQL schemas↗ news.ycombinator.com · HN · 32 comments · 55 points · Jan 06, 2026
Dec 29, 202523 stories
AI SRE needs better observability, not bigger models↗ clickhouse.com · HN · 2 comments · 10 points · Jan 01, 2026
Show HN: Running Analytics on 22 GB of Hacker News in SQLite↗ philippdubach.com · HN · 0 comments · 5 points · Dec 31, 2025
Dec 22, 202529 stories
Datafun – functional language that generalizes Datalog↗ rntz.net · HN · 0 comments · 12 points · Dec 24, 2025
Dec 15, 202547 stories
Show HN: Misata – synthetic data engine using LLM and Vectorized NumPy↗ github.com · HN · 2 comments · 24 points · Dec 16, 2025
All about Parameters and Widgets in Databricks Workflows (2024)↗ medium.com · HN · 0 comments · 10 points · Dec 16, 2025
Datasetq: jq for Datasets; Polars-powered Parquet/JSON/CSV query lang/cli↗ github.com · HN · 2 comments · 15 points · Dec 16, 2025
Dec 08, 202538 stories
Data Infrastructure for All: Free Kafka and $5 PostgreSQL↗ aiven.io · HN · 2 comments · 33 points · Dec 10, 2025
Sharding to Contain the Blast Radius of Data Breaches↗ mimirsec.com · HN · 9 comments · 22 points · Dec 08, 2025
Dec 01, 202536 stories
Introducing Proton Sheets: Protect the data that drives your business↗ proton.me · HN · 1 comments · 37 points · Dec 06, 2025
Show HN: Flooder – Making Persistent Homology Practical for Industrial Use Cases↗ plus-rkwitt.github.io · HN · 3 comments · 8 points · Dec 05, 2025
Show HN: Marmot – Single-binary data catalog (no Kafka, no Elasticsearch)↗ github.com · HN · 21 comments · 103 points · Dec 02, 2025
Nov 24, 202528 stories
I Built an EF Core Connector for Azure Data Explorer↗ anasismail.com · HN · 0 comments · 14 points · Nov 24, 2025
Nov 17, 202550 stories
Show HN: Datamorph – A clean JSON ⇄ CSV converter with auto-detect↗ datamorphio.vercel.app · HN · 9 comments · 26 points · Nov 21, 2025
Ask HN: How would you architect a RAG system for 10M+ documents today?↗ news.ycombinator.com · HN · 9 comments · 23 points · Nov 21, 2025
Nov 10, 202543 stories
650GB of Data (Delta Lake on S3). Polars vs. DuckDB vs. Daft vs. Spark↗ dataengineeringcentral.substack.com · HN · 107 comments · 263 points · Nov 13, 2025
Show HN: Data Formulator – interactive AI agents for data analysis (Microsoft)↗ data-formulator.ai · HN · 13 comments · 38 points · Nov 11, 2025
Launch HN: Hypercubic (YC F25) – AI for COBOL and Mainframes↗ news.ycombinator.com · HN · 58 comments · 91 points · Nov 10, 2025
Nov 03, 202557 stories
Disassembling terabytes of random data with Zig and Capstone to prove a point↗ jstrieb.github.io · HN · 17 comments · 45 points · Nov 08, 2025
I just trained a physics-based earthquake forecasting model on a $1000 GPU↗ news.ycombinator.com · HN · 7 comments · 15 points · Nov 04, 2025
The APM paradox: Too much data, too few answers↗ honeybadger.io · HN · 0 comments · 8 points · Nov 03, 2025
Oct 27, 202528 stories
DataGrip Is Now Free for Non-Commercial Use↗ blog.jetbrains.com · HN · 0 comments · 16 points · Oct 28, 2025
Is Your Data Valid? Why Bufstream Guarantees What Kafka Can't↗ vutr.substack.com · HN · 0 comments · 10 points · Oct 28, 2025
Oct 20, 202544 stories
You Probably Don't Need to Switch from Pandas to Polars↗ datamethods.substack.com · HN · 6 comments · 13 points · Oct 21, 2025
Show HN: I built TicketData – Ticket price trends and data for Sports & Concerts↗ ticketdata.com · HN · 5 comments · 7 points · Oct 21, 2025
Production RAG: what I learned from processing 5M+ documents↗ blog.abdellatif.io · HN · 114 comments · 551 points · Oct 20, 2025
Oct 13, 202544 stories
Show HN: Datapizza AI – Lightweight open source framework for GenAI apps↗ github.com · HN · 27 comments · 43 points · Oct 17, 2025
Oct 06, 202536 stories
A whirlwind introduction to dataflow graphs (2018)↗ fgiesen.wordpress.com · HN · 0 comments · 37 points · Oct 11, 2025
I switched from Htmx to Datastar↗ everydaysuperpowers.dev · HN · 380 comments · 315 points · Oct 10, 2025
Sep 29, 202538 stories
F3: Open-source data file format for the future [pdf]↗ db.cs.cmu.edu · HN · 125 comments · 386 points · Oct 01, 2025
Ask HN: What's your experience with using graph databases for agentic use-cases?↗ news.ycombinator.com · HN · 53 comments · 83 points · Oct 01, 2025
A Deep Research Agent for Curating Vision Datasets↗ arxiv.org · HN · 0 comments · 12 points · Sep 29, 2025
Sep 22, 202536 stories
Bayesian Data Analysis, Third edition (2013) [pdf]↗ sites.stat.columbia.edu · HN · 68 comments · 347 points · Sep 28, 2025
DeepFabric – Generate high-quality synthetic datasets at scale↗ lukehinds.github.io · HN · 19 comments · 106 points · Sep 26, 2025
Testing is better than data structures and algorithms↗ nedbatchelder.com · HN · 176 comments · 188 points · Sep 22, 2025
Sep 15, 202541 stories
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text↗ arxiv.org · HN · 17 comments · 68 points · Sep 19, 2025
Show HN: Semlib – Semantic Data Processing↗ github.com · HN · 12 comments · 63 points · Sep 15, 2025
Sep 08, 202534 stories
How Palantir is mapping the nation’s data↗ theconversation.com · HN · 86 comments · 230 points · Sep 11, 2025
Will Amazon S3 Vectors kill vector databases or save them?↗ zilliz.com · HN · 122 comments · 280 points · Sep 08, 2025
Sep 01, 202531 stories
Data modeling guide for real-time analytics with ClickHouse↗ ssp.sh · HN · 22 comments · 84 points · Sep 05, 2025
330+ data teams share what's working (and what's not)↗ metabase.com · HN · 6 comments · 27 points · Sep 04, 2025
Launch HN: Datafruit (YC S25) – AI for DevOps↗ news.ycombinator.com · HN · 48 comments · 65 points · Sep 02, 2025
Aug 25, 202541 stories
Data engineering and software engineering are converging↗ clickhouse.com · HN · 66 comments · 108 points · Aug 29, 2025
Bluesky now platform of choice for science community↗ arstechnica.com · HN · 72 comments · 88 points · Aug 27, 2025
TreeStore: Endowing data with hierarchical structure↗ blosc.org · HN · 0 comments · 15 points · Aug 27, 2025
Aug 18, 202527 stories
Materialized views are obviously useful↗ sophiebits.com · HN · 78 comments · 173 points · Aug 23, 2025
Why was Apache Kafka created?↗ bigdata.2minutestreaming.com · HN · 221 comments · 201 points · Aug 22, 2025
Aug 11, 202523 stories
How Keeta processes 11M financial transactions per second with Spanner↗ cloud.google.com · HN · 60 comments · 30 points · Aug 14, 2025
AI for data engineers with Simon Willison↗ talkingpostgres.com · HN · 0 comments · 21 points · Aug 11, 2025
Aug 04, 202524 stories
Spatio-temporal indexing the Bluesky firehose↗ joelgustafson.com · HN · 0 comments · 46 points · Aug 07, 2025
Show HN: Dataset Explorer – Free tool to search any public datasets↗ hunch.dev · HN · 2 comments · 7 points · Aug 05, 2025
Jul 28, 202526 stories
Stream Kafka Topic to the Iceberg Tables with Zero-ETL↗ vutr.substack.com · HN · 2 comments · 9 points · Jul 31, 2025
A major AI training data set contains millions of examples of personal data↗ technologyreview.com · HN · 118 comments · 167 points · Jul 30, 2025
Lance v2: A columnar container format for modern data (2024)↗ blog.lancedb.com · HN · 0 comments · 8 points · Jul 28, 2025
Jul 21, 202535 stories
Show HN: Slice and Dice – analyze and explore User Prompts at scale↗ slice-dice.notion.site · HN · 2 comments · 11 points · Jul 24, 2025
Positron – A next-generation data science IDE↗ positron.posit.co · HN · 92 comments · 220 points · Jul 21, 2025
Jul 14, 202527 stories
Great Tables: The Polars DataFrame Styler of Your Dreams↗ posit-dev.github.io · HN · 0 comments · 9 points · Jul 18, 2025
Data manipulation within the US Federal Government↗ thelancet.com · HN · 1 comments · 9 points · Jul 15, 2025
My Participation in the METR AI Productivity Study↗ domenic.me · HN · 1 comments · 13 points · Jul 15, 2025
Jul 07, 202550 stories
Big Data was used to see if TCM was scientific (2023)↗ mcgill.ca · HN · 124 comments · 122 points · Jul 13, 2025
HNSW as abstract data structure: video intro to Redis vector sets [video]↗ youtube.com · HN · 0 comments · 49 points · Jul 10, 2025
Show HN: Sumble – knowledge graph for GTM data – query tech stack, key projects↗ sumble.com · HN · 47 comments · 91 points · Jul 08, 2025
Jun 30, 202527 stories
How to Use JSONPath to Query and Extract JSON Data Efficiently↗ postpilot.dev · HN · 1 comments · 26 points · Jul 05, 2025
Stalking the Statistically Improbable Restaurant with Data↗ ethanzuckerman.com · HN · 38 comments · 84 points · Jul 03, 2025
Datadog's $65M/year customer mystery solved↗ blog.pragmaticengineer.com · HN · 62 comments · 152 points · Jun 30, 2025
Jun 23, 202542 stories
It doesn't make sense to wrap modern data in a 1979 format, introducing .ptar↗ plakar.io · HN · 11 comments · 14 points · Jun 28, 2025
Launch HN: Reducto Studio (YC W24) – Build accurate document pipelines, fast↗ news.ycombinator.com · HN · 55 comments · 85 points · Jun 23, 2025
Jun 16, 202532 stories
Jun 09, 202548 stories
Model Once, Represent Everywhere: UDA (Unified Data Architecture) at Netflix↗ netflixtechblog.com · HN · 103 comments · 140 points · Jun 14, 2025
Jun 02, 202544 stories
Making Magic with MCP: From Data Retrieval to Real Analysis and Insights↗ jellyfish.co · HN · 0 comments · 23 points · Jun 03, 2025
Archil: From a file system, to a data company↗ archil.com · HN · 3 comments · 12 points · Jun 02, 2025
May 26, 202542 stories
DuckLake is an integrated data lake and catalog format↗ ducklake.select · HN · 107 comments · 276 points · May 27, 2025
May 19, 202528 stories
New DSL "MassQL" lets scientists query mass spectrometry data↗ news.ucr.edu · HN · 10 comments · 23 points · May 23, 2025
Show HN: Bricks – One Click Dashboards from Your Data Using AI↗ app.thebricks.com · HN · 12 comments · 25 points · May 20, 2025
A lost decade chasing distributed architectures for data analytics?↗ duckdb.org · HN · 113 comments · 215 points · May 19, 2025
May 12, 202544 stories
Show HN: Fahmatrix – A Lightweight, Pandas-Like DataFrame Library for Java↗ github.com · HN · 13 comments · 46 points · May 17, 2025
Launch HN: ParaQuery (YC X25) – GPU Accelerated Spark/SQL↗ news.ycombinator.com · HN · 81 comments · 135 points · May 12, 2025
May 05, 202544 stories
Launch HN: Nao Labs (YC X25) – Cursor for Data↗ news.ycombinator.com · HN · 64 comments · 158 points · May 09, 2025
Show HN: Vom Decision Platform (Cursor for Decision Analyst)↗ vomdecision.com · HN · 16 comments · 15 points · May 09, 2025
NASA jettisons Neo4j database for Memgraph citing costs↗ theregister.com · HN · 1 comments · 7 points · May 07, 2025
Apr 28, 202536 stories
Show HN: Hyperparam: OSS tools for exploring datasets locally in the browser↗ hyperparam.app · HN · 21 comments · 77 points · May 01, 2025
Palantir's Justification for Building ICE's Master Database↗ 404media.co · HN · 17 comments · 76 points · Apr 29, 2025
Apr 21, 202520 stories
CloudEvents: A specification for describing event data in a common way↗ cloudevents.io · HN · 0 comments · 6 points · Apr 27, 2025
Show HN: SemHash – Semantic Text Deduplication, Outlier Filtering and Sampling↗ github.com · HN · 0 comments · 7 points · Apr 27, 2025
Show HN: CocoIndex – Open-Source Data framework for AI, built for data freshness↗ github.com · HN · 11 comments · 14 points · Apr 23, 2025
Apr 14, 202536 stories
Lessons learned operating petabyte-scale ClickHouse clusters: Part II↗ tinybird.co · HN · 2 comments · 61 points · Apr 20, 2025
Kaggle and the Wikimedia Foundation are partnering on open data↗ blog.google · HN · 38 comments · 136 points · Apr 16, 2025
Apr 07, 202524 stories
Ask HN: Why don't we have a functional DSL for data+embedding+API pipelines?↗ news.ycombinator.com · HN · 21 comments · 21 points · Apr 13, 2025
Show HN: Crystal, the most accurate U.S. gov't data search tool↗ askcrystal.info · HN · 8 comments · 23 points · Apr 13, 2025
Palantir Is Helping Doge with a Massive IRS Data Project↗ wired.com · HN · 23 comments · 68 points · Apr 11, 2025
Mar 31, 202530 stories
DEDA – Tracking Dots Extraction, Decoding and Anonymisation Toolkit↗ github.com · HN · 99 comments · 286 points · Apr 01, 2025
Lessons learned from operating large ClickHouse clusters in the last 6 years↗ tinybird.co · HN · 0 comments · 7 points · Apr 01, 2025
Show HN: Crystal, a tool for researching government data via plain English↗ askcrystal.info · HN · 1 comments · 6 points · Apr 01, 2025
Mar 24, 202551 stories
Palantir suggests 'common operating system' for UK govt data↗ theregister.com · HN · 76 comments · 49 points · Mar 25, 2025
The Missing Data Infrastructure for Physical AI↗ rerun.io · HN · 0 comments · 14 points · Mar 24, 2025
Mar 17, 202538 stories
Euclid opens data treasure trove, offers glimpse of deep fields↗ esa.int · HN · 21 comments · 107 points · Mar 20, 2025
Show HN: AGX – Open-Source Data Exploration for ClickHouse (The New Standard?)↗ github.com · HN · 3 comments · 48 points · Mar 19, 2025
Mar 10, 202529 stories
Undergraduate Upends a 40-Year-Old Data Science Conjecture↗ wired.com · HN · 0 comments · 4 points · Mar 16, 2025
Predictive Data Selection: The Data That Predicts Is the Data That Teaches↗ arxiv.org · HN · 0 comments · 5 points · Mar 14, 2025
Palantir and Databricks Announce Strategic Partnership↗ databricks.com · HN · 0 comments · 6 points · Mar 13, 2025
Mar 03, 202543 stories
Demos of Stream Processing Solving Real-World Problems↗ github.com · HN · 3 comments · 45 points · Mar 06, 2025
Apache iceberg the Hadoop of the modern-data-stack?↗ blog.det.life · HN · 65 comments · 115 points · Mar 06, 2025
Feb 24, 202537 stories
Exploring UK Environment Agency Data with DuckDB and Rill↗ rmoff.net · HN · 1 comments · 29 points · Feb 28, 2025
Show HN: I scrape Steam data every month and it's yours to download for free↗ gginsights.io · HN · 60 comments · 161 points · Feb 24, 2025
Feb 17, 202537 stories
Some critical issues with the SWE-bench dataset↗ arxiv.org · HN · 116 comments · 350 points · Feb 21, 2025
Building an Open, Multi-Engine Data Lakehouse with S3 and Python↗ tower.dev · HN · 11 comments · 71 points · Feb 18, 2025
Show HN: Bag of words – Build and share smart data apps using AI↗ github.com · HN · 8 comments · 12 points · Feb 17, 2025
Feb 10, 202540 stories
Federal data is disappearing. On Thursday, meet the teams working to rescue it↗ muckrock.com · HN · 86 comments · 180 points · Feb 13, 2025
Apache Iceberg now supports geospatial data types natively↗ wherobots.com · HN · 21 comments · 105 points · Feb 12, 2025
Show HN: OLake[open source] Fastest database to Iceberg data replication tool↗ news.ycombinator.com · HN · 3 comments · 14 points · Feb 10, 2025
Feb 03, 202538 stories
Classic Data science pipelines built with LLMs↗ github.com · HN · 86 comments · 196 points · Feb 09, 2025
Announcing the data.gov archive↗ lil.law.harvard.edu · HN · 132 comments · 382 points · Feb 07, 2025
Jan 27, 202538 stories
Archivists work to save disappearing data.gov datasets↗ 404media.co · HN · 238 comments · 801 points · Jan 30, 2025
Jan 20, 202543 stories
Apache Hudi: an open data lakehouse platform↗ github.com · HN · 5 comments · 27 points · Jan 22, 2025
Jan 13, 202533 stories
Show HN: Curator – an open-source library for synthetic data generation↗ github.com · HN · 6 comments · 13 points · Jan 15, 2025
Jan 06, 202540 stories
Dec 30, 202420 stories
B-Trees: More Than I Thought I'd Want to Know (2021)↗ benjamincongdon.me · HN · 16 comments · 319 points · Jan 03, 2025
A simple way to explore data through a Tableau-like UI directly in your data app↗ github.com · HN · 13 comments · 123 points · Dec 30, 2024
Dec 23, 202419 stories
Colab Notebook – RAG on Your Unstructured Data↗ github.com · HN · 0 comments · 14 points · Dec 26, 2024
Show HN: Augini, an AI-Powered Tabular Data Assistant↗ github.com · HN · 0 comments · 6 points · Dec 25, 2024
Dec 16, 202423 stories
Why R is the best coding language for data journalism↗ economist.com · HN · 12 comments · 19 points · Dec 18, 2024
Leadership Power Tools: SQL and Statistics↗ matt.blwt.io · HN · 19 comments · 51 points · Dec 18, 2024
Real-Time Feature Engineering with Denormalized and Feast↗ denormalized.io · HN · 0 comments · 13 points · Dec 17, 2024
Dec 09, 202442 stories
Surfer: Open-Source Personal Data Warehouse↗ github.com · HN · 21 comments · 147 points · Dec 14, 2024
How big data created the modern dairy cow↗ worksinprogress.co · HN · 58 comments · 48 points · Dec 13, 2024
Show HN: DataFuel.dev – Turn websites into LLM-ready data↗ datafuel.dev · HN · 34 comments · 43 points · Dec 13, 2024
Dec 02, 202444 stories
1,600 days of a failed hobby data science project↗ lellep.xyz · HN · 70 comments · 158 points · Dec 08, 2024
Data-engineer-handbook: everything to learn about data engineering↗ kifinity.com · HN · 1 comments · 61 points · Dec 03, 2024
Nov 25, 202441 stories
Data Modeling with Sums and Products↗ funktionale-programmierung.de · HN · 20 comments · 54 points · Nov 26, 2024
Nov 18, 202425 stories
Autoflow, a Graph RAG based and conversational knowledge base tool↗ github.com · HN · 36 comments · 280 points · Nov 22, 2024
Ask HN: Can PowerBI be replaced by Python?↗ news.ycombinator.com · HN · 2 comments · 3 points · Nov 21, 2024
Nov 11, 202438 stories
Non-elementary group-by aggregations in Polars vs pandas↗ labs.quansight.org · HN · 74 comments · 135 points · Nov 15, 2024
Show HN: Python Stream Processing with Denormalized↗ github.com · HN · 2 comments · 20 points · Nov 13, 2024
45% of U.S. Data Scientists Work at Just 20 Companies↗ index42.com · HN · 9 comments · 44 points · Nov 11, 2024
Nov 04, 202440 stories
Rd-TableBench – Accurately evaluating table extraction↗ reducto.ai · HN · 6 comments · 29 points · Nov 05, 2024
Oct 28, 202436 stories
Oct 21, 202443 stories
My NumPy year: Creating a DType for the next generation of scientific computing↗ quansight.com · HN · 15 comments · 96 points · Oct 23, 2024
Show HN: Data Formulator – AI-powered data visualization from Microsoft Research↗ github.com · HN · 34 comments · 212 points · Oct 21, 2024
Oct 14, 202453 stories
Drasi: Microsoft's open source data processing platform for event-driven systems↗ github.com · HN · 67 comments · 331 points · Oct 20, 2024
Ask HN: How big of a problem is unstructured data for companies?↗ news.ycombinator.com · HN · 16 comments · 16 points · Oct 14, 2024
A FLOSS platform for data analysis pipelines that you probably haven't heard of↗ arvados.org · HN · 6 comments · 47 points · Oct 14, 2024
Oct 07, 202444 stories
Defining Statistical Models in Jax?↗ statmodeling.stat.columbia.edu · HN · 18 comments · 118 points · Oct 13, 2024
Extracting financial disclosure and police reports with OpenAI Structured Output↗ gist.github.com · HN · 89 comments · 254 points · Oct 10, 2024
Improving Parquet Dedupe on Hugging Face Hub↗ huggingface.co · HN · 17 comments · 47 points · Oct 08, 2024
Sep 30, 202445 stories
Show HN: Loata; You've stored your data in the cloud, now store your knowledge↗ loata.ai · HN · 2 comments · 5 points · Oct 03, 2024
Alert Evaluations: Incremental Merges in ClickHouse↗ highlight.io · HN · 7 comments · 38 points · Oct 02, 2024
Sep 23, 202436 stories
Insights after 11 years with Datomic [video]↗ youtube.com · HN · 106 comments · 205 points · Sep 25, 2024
I spent 5 hours learning how ClickHouse built their internal data warehouse↗ vutr.substack.com · HN · 5 comments · 47 points · Sep 24, 2024
Data Science Agent and Code Transformation↗ labs.google.com · HN · 12 comments · 113 points · Sep 23, 2024
Sep 16, 202437 stories
Is anyone deploying GraphRAG in prod?↗ news.ycombinator.com · HN · 5 comments · 13 points · Sep 20, 2024
Show HN: Sourcetable – AI Spreadsheet and Data Platform↗ news.ycombinator.com · HN · 88 comments · 188 points · Sep 19, 2024
Netflix's Key-Value Data Abstraction Layer↗ netflixtechblog.com · HN · 18 comments · 93 points · Sep 19, 2024
Sep 09, 202445 stories
Data Engineering Vault: A 1000 Node Second Brain for DE Knowledge↗ ssp.sh · HN · 19 comments · 123 points · Sep 15, 2024
Sail – Unify stream processing, batch processing and compute-intensive workloads↗ github.com · HN · 15 comments · 79 points · Sep 10, 2024
Why I Wrote Data Science for Crime Analysis with Python (2023)↗ crimede-coder.com · HN · 15 comments · 123 points · Sep 09, 2024
Sep 02, 202430 stories
Show HN: Bridge-ds – Dataset handling for any modality a la Pandas↗ github.com · HN · 0 comments · 7 points · Sep 03, 2024
How to Build a Scalable Ingestion Pipeline for Enterprise GenAI Applications↗ enterprisebot.ai · HN · 0 comments · 7 points · Sep 03, 2024
Aug 26, 202448 stories
The Fall of StackOverflow: A Data-Driven Analysis↗ pdftranslate.ai · HN · 125 comments · 92 points · Aug 28, 2024
Aug 19, 202445 stories
Stripe Data vs. Open‐Source Alternatives: A MRR Example↗ github.com · HN · 69 comments · 167 points · Aug 25, 2024
Data Modeling SaaS Entitlements and Pricing↗ garrettdimon.com · HN · 1 comments · 51 points · Aug 22, 2024
Aug 12, 202432 stories
Project Oak: Meaningful control of data in distributed systems↗ github.com · HN · 50 comments · 147 points · Aug 14, 2024
Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data↗ news.ycombinator.com · HN · 116 comments · 234 points · Aug 13, 2024
Go Algorithms and Data Structures: Best Practices for Beginners↗ github.com · HN · 16 comments · 76 points · Aug 13, 2024
Aug 05, 202441 stories
Building data infrastructure that will last↗ seattledataguy.substack.com · HN · 82 comments · 162 points · Aug 11, 2024
Launch HN: Roe AI (YC W24) – AI-powered data warehouse to query multimodal data↗ news.ycombinator.com · HN · 35 comments · 60 points · Aug 09, 2024
Jul 29, 202448 stories
Show HN: Dashboards as Code – We make dashboards more programmable↗ news.ycombinator.com · HN · 1 comments · 23 points · Jul 31, 2024
Call of Duty: Warzone Caldera Data Set for Academic Use↗ blog.activision.com · HN · 95 comments · 178 points · Jul 31, 2024
Memory Efficient Data Streaming to Parquet Files↗ estuary.dev · HN · 2 comments · 28 points · Jul 29, 2024
Jul 22, 202444 stories
A User’s Guide to Statistical Inference and Regression↗ mattblackwell.github.io · HN · 7 comments · 148 points · Jul 25, 2024
Applied Machine Learning for Tabular Data↗ aml4td.org · HN · 20 comments · 154 points · Jul 25, 2024
A multimodal dataset with one trillion tokens↗ github.com · HN · 52 comments · 224 points · Jul 24, 2024
Jul 15, 202432 stories
Veles: Open-source tool for binary data analysis↗ codisec.com · HN · 23 comments · 128 points · Jul 21, 2024
Surprise, your data warehouse can RAG↗ rainforestqa.com · HN · 13 comments · 18 points · Jul 16, 2024
Jul 08, 202439 stories
Toto: A State-of-the-Art Time Series Foundation Model by Datadog↗ datadoghq.com · HN · 3 comments · 23 points · Jul 11, 2024
Jul 01, 202431 stories
Show HN: I Made an Open Source Platform for Structuring Any Unstructured Data↗ github.com · HN · 13 comments · 88 points · Jul 02, 2024
Jun 24, 202437 stories
The Missing Element in Your Data Architecture↗ materialize.com · HN · 0 comments · 13 points · Jun 27, 2024
Surprise, your data warehouse can RAG↗ rainforestqa.com · HN · 3 comments · 34 points · Jun 25, 2024
Jun 17, 202439 stories
Lessons Learned from Scaling to Multi-Terabyte Datasets↗ v2thegreat.com · HN · 45 comments · 149 points · Jun 20, 2024
Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data↗ epochai.org · HN · 58 comments · 56 points · Jun 18, 2024
Apache Arrow DataFusion: A Fast, Embeddable, Modular Analytic Query Engine↗ dl.acm.org · HN · 1 comments · 25 points · Jun 17, 2024
Jun 10, 202443 stories
Show HN: Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used)↗ github.com · HN · 19 comments · 73 points · Jun 13, 2024
Jun 03, 202443 stories
Ask HN: Is KDB a sane choice for a datalake in 2024?↗ news.ycombinator.com · HN · 40 comments · 40 points · Jun 09, 2024
Databricks to Buy Data-Management Startup Tabular↗ wsj.com · HN · 22 comments · 95 points · Jun 04, 2024
Koheesio: Nike's Python-based framework to build advanced data-pipelines↗ github.com · HN · 73 comments · 217 points · Jun 04, 2024
May 27, 202441 stories
FineWeb: Decanting the web for the finest text data at scale↗ huggingface.co · HN · 14 comments · 127 points · Jun 02, 2024
ROOT: analyzing petabytes of data scientifically↗ root.cern · HN · 93 comments · 356 points · Jun 01, 2024
May 20, 202438 stories
I uploaded 11 years of my Day One Journal data to GPT-4o↗ linkedin.com · HN · 25 comments · 34 points · May 24, 2024
Show HN: Cookiecutter Data Science – V2 Launch↗ cookiecutter-data-science.drivendata.org · HN · 0 comments · 9 points · May 22, 2024
Keeping It 100(x) with Real-Time Data at Scale – Figma Blog↗ figma.com · HN · 0 comments · 18 points · May 20, 2024
May 13, 202446 stories
Show HN: Open-source BI and analytics for engineers↗ github.com · HN · 75 comments · 293 points · May 15, 2024
May 06, 202438 stories
Launch HN: Baselit (YC W23) – Automatically Reduce Snowflake Costs↗ news.ycombinator.com · HN · 30 comments · 48 points · May 08, 2024
Apr 29, 202432 stories
Data Science at the Command Line, 2nd Edition (2021)↗ jeroenjanssens.com · HN · 30 comments · 173 points · May 03, 2024
The Sisyphean struggle and the new era of data infrastructure↗ jack-vanlightly.com · HN · 3 comments · 17 points · May 02, 2024
Show HN: Hamilton's UI – observability, lineage, and catalog for data pipelines↗ github.com · HN · 10 comments · 40 points · May 02, 2024
Apr 22, 202440 stories
So you want to scrape like the big boys (2021)↗ incolumitas.com · HN · 159 comments · 321 points · Apr 27, 2024
Building an open data pipeline in 2024↗ blog.twingdata.com · HN · 32 comments · 103 points · Apr 26, 2024
AI for Data Journalism: demonstrating what we can do with this stuff↗ simonwillison.net · HN · 32 comments · 167 points · Apr 22, 2024
Apr 15, 202449 stories
Dataflow Analyses and Compiler Optimizations That Use Them, for Free↗ blog.regehr.org · HN · 13 comments · 79 points · Apr 21, 2024
Ask HN: How are Data Scientist keeping themselves updated?↗ news.ycombinator.com · HN · 12 comments · 10 points · Apr 18, 2024
Loading a trillion rows of weather data into TimescaleDB↗ aliramadhan.me · HN · 177 comments · 405 points · Apr 16, 2024
Apr 08, 202444 stories
Data-as-a-Product and Data-Contract: An evolutionary approach to data maturity↗ blog.owulveryck.info · HN · 11 comments · 88 points · Apr 12, 2024
The River of Bytes: Elasticsearch Data Transfer Cost Optimization↗ tech.forter.com · HN · 0 comments · 18 points · Apr 10, 2024
Apr 01, 202449 stories
DataStax has acquired Langflow to accelerate generative AI development↗ datastax.com · HN · 4 comments · 54 points · Apr 04, 2024
DataStax just bought our startup Langflow↗ medium.com · HN · 24 comments · 107 points · Apr 04, 2024
Mar 25, 202447 stories
DeWitt and Stonebraker's "MapReduce: A major step backwards" (2009)↗ craig-henderson.blogspot.com · HN · 111 comments · 131 points · Mar 30, 2024
Launch HN: Thorntale (YC W24) – Presentation software that works with your data↗ news.ycombinator.com · HN · 39 comments · 77 points · Mar 28, 2024
Show HN: Spice.ai – materialize, accelerate, and query SQL data from any source↗ github.com · HN · 49 comments · 177 points · Mar 28, 2024
Mar 18, 202447 stories
Ten years of selling web data: a personal perspective↗ substack.thewebscraping.club · HN · 10 comments · 13 points · Mar 24, 2024
Nancy Pelosi discloses an up to $5M investment in Databricks [pdf]↗ disclosures-clerk.house.gov · HN · 0 comments · 5 points · Mar 22, 2024
Show HN: Tidepool – analytics for large text datasets↗ tidepool.so · HN · 1 comments · 5 points · Mar 21, 2024
Mar 11, 202442 stories
Vector: A high-performance observability data pipeline↗ github.com · HN · 57 comments · 171 points · Mar 17, 2024
'Data Is Rectangular' and Other Common Misconceptions [video]↗ youtube.com · HN · 0 comments · 58 points · Mar 17, 2024
How Figma's databases team lived to tell the scale↗ figma.com · HN · 223 comments · 568 points · Mar 14, 2024
Mar 04, 202451 stories
The Pile is a 825 GiB diverse, open-source language modelling data set (2020)↗ pile.eleuther.ai · HN · 234 comments · 332 points · Mar 07, 2024
Feb 26, 202457 stories
R: Introduction to Data Science (2019)↗ rafalab.dfci.harvard.edu · HN · 133 comments · 187 points · Mar 02, 2024
Show HN: I built an open-source data copy tool called ingestr↗ github.com · HN · 48 comments · 156 points · Feb 27, 2024
Blazer: Business intelligence made simple↗ github.com · HN · 37 comments · 215 points · Feb 27, 2024
Feb 19, 202440 stories
Lightning Data: Blazingly fast, distributed streaming of data from cloud storage↗ github.com · HN · 1 comments · 10 points · Feb 23, 2024
Data will not tell you what to do↗ mikkeldengsoe.substack.com · HN · 73 comments · 184 points · Feb 21, 2024
Show HN: Open-source reverse ETL – for devs and analysts to show business impact↗ github.com · HN · 0 comments · 11 points · Feb 20, 2024
Feb 12, 202427 stories
Depth Anything: Unleashing the power of large-scale unlabeled data↗ depth-anything.github.io · HN · 2 comments · 24 points · Feb 13, 2024
Show HN: Taipy: Building data and AI apps, an alternative to Streamlit↗ taipy.io · HN · 1 comments · 14 points · Feb 12, 2024
Feb 05, 202454 stories
Is the "modern data stack" still a useful idea?↗ roundup.getdbt.com · HN · 128 comments · 129 points · Feb 11, 2024
Data Structures for Data-Intensive Applications [pdf] (2023)↗ cs-people.bu.edu · HN · 26 comments · 583 points · Feb 09, 2024
SQL for data scientists in 100 queries↗ gvwilson.github.io · HN · 158 comments · 834 points · Feb 06, 2024
Jan 29, 202449 stories
A/B Testing – Everyday Data Science↗ everyday-data-science.tigyog.app · HN · 0 comments · 16 points · Feb 04, 2024
Build an open data stack with Airbyte, dbt and Dagster↗ github.com · HN · 0 comments · 25 points · Feb 01, 2024
Show HN: Dera – A platform to manage chunks and embeddings for building RAG apps↗ getdera.com · HN · 2 comments · 25 points · Feb 01, 2024
Jan 22, 202454 stories
Good DevEx increases productivity. Here is the data↗ github.blog · HN · 128 comments · 243 points · Jan 24, 2024
Ask HN: Does (or why does) anyone use MapReduce anymore?↗ news.ycombinator.com · HN · 67 comments · 106 points · Jan 24, 2024
Jan 15, 202441 stories
RAG Using Unstructured Data and Role of Knowledge Graphs↗ kuzudb.com · HN · 31 comments · 177 points · Jan 15, 2024
Tracking and mapping the data broker ecosystem↗ databrokerswatch.org · HN · 17 comments · 96 points · Jan 15, 2024
Jan 08, 202439 stories
Kafka Troubleshooting in Production (book launch)↗ news.ycombinator.com · HN · 2 comments · 13 points · Jan 13, 2024
Fine tuning is just synthetic data engineering↗ generatingconversation.substack.com · HN · 0 comments · 6 points · Jan 11, 2024
UI considerations for designing large data tables↗ coyleandrew.medium.com · HN · 29 comments · 39 points · Jan 10, 2024
Jan 01, 202430 stories
RAG Using Structured Data: Overview and Important Questions↗ kuzudb.com · HN · 4 comments · 5 points · Jan 04, 2024
Show HN: Data Painter – different way to interact with data in Jupyter notebook↗ github.com · HN · 1 comments · 7 points · Jan 02, 2024
Dec 25, 202325 stories
Understanding Parquet, Iceberg and Data Lakehouses↗ davidgomes.com · HN · 101 comments · 312 points · Dec 29, 2023
Natural Language Corpus Data: Beautiful Data (2009)↗ norvig.com · HN · 3 comments · 53 points · Dec 29, 2023
Data scientists need to learn about significant digits↗ lemire.me · HN · 65 comments · 68 points · Dec 25, 2023
Dec 18, 202331 stories
The Feature Store: what's new – a deep dive↗ hopsworks.ai · HN · 0 comments · 7 points · Dec 21, 2023
Dec 11, 202339 stories
Python Charting: Taming Big Data Without Crashing↗ taipy.io · HN · 3 comments · 8 points · Dec 12, 2023
On a roll: Qdrant brings sparse vectors, discovery search and custom sharding↗ qdrant.tech · HN · 0 comments · 16 points · Dec 12, 2023
What accounting goes into Snowflake's earnings report?↗ withorb.com · HN · 0 comments · 6 points · Dec 11, 2023
Dec 04, 202334 stories
The Future of BI: Exploring the Impact of BI-as-Code Tools with DuckDB↗ motherduck.com · HN · 0 comments · 3 points · Dec 07, 2023
Show HN: Open-Source Data Replication and Anonymization↗ github.com · HN · 0 comments · 9 points · Dec 07, 2023
Nov 27, 202340 stories
Ask HN: Does your work involve using data to make falsifiable predictions?↗ news.ycombinator.com · HN · 0 comments · 3 points · Dec 01, 2023
Show HN: Taipy – Turns Data and AI algorithms into full web applications↗ github.com · HN · 2 comments · 22 points · Nov 30, 2023
Show HN: Definite – Get a data stack (database, ETL, and BI) in one app↗ definite.app · HN · 1 comments · 4 points · Nov 29, 2023
Nov 20, 202335 stories
"Change Data Capture Breaks Encapsulation". Does It, Though?↗ decodable.co · HN · 2 comments · 3 points · Nov 25, 2023
Show HN: I've Published 28B Molecule Embeddings on AWS Open Data↗ ashvardanian.com · HN · 0 comments · 10 points · Nov 21, 2023
Nov 13, 202338 stories
Ask HN: Why Datadog succeed but New Relic failed?↗ news.ycombinator.com · HN · 1 comments · 4 points · Nov 16, 2023
Generative BI: Setting a New Standard for Business Intelligence↗ narrative.bi · HN · 0 comments · 3 points · Nov 16, 2023
Mison: A Fast JSON Parser for Data Analytics [pdf]↗ microsoft.com · HN · 0 comments · 5 points · Nov 14, 2023
Nov 06, 202337 stories
Domain-Specific Code Generation: How We Designed a System for Data Science↗ einblick.ai · HN · 0 comments · 4 points · Nov 08, 2023
ELTP: Extending ELT for Modern AI and Analytics↗ airbyte.com · HN · 0 comments · 16 points · Nov 07, 2023
AI product development is being held back by data engineering↗ arch.dev · HN · 3 comments · 8 points · Nov 07, 2023
Oct 30, 202340 stories
Show HN: Panda Patrol: Open-Source Data Pipeline Monitoring in < 5 Lines of Code↗ panda-patrol.vercel.app · HN · 1 comments · 5 points · Nov 03, 2023
US consumer spending dashboard built on data from 50M+ cards now on Snowflake↗ app.snowflake.com · HN · 0 comments · 8 points · Nov 01, 2023
Show HN: All I wanted was to grok some real-time data↗ teamdenormalized.substack.com · HN · 1 comments · 6 points · Oct 31, 2023
Oct 23, 202340 stories
Why we're building Scanner: data lake search must be fast↗ blog.scanner.dev · HN · 0 comments · 4 points · Oct 25, 2023
Ensuring Data Consistency for Continuously-Modified Data Across Replicas↗ engineering.mixpanel.com · HN · 0 comments · 10 points · Oct 25, 2023
Oct 16, 202338 stories
Show HN: Dask-expr – query optimization for faster Dask DataFrames↗ github.com · HN · 0 comments · 6 points · Oct 20, 2023
Joining CSV Data Without SQL: An IP Geolocation Use Case↗ brimdata.io · HN · 2 comments · 10 points · Oct 19, 2023
Five Design Principles for Building a Data Pipeline Engine↗ docs.tenzir.com · HN · 2 comments · 16 points · Oct 17, 2023
Oct 09, 202337 stories
Causal inference as a blind spot of data scientists↗ dzidas.com · HN · 1 comments · 17 points · Oct 15, 2023
Introduction to Modern Statistics↗ openintro-ims2.netlify.app · HN · 1 comments · 6 points · Oct 12, 2023
RAG at scale: Synchronizing and ingesting billions of text embeddings↗ medium.com · HN · 0 comments · 5 points · Oct 09, 2023
Oct 02, 202335 stories
Using Awk and R to parse 25tb (2019)↗ livefreeordichotomize.com · HN · 0 comments · 12 points · Oct 07, 2023
Sep 25, 202350 stories
Show HN: SapientML – Generative AutoML for Tabular Data↗ github.com · HN · 0 comments · 9 points · Sep 29, 2023
Ask HN: How do your ML teams version datasets and models?↗ news.ycombinator.com · HN · 4 comments · 9 points · Sep 28, 2023
Vectorize: A vector database for shipping AI-powered applications↗ blog.cloudflare.com · HN · 0 comments · 4 points · Sep 27, 2023
Sep 18, 202335 stories
Ask HN: Are there any uses of sorting algorithms in real life?↗ news.ycombinator.com · HN · 1 comments · 2 points · Sep 22, 2023
ClickHouse for usage metering with Kafka Connect↗ news.ycombinator.com · HN · 0 comments · 5 points · Sep 20, 2023
Sep 11, 202356 stories
BtrBlocks: Efficient Columnar Compression for Data Lakes [pdf]↗ cs.cit.tum.de · HN · 1 comments · 7 points · Sep 16, 2023
Show HN: InsightsGPT – Discovering GPT Use Cases, Data, and Productivity Gains↗ officeautomata.com · HN · 3 comments · 5 points · Sep 14, 2023
Evidence.dev raises $2M to bring open source BI to technical data teams↗ techcrunch.com · HN · 1 comments · 7 points · Sep 14, 2023
Sep 04, 202342 stories
AI Helped Us Add Vector Search to Cassandra in 6 Weeks↗ thenewstack.io · HN · 0 comments · 5 points · Sep 06, 2023
Show HN: PipeRider – open-source Data Impact Analysis for dbt changes↗ github.com · HN · 2 comments · 11 points · Sep 06, 2023
Show HN: Specfy – Stack Intelligence Platform↗ github.com · HN · 0 comments · 16 points · Sep 05, 2023
Aug 28, 202338 stories
Lilac: Analyze, structure, and clean unstructured data with AI↗ lilacml.com · HN · 2 comments · 2 points · Aug 30, 2023
Aug 21, 202343 stories
High-Dimensional Probability and Applications in Data Science↗ math.uci.edu · HN · 0 comments · 3 points · Aug 27, 2023
Show HN: Dataherald AI – Natural Language to SQL Engine↗ github.com · HN · 0 comments · 6 points · Aug 23, 2023
Evidence Labs – experimental data viz components for evidence.dev↗ labs.evidence.dev · HN · 0 comments · 3 points · Aug 22, 2023
Aug 14, 202335 stories
Ask HN: Where to find free public datasets besides Kaggle/BigQuery/HuggingFace?↗ news.ycombinator.com · HN · 2 comments · 3 points · Aug 18, 2023
The uses and abuses of Cloud Data Warehouses↗ materialize.com · HN · 1 comments · 9 points · Aug 16, 2023
Aug 07, 202334 stories
Show HN: Netdata released v1.42 and it includes 800 integrations↗ github.com · HN · 1 comments · 7 points · Aug 10, 2023
Show HN: Instant Datasets: Production data for development↗ news.ycombinator.com · HN · 0 comments · 10 points · Aug 09, 2023
Jul 31, 202327 stories
OpenDataDiscovery 0.15 with Data Deprecation and Metadata Stale↗ github.com · HN · 0 comments · 3 points · Aug 04, 2023
Ask HN: What plotting tools should I invest in learning?↗ news.ycombinator.com · HN · 1 comments · 3 points · Aug 02, 2023
Jul 24, 202346 stories
Data diffs: Algorithms for explaining what changed in a dataset (2022)↗ blog.marcua.net · HN · 0 comments · 16 points · Jul 27, 2023
Whom the Gods Would Destroy, They First Give Real-Time Analytics↗ mcfunley.com · HN · 3 comments · 17 points · Jul 25, 2023
Jul 17, 202332 stories
ClickPipes: Seamlessly Connect Kafka to ClickHouse↗ clickhouse.com · HN · 0 comments · 9 points · Jul 18, 2023
Launch HN: Grai (YC S22) – Open-Source Data Observability Platform↗ news.ycombinator.com · HN · 0 comments · 10 points · Jul 17, 2023
Jul 10, 202358 stories
Snakemake – A framework for reproducible data analysis↗ snakemake.github.io · HN · 3 comments · 26 points · Jul 15, 2023
The Barbie-Oppenheimer Double Feature Is Happening, Data Shows↗ bnnbloomberg.ca · HN · 6 comments · 16 points · Jul 12, 2023
Jul 03, 202328 stories
Powering rapid data applications using your data warehouse with VulcanSQL↗ vulcansql.com · HN · 0 comments · 2 points · Jul 09, 2023
The Era of ‘Quantitative Aesthetics,’ Where Data Points Dictate Taste↗ news.artnet.com · HN · 0 comments · 2 points · Jul 09, 2023
Ask HN: Resources for learning Scala and its practical usecases↗ news.ycombinator.com · HN · 1 comments · 2 points · Jul 08, 2023
Jun 26, 202346 stories
English as the new programming language for Apache spark↗ databricks.com · HN · 4 comments · 13 points · Jun 29, 2023
Qsv: CSVs sliced, diced and analyzed (fork of xsv)↗ github.com · HN · 1 comments · 3 points · Jun 27, 2023
How do you guys handle data migrations (not schema migrations)?↗ news.ycombinator.com · HN · 0 comments · 4 points · Jun 26, 2023
Jun 19, 202343 stories
Show HN: Download HuggingFace Models/Datasets easily and super fast↗ github.com · HN · 1 comments · 9 points · Jun 24, 2023
Show HN: MotherDuck – a serverless data analytics platform powered by DuckDB↗ news.ycombinator.com · HN · 0 comments · 3 points · Jun 22, 2023
BBC Visual and Data Journalism cookbook for R graphics↗ bbc.github.io · HN · 0 comments · 3 points · Jun 21, 2023
Jun 12, 202339 stories
Data-Driven Science and Engineering 2nd Edition [pdf]↗ faculty.washington.edu · HN · 1 comments · 6 points · Jun 17, 2023
Know compute cost of every pipeline and model in your BigQuery↗ mastheadata.com · HN · 1 comments · 6 points · Jun 15, 2023
Jun 05, 202337 stories
Show HN: Pluto – AI for investing, data visualization, automation and analysis↗ pluto.fi · HN · 3 comments · 3 points · Jun 09, 2023
Show HN: Open-source vector+data lake to debug, curate and version AI data↗ loom.com · HN · 0 comments · 5 points · Jun 07, 2023
Daft: A High-Performance Distributed Dataframe Library for Multimodal Data↗ blog.getdaft.io · HN · 1 comments · 8 points · Jun 07, 2023
May 29, 202338 stories
Ask HN: Literature on Ultra-Wide Tables?↗ news.ycombinator.com · HN · 0 comments · 4 points · Jun 04, 2023
Show HN: Renumics Spotlight – Multimodal exploration tool for your datasets↗ renumics.com · HN · 0 comments · 5 points · May 30, 2023
May 22, 202359 stories
A Guide to Continuous Data Quality Monitoring for AWS Athena↗ lariatdata.com · HN · 0 comments · 8 points · May 23, 2023
May 15, 202349 stories
Visualizing IoT data at scale by Michael Stonebraker↗ timescale.com · HN · 0 comments · 5 points · May 18, 2023
May 08, 202337 stories
Datadog’s $65M/year customer mystery solved↗ newsletter.pragmaticengineer.com · HN · 4 comments · 7 points · May 11, 2023
Analyze and Process Log Files with Azure Blob Storage, Azure Batch, and Kestra↗ kestra.io · HN · 0 comments · 4 points · May 09, 2023
Apache Sedona: Big Geospatial Data and AI Engine↗ github.com · HN · 2 comments · 6 points · May 08, 2023
May 01, 202341 stories
The Data Warehouse is dead – Long live the Lakehouse↗ medium.com · HN · 0 comments · 5 points · May 07, 2023
Show HN: Spotlight – The data quality platform for developers↗ spotlight.dev · HN · 0 comments · 5 points · May 03, 2023
Apr 24, 202334 stories
Palantir Foundry – A data operating system that is not talked about enough↗ medium.com · HN · 0 comments · 8 points · Apr 26, 2023
Stanford, Harvard data science no more↗ stanforddaily.com · HN · 5 comments · 9 points · Apr 26, 2023
ResearchGPT: Automated Data Analysis and Interpretation↗ phasellm.com · HN · 1 comments · 12 points · Apr 25, 2023
Apr 17, 202340 stories
Scaling Databases at Activision [pdf]↗ static.sched.com · HN · 0 comments · 15 points · Apr 21, 2023
A Tale of Three Real-Time OLAP Databases: Apache Pinot, Apache Druid, ClickHouse↗ startree.ai · HN · 2 comments · 40 points · Apr 20, 2023
Apr 10, 202336 stories
Data wrangling in Elixir with Explorer, the power of Rust, the elegance of R↗ news.livebook.dev · HN · 2 comments · 6 points · Apr 14, 2023
My Rusty Discovery: creating real-time data APIs with Rust, SQL, and YAML↗ news.ycombinator.com · HN · 1 comments · 9 points · Apr 10, 2023
Apr 03, 202348 stories
Show HN: Datree (YC W20) – End-to-End Policy Management for Kubernetes↗ news.ycombinator.com · HN · 3 comments · 24 points · Apr 04, 2023
The Great MLOps Hoax: Is It Just Data Engineering in Disguise?↗ cpard.xyz · HN · 4 comments · 7 points · Apr 04, 2023
Mar 27, 202347 stories
Data Scientists: the most ambiguous role of the 21st century↗ remiounadjela.substack.com · HN · 3 comments · 7 points · Mar 29, 2023
Mar 20, 202346 stories
I built a ChatGPT plugin to answer questions about data hosted in Datasette↗ simonwillison.net · HN · 0 comments · 4 points · Mar 24, 2023
Beginner's guide for new analytics engineers – 4 resources↗ news.ycombinator.com · HN · 0 comments · 6 points · Mar 22, 2023
Yandex open-sources its exabyte-scale big data platform↗ medium.com · HN · 0 comments · 13 points · Mar 22, 2023
Mar 13, 202350 stories
Analyzing multi-gigabyte JSON files locally↗ thenybble.de · HN · 13 comments · 19 points · Mar 18, 2023
Show HN: Ingest data from your customers (Prequel YC W21)↗ news.ycombinator.com · HN · 0 comments · 15 points · Mar 15, 2023
Launch HN: BuildFlow (YC W23) – The FastAPI of data pipelines↗ news.ycombinator.com · HN · 5 comments · 6 points · Mar 15, 2023
Mar 06, 202342 stories
Data Science, cutting through the Bullshit↗ remiounadjela.substack.com · HN · 0 comments · 8 points · Mar 08, 2023
Launch HN: DAGWorks – ML platform for data science teams↗ news.ycombinator.com · HN · 1 comments · 11 points · Mar 07, 2023
Databricks Announces General Availability of Model Serving↗ databricks.com · HN · 0 comments · 8 points · Mar 07, 2023
Feb 27, 202355 stories
Interactive Data Frames Powered by Foundation Models↗ meerkat.wiki · HN · 1 comments · 3 points · Mar 02, 2023
Understanding BigQuery Table Usage↗ tech.loveholidays.com · HN · 0 comments · 5 points · Feb 27, 2023
Most Data Work Seems Fundamentally Worthless↗ ludic.mataroa.blog · HN · 20 comments · 57 points · Feb 27, 2023
Feb 20, 202329 stories
Lawyer Reacts: Web Scraping Data? Watch This First↗ youtube.com · HN · 1 comments · 2 points · Feb 23, 2023
Data Science has a tooling problem↗ counting.substack.com · HN · 0 comments · 3 points · Feb 22, 2023
Ask HN: Best Resources to Learn ML Infra and API Creation for Web Devs?↗ news.ycombinator.com · HN · 0 comments · 3 points · Feb 21, 2023
Feb 13, 202343 stories
Ask HN: Is data science a dying profession?↗ news.ycombinator.com · HN · 1 comments · 4 points · Feb 16, 2023
Show HN: Oxen.ai – Fast Unstructured Data Version Control↗ github.com · HN · 0 comments · 7 points · Feb 16, 2023
Show HN: Data driven decisions in OSS [helm-dashboard]↗ docs.google.com · HN · 0 comments · 3 points · Feb 16, 2023
Feb 06, 202346 stories
Replacing a SQL Analyst with Recursive GPT Prompts – Part II, Matplotlib↗ patterns.app · HN · 1 comments · 9 points · Feb 09, 2023
Synchronize Data Between Memgraph Graph Database and Elasticsearch↗ memgraph.com · HN · 1 comments · 6 points · Feb 06, 2023
Jan 30, 202348 stories
Datadog asked developer to kill open source exporter↗ theregister.com · HN · 0 comments · 4 points · Feb 05, 2023
Fix your dirty Salesforce – Datadog for CRM↗ fixyoursalesforce.com · HN · 1 comments · 4 points · Feb 02, 2023
Jan 23, 202344 stories
Show HN: Sync’ing data to your customer’s Google Sheets↗ news.ycombinator.com · HN · 0 comments · 8 points · Jan 27, 2023
DataDog asked OpenTelemetry contributor to kill pull request↗ github.com · HN · 29 comments · 143 points · Jan 26, 2023
Two-way syncs across your data stack and SaaS tools↗ news.ycombinator.com · HN · 1 comments · 5 points · Jan 24, 2023
Jan 16, 202343 stories
Show HN: New kind of data notebook for SQL, Python, and viz↗ hyperquery.ai · HN · 1 comments · 4 points · Jan 20, 2023
Stay Ahead of Cyber Threats with Graph Databases↗ memgraph.com · HN · 1 comments · 6 points · Jan 20, 2023
Xorbits: Fits all your imagination about scaling Python data science world↗ xorbits.io · HN · 2 comments · 5 points · Jan 18, 2023
Jan 09, 202348 stories
Perform What-If Analysis of Your Network Without Compromising Data Integrity↗ memgraph.com · HN · 1 comments · 11 points · Jan 13, 2023
UC Berkeley researchers set new visions for making CRDT data stores practical [pdf]↗ vldb.org · HN · 1 comments · 11 points · Jan 10, 2023
Jan 02, 202338 stories
Salesforce Guts Tableau After Spending $15.7B in 2019 Deal↗ bloomberg.com · HN · 1 comments · 2 points · Jan 06, 2023
Dec 26, 202226 stories
Pouria Hadjibagheri and the UK’s abandoned open data revolution↗ spectator.co.uk · HN · 0 comments · 3 points · Jan 01, 2023
Improving load times of data-rich dashboards↗ medium.com · HN · 1 comments · 3 points · Dec 27, 2022
Dec 19, 202237 stories
Ask HN: Is the industry saturated with data scientists already?↗ news.ycombinator.com · HN · 1 comments · 13 points · Dec 22, 2022
Data Warehouse vs. Operational Database What? How? Which One?↗ airbyte.com · HN · 1 comments · 4 points · Dec 19, 2022
Dec 12, 202237 stories
Show HN: Complete catalog of data connectors and ETL providers↗ connectorcatalog.com · HN · 2 comments · 5 points · Dec 15, 2022
Dec 05, 202240 stories
ELT Schedules Can Improve Root Cause Analysis for Data Engineers↗ montecarlodata.com · HN · 1 comments · 5 points · Dec 08, 2022
Free E-Book: The Cost of Building Your Own Marketing Data Pipeline↗ improvado.io · HN · 0 comments · 4 points · Dec 07, 2022
Write Airflow DAGs using vanilla Python and SQL, notebook style↗ astronomer.io · HN · 0 comments · 4 points · Dec 05, 2022
Nov 28, 202258 stories
Netdata 1.37.0: Infinite scalability, database tiering, and much more↗ blog.netdata.cloud · HN · 2 comments · 6 points · Nov 30, 2022
A Julia package for high-throughput manipulation of structured signal data↗ github.com · HN · 0 comments · 4 points · Nov 30, 2022
GreptimeDB Announces to Offer Distributed Solutions for Time-Series↗ greptime.com · HN · 0 comments · 8 points · Nov 29, 2022
Nov 21, 202229 stories
Data Rabbit – canvas for composable data exploration and experimentation↗ datarabbit.com · HN · 9 comments · 61 points · Nov 21, 2022
Kangas: Explore Multimedia Datasets at Scale↗ github.com · HN · 2 comments · 9 points · Nov 21, 2022
Nov 14, 202246 stories
Ask HN: What are the funnest data sets you’ve played with?↗ news.ycombinator.com · HN · 10 comments · 28 points · Nov 20, 2022
Show HN: Narrative BI – Actionable analytics for growth teams↗ narrative.bi · HN · 2 comments · 41 points · Nov 16, 2022
Launch HN: Activeloop (YC S18) – Data lake for deep learning↗ news.ycombinator.com · HN · 24 comments · 64 points · Nov 15, 2022
Nov 07, 202238 stories
Big data is at least four different problems (2015) [video]↗ youtube.com · HN · 5 comments · 48 points · Nov 07, 2022
Oct 31, 202252 stories
Show HN: I used streaming to skip downloading my 45GB dataset↗ github.com · HN · 0 comments · 31 points · Nov 02, 2022
Phlare: open-source database for continuous profiling at scale↗ grafana.com · HN · 40 comments · 240 points · Nov 02, 2022
Oct 24, 202246 stories
Launch HN: MovingLake (YC S22) – Real-time data connectors for almost anything↗ app.movinglake.com · HN · 65 comments · 132 points · Oct 27, 2022
Vercel acquires Splitbee to expand first-party analytics↗ vercel.com · HN · 2 comments · 85 points · Oct 25, 2022
The evolution of the data engineer role↗ airbyte.com · HN · 117 comments · 167 points · Oct 24, 2022
Oct 17, 202236 stories
Show HN: Lance – Deep Learning with DuckDB and Arrow↗ github.com · HN · 1 comments · 65 points · Oct 19, 2022
Oct 10, 202234 stories
State of Data Science and Machine Learning 2022↗ kaggle.com · HN · 0 comments · 24 points · Oct 10, 2022
Show HN: Cito – Actionable data observability for every data team↗ citodata.com · HN · 1 comments · 5 points · Oct 10, 2022
Kuser_shared_data (Windows Data Structure)↗ geoffchappell.com · HN · 3 comments · 26 points · Oct 10, 2022
Oct 03, 202237 stories
Lightdash takes on Looker with an open source BI platform built for dbt↗ techcrunch.com · HN · 2 comments · 20 points · Oct 06, 2022
Cassandra at Netflix: 900 clusters, 22,000 nodes, 12 PB of data, 12M req/s↗ twitter.com · HN · 9 comments · 19 points · Oct 06, 2022
Sep 26, 202241 stories
DocArray: a data structure for unstructured data↗ docarray.jina.ai · HN · 13 comments · 83 points · Oct 02, 2022
Launch HN: Prequel (YC W21) – Sync data to your customer’s data warehouse↗ news.ycombinator.com · HN · 40 comments · 118 points · Sep 26, 2022
Sep 19, 202238 stories
Powering AI with Vector Databases: A Benchmark↗ farfetchtechblog.com · HN · 0 comments · 14 points · Sep 23, 2022
Show HN: Open Prompts – dataset of 10M Stable Diffusion generations↗ github.com · HN · 71 comments · 279 points · Sep 22, 2022
_synthesize, the Developer Conference for Synthetic Data↗ gretel.ai · HN · 2 comments · 25 points · Sep 20, 2022
Sep 12, 202243 stories
Retrospection and Learnings from Dgraph Labs↗ manishrjain.com · HN · 38 comments · 106 points · Sep 16, 2022
Entity Resolution: Reflections on the most common data science challenge↗ tilores.io · HN · 8 comments · 60 points · Sep 13, 2022
Sep 05, 202244 stories
Meta Doesn't Know Where All Your Data Is, Engineers Say↗ businessinsider.com · HN · 6 comments · 15 points · Sep 09, 2022
Aug 29, 202231 stories
Scraping data from an app: real world example↗ substack.thewebscraping.club · HN · 24 comments · 117 points · Sep 04, 2022
Show HN: Open-source infra for building embedded data pipelines↗ github.com · HN · 20 comments · 41 points · Sep 01, 2022
Show HN: DataSloth - Query Pandas using natural language↗ github.com · HN · 2 comments · 14 points · Aug 31, 2022
Aug 22, 202243 stories
Show HN: Directly query 100M+ raw records of cleaned, refreshed financial data↗ wizdata.io · HN · 0 comments · 8 points · Aug 28, 2022
Show HN: We made an open-source visual data ETL↗ go.instill.tech · HN · 2 comments · 8 points · Aug 25, 2022
Show HN: VDP – open-source unstructured visual data ETL↗ github.com · HN · 1 comments · 11 points · Aug 22, 2022
Aug 15, 202244 stories
OLTP, OLAP, Data Warehouses, Data Lakes: what does it all mean?↗ cloudnatively.com · HN · 14 comments · 32 points · Aug 17, 2022
Physically Based: A Database of PBR Values for Real-World Materials↗ 80.lv · HN · 46 comments · 121 points · Aug 17, 2022
Show HN: Data pipelines for data scientists (new open-source notebook)↗ mage.ai · HN · 0 comments · 36 points · Aug 16, 2022
Aug 08, 202248 stories
Launch HN: Sematic (YC S22) – Open-source framework to build ML pipelines faster↗ news.ycombinator.com · HN · 47 comments · 121 points · Aug 10, 2022
Aug 01, 202254 stories
Loading Data into Pandas: Tips and Tricks You May or May Not Know↗ dataground.io · HN · 12 comments · 49 points · Aug 03, 2022
JuliaEpi: Collaborative Computational Epidemiology in Julia↗ forem.julialang.org · HN · 5 comments · 48 points · Aug 02, 2022
Jul 25, 202244 stories
CS109a: Introduction to Data Science – Resources↗ harvard-iacs.github.io · HN · 25 comments · 223 points · Jul 31, 2022
The Plausibly Deniable DataBase (PDDB): It’s Real Now↗ bunniestudios.com · HN · 40 comments · 95 points · Jul 28, 2022
Jul 18, 202237 stories
Arquero – data tables wrangling in JavaScript↗ uwdata.github.io · HN · 2 comments · 67 points · Jul 22, 2022
New method identifies the root causes of statistical outliers↗ amazon.science · HN · 53 comments · 107 points · Jul 19, 2022
Jul 11, 202233 stories
We demand data – The story of Lucid and eduction↗ billwadge.com · HN · 4 comments · 34 points · Jul 17, 2022
Interactive course about “everyday” data science↗ tigyog.app · HN · 52 comments · 153 points · Jul 16, 2022
Show HN: A benchmark for analytical databases (Snowflake, Druid, Redshift)↗ benchmark.clickhouse.com · HN · 58 comments · 137 points · Jul 13, 2022
Jul 04, 202234 stories
The new frontier of the Modern Data Stack: data-driven operations↗ logicloop.com · HN · 0 comments · 5 points · Jul 08, 2022
How much should you be earning as a Data Engineer?↗ evidence.dev · HN · 1 comments · 7 points · Jul 05, 2022
Jun 27, 202241 stories
I just got access to DALL·E 2 and here are my first few results↗ currentlyobsessed.com · HN · 32 comments · 67 points · Jul 02, 2022
The Log: Real-time data's unifying abstraction (2013)↗ engineering.linkedin.com · HN · 3 comments · 30 points · Jul 01, 2022
Show HN: Featureform – An open-source Feature Store for ML↗ github.com · HN · 27 comments · 136 points · Jun 30, 2022
Jun 20, 202255 stories
Ask HN: What Happened to Big Data?↗ news.ycombinator.com · HN · 74 comments · 66 points · Jun 23, 2022
Show HN: Data Diff – compare tables of any size across databases↗ news.ycombinator.com · HN · 21 comments · 127 points · Jun 22, 2022
A Data-Centric Introduction to Computing↗ dcic-world.org · HN · 15 comments · 106 points · Jun 20, 2022
Jun 13, 202243 stories
Making the collective knowledge of chemistry open and machine actionable↗ nature.com · HN · 29 comments · 102 points · Jun 14, 2022
Comparison of Data Lake Table Formats (Iceberg, Hudi and Delta Lake)↗ dremio.com · HN · 38 comments · 96 points · Jun 13, 2022
Jun 06, 202236 stories
Automating Data Protection at Scale, Part 3↗ medium.com · HN · 1 comments · 28 points · Jun 09, 2022
Bad data analysis questions I see every week (and how to fix them)↗ narrator.ai · HN · 1 comments · 12 points · Jun 08, 2022
May 30, 202247 stories
Entity Resolution: The most common data science challenge↗ docs.magniv.io · HN · 26 comments · 85 points · Jun 01, 2022
Show HN: DataStation – App to easily query, script, and visualize data↗ github.com · HN · 31 comments · 131 points · May 31, 2022
May 23, 202240 stories
Data Science in Context – Peter Norvig's New Book↗ datascienceincontext.com · HN · 40 comments · 308 points · May 24, 2022
Lessons learned from running Apache Airflow at scale↗ shopify.engineering · HN · 182 comments · 285 points · May 23, 2022
May 16, 202236 stories
Data Integration Guide: Techniques, Technologies, and Tools↗ airbyte.com · HN · 2 comments · 57 points · May 20, 2022
Super-Structured Data: Rethinking the Schema↗ brimdata.io · HN · 46 comments · 108 points · May 17, 2022
May 09, 202245 stories
Thinking Machines – Introduction to Data Parallel Supercomputing (1989)↗ youtube.com · HN · 23 comments · 58 points · May 15, 2022
Data-Driven Deportation in the 21st Century↗ americandragnet.org · HN · 56 comments · 63 points · May 13, 2022
Ask HN: How to get good at data modelling?↗ news.ycombinator.com · HN · 4 comments · 29 points · May 09, 2022
May 02, 202242 stories
Ask HN: Where do you get your financial data?↗ news.ycombinator.com · HN · 17 comments · 25 points · May 05, 2022
Apr 25, 202242 stories
Explorer – A library to bring dataframes to Elixir↗ cigrainger.com · HN · 24 comments · 124 points · Apr 28, 2022
Apr 18, 202246 stories
Max Datom: Interactive Datomic Tutorial↗ max-datom.com · HN · 30 comments · 62 points · Apr 21, 2022
Show HN: DTL: a language and JavaScript lib to transform and manipulate data↗ getdtl.org · HN · 20 comments · 63 points · Apr 20, 2022
Ask HN: Which new skills for a data science career?↗ news.ycombinator.com · HN · 42 comments · 75 points · Apr 20, 2022
Apr 11, 202240 stories
How to balance between delete compliance and backups↗ news.ycombinator.com · HN · 52 comments · 69 points · Apr 13, 2022
Apr 04, 202243 stories
Atlassian outage lingers, sparking data loss fears↗ theregister.com · HN · 16 comments · 54 points · Apr 10, 2022
Show HN: Turn any data into a fast analytical API↗ columns.ai · HN · 9 comments · 26 points · Apr 08, 2022
Meta quickly detects silent data corruptions at scale↗ engineering.fb.com · HN · 95 comments · 167 points · Apr 04, 2022
Mar 28, 202249 stories
The interesting ideas in Datasette (2018)↗ simonwillison.net · HN · 15 comments · 86 points · Apr 02, 2022
Mar 21, 202239 stories
A Year of BonsaiDb: A retrospective and looking to the future↗ bonsaidb.io · HN · 7 comments · 81 points · Mar 25, 2022
Aggregating away the signal in your data↗ stackoverflow.blog · HN · 11 comments · 113 points · Mar 21, 2022
Mar 14, 202243 stories
Launch HN: Sarus (YC W22) – Work on sensitive data with differential privacy↗ news.ycombinator.com · HN · 49 comments · 136 points · Mar 16, 2022
Organizing and scaling an effective data team↗ robdearborn.com · HN · 7 comments · 62 points · Mar 15, 2022
Mar 07, 202241 stories
Shopify's Data Science and Engineering Foundations (2020)↗ shopify.engineering · HN · 25 comments · 205 points · Mar 11, 2022
Phases of Netflix’s real-time data infrastructure↗ zhenzhongxu.com · HN · 47 comments · 164 points · Mar 10, 2022
Feb 28, 202235 stories
Launch HN: Elementary (YC W22) – Open-source data observability↗ news.ycombinator.com · HN · 30 comments · 122 points · Mar 04, 2022
Snowflake acquires Streamlit for $800M to help customers build data-based apps↗ techcrunch.com · HN · 44 comments · 129 points · Mar 02, 2022
Ask HN: I realise data science is not for me and feel depressed↗ news.ycombinator.com · HN · 140 comments · 146 points · Mar 01, 2022
Feb 21, 202236 stories
PlanetScale increases plans to include billions of reads↗ planetscale.com · HN · 68 comments · 136 points · Feb 24, 2022
Become a better data engineer on a shoestring (free resources)↗ dataengineering.academy · HN · 25 comments · 161 points · Feb 24, 2022
A curated list of libraries, packages and resources for Quants↗ github.com · HN · 1 comments · 30 points · Feb 22, 2022
Feb 14, 202244 stories
Data diffs: Algorithms for explaining what changed in a dataset↗ blog.marcua.net · HN · 1 comments · 21 points · Feb 20, 2022
Show HN: Magniv, One line data science infra↗ magniv.app · HN · 9 comments · 40 points · Feb 14, 2022
Feb 07, 202236 stories
Building real-time ETL pipelines with Apache Kafka↗ datacater.io · HN · 10 comments · 22 points · Feb 11, 2022
The Plausibly Deniable DataBase (PDDB)↗ bunniestudios.com · HN · 30 comments · 107 points · Feb 08, 2022
Jan 31, 202246 stories
Launch HN: Ploomber (YC W22) – Quickly deploy data pipelines from Jupyter/VSCode↗ github.com · HN · 23 comments · 126 points · Feb 03, 2022
The Economics of Data Businesses↗ pivotal.substack.com · HN · 5 comments · 109 points · Feb 01, 2022
Jan 24, 202255 stories
Show HN: Electric Tables – an experiment in personal databases↗ tomcritchlow.com · HN · 37 comments · 231 points · Jan 26, 2022
Introduction to Probability for Data Science↗ probability4datascience.com · HN · 15 comments · 235 points · Jan 24, 2022
Jan 17, 202240 stories
Ask HN: What language will you pick if you are to reinvent the data ecosystem?↗ news.ycombinator.com · HN · 21 comments · 8 points · Jan 23, 2022
Kedro – Creating reproducible, maintainable and modular data science code↗ github.com · HN · 40 comments · 96 points · Jan 22, 2022
Jan 10, 202238 stories
Project Cybersyn and the Origins of the Big Data Nation (2014)↗ newyorker.com · HN · 39 comments · 55 points · Jan 14, 2022
SAGE jumpstarted today’s technology and built IBM into a powerhouse↗ arstechnica.com · HN · 31 comments · 76 points · Jan 12, 2022
Apache Flink ML 2.0 is released: ML for streaming data↗ flink.apache.org · HN · 2 comments · 22 points · Jan 10, 2022
Jan 03, 202235 stories
Drinking from the Fetid Well: Data Poisoning and Machine Learning↗ usni.org · HN · 0 comments · 19 points · Jan 09, 2022
The Modern Metadata Platform: What, Why, and How?↗ metaphor.io · HN · 18 comments · 35 points · Jan 06, 2022
Ask HN: Any data scientists interested in dating app data?↗ news.ycombinator.com · HN · 8 comments · 7 points · Jan 04, 2022
Dec 27, 202132 stories
GLADOS – Generic Life Activity Data Organization System↗ github.com · HN · 41 comments · 80 points · Jan 01, 2022
The Science of Visual Data Communication: What Works↗ journals.sagepub.com · HN · 11 comments · 117 points · Dec 30, 2021
Big data may not know your name, but it knows everything else↗ wired.com · HN · 69 comments · 190 points · Dec 30, 2021
Dec 20, 202135 stories
Ask HN: What are the best tech books you read in 2021?↗ news.ycombinator.com · HN · 30 comments · 95 points · Dec 26, 2021
Annotated explanation of David Beazley's dataklasses↗ til.simonwillison.net · HN · 16 comments · 108 points · Dec 20, 2021
The death of feature engineering is greatly exaggerated↗ petewarden.com · HN · 5 comments · 60 points · Dec 20, 2021
Dec 13, 202137 stories
Show HN: A labelling tool to easily extract and label Wikipedia data↗ news.ycombinator.com · HN · 13 comments · 116 points · Dec 17, 2021
Using an ETL framework vs. writing yet another ETL script↗ airbyte.io · HN · 74 comments · 114 points · Dec 17, 2021
Show HN: Dataherald (YC W21) – Tableau with ready-to-use data↗ dataherald.com · HN · 8 comments · 25 points · Dec 14, 2021
Dec 06, 202155 stories
Ask HN: What is the best book on data visualization in 2021?↗ news.ycombinator.com · HN · 33 comments · 105 points · Dec 06, 2021
Nov 29, 202136 stories
Hamilton: A Microframework for Dataframe Generation↗ multithreaded.stitchfix.com · HN · 21 comments · 56 points · Dec 02, 2021
The Guide to Data Analysis with DuckDB↗ analyticsvidhya.com · HN · 0 comments · 12 points · Dec 01, 2021
Nov 22, 202129 stories
How to create a data catalog, a step by step guide↗ news.ycombinator.com · HN · 0 comments · 12 points · Nov 25, 2021
It's that Data Science meme plane with dots again↗ counting.substack.com · HN · 0 comments · 12 points · Nov 23, 2021
Examples of the Importance of Context-Sensitivity in Data-Centric AI↗ surgehq.ai · HN · 1 comments · 19 points · Nov 23, 2021
Nov 15, 202151 stories
Processing billions of events in real time↗ blog.twitter.com · HN · 104 comments · 200 points · Nov 15, 2021
Databricks response to Snowflake's accusation of lacking integrity↗ databricks.com · HN · 156 comments · 217 points · Nov 15, 2021
Launch HN: Metaplane (YC W20) – Datadog for Data↗ news.ycombinator.com · HN · 67 comments · 163 points · Nov 15, 2021
Nov 08, 202149 stories
Snowflake’s response to Databricks’ TPC-DS post↗ snowflake.com · HN · 102 comments · 80 points · Nov 13, 2021
Launch HN: Hightouch (YC S19) – Sync data from data warehouses to SaaS tools↗ news.ycombinator.com · HN · 47 comments · 132 points · Nov 11, 2021
Show HN: Hamilton, a Microframework for Creating Dataframes↗ github.com · HN · 41 comments · 64 points · Nov 09, 2021
Nov 01, 202140 stories
Spending $5k to learn how database indexes work↗ briananglin.me · HN · 199 comments · 236 points · Nov 06, 2021
Launch HN: Jitsu (YC S20) – Open-Source Segment Alternative↗ news.ycombinator.com · HN · 110 comments · 265 points · Nov 04, 2021
Show HN: Open-source, self-hosted data management app and internal tool builder↗ news.ycombinator.com · HN · 27 comments · 82 points · Nov 02, 2021
Oct 25, 202157 stories
Tao: Facebook’s distributed data store for the social graph↗ micahlerner.com · HN · 39 comments · 170 points · Oct 30, 2021
Retro collectors are uncovering hoards of old data↗ wired.co.uk · HN · 59 comments · 101 points · Oct 29, 2021
We need new data books, so we published one↗ theinformedcompany.com · HN · 31 comments · 171 points · Oct 26, 2021
Oct 18, 202147 stories
Issues we've encountered while building a Kafka based data processing pipeline↗ sixfold.medium.com · HN · 93 comments · 158 points · Oct 18, 2021
How Time Series Databases Work, and Where They Don’t↗ honeycomb.io · HN · 55 comments · 228 points · Oct 18, 2021
Oct 11, 202139 stories
Wavelets allow researchers to transform and understand data↗ quantamagazine.org · HN · 40 comments · 129 points · Oct 13, 2021
I’m a Data Scientist – Here’s why I work at Facebook↗ medium.com · HN · 15 comments · 20 points · Oct 12, 2021
Show HN: Lago (YC S21): Operationalize Metabase dashboards in a few clicks↗ getlago.com · HN · 3 comments · 17 points · Oct 11, 2021
Oct 04, 202137 stories
ETL Pipelines with Airflow: The Good, the Bad and the Ugly↗ airbyte.io · HN · 77 comments · 144 points · Oct 08, 2021
Why Your Data Warehouse Should Be the Foundation of Your CDP↗ rudderstack.com · HN · 0 comments · 6 points · Oct 07, 2021
Sep 27, 202131 stories
Statistics as algorithmic summarization↗ benjamin-recht.github.io · HN · 18 comments · 56 points · Sep 29, 2021
Ask HN: Is the job market for data scientists saturated?↗ news.ycombinator.com · HN · 0 comments · 8 points · Sep 28, 2021
Big data in the age of the telegraph (2013)↗ mckinsey.com · HN · 5 comments · 41 points · Sep 27, 2021
Sep 20, 202136 stories
Why I Use Nim instead of Python for Data Processing↗ benjamindlee.com · HN · 178 comments · 256 points · Sep 23, 2021
Ask HN: Why do product devs switch into data eng/data ops roles?↗ news.ycombinator.com · HN · 15 comments · 11 points · Sep 21, 2021
Sep 13, 202138 stories
Is BI dead? – On dismantling data's ship of Theseus↗ benn.substack.com · HN · 116 comments · 174 points · Sep 18, 2021
Sep 06, 202142 stories
The IDE for Professional Data Scientists↗ jetbrains.com · HN · 5 comments · 18 points · Sep 09, 2021
Datasette Desktop–a macOS desktop application for Datasette↗ simonwillison.net · HN · 1 comments · 12 points · Sep 08, 2021
Show HN: Cogram – Copilot for data science and ML↗ cogram.ai · HN · 5 comments · 10 points · Sep 06, 2021
Aug 30, 202130 stories
What Is an OLAP Cube? An Exhaustive Explainer↗ holistics.io · HN · 19 comments · 86 points · Sep 03, 2021
Lessons Learned from Two Years as a Data Scientist↗ dawndrain.github.io · HN · 83 comments · 141 points · Sep 02, 2021
Aug 23, 202128 stories
Launch HN: Evidence (YC S21) – Web framework for data analysts↗ news.ycombinator.com · HN · 91 comments · 263 points · Aug 25, 2021
Statistical process control after W. Edwards Deming (2020)↗ 2uo.de · HN · 47 comments · 81 points · Aug 25, 2021
Apache Arrow Datafusion 5.0.0 release↗ arrow.apache.org · HN · 44 comments · 78 points · Aug 24, 2021
Aug 16, 202137 stories
B-Trees: More Than I Thought I'd Want to Know↗ benjamincongdon.me · HN · 63 comments · 239 points · Aug 18, 2021
Classical data structures that can outperform learned indexes (2018)↗ dawn.cs.stanford.edu · HN · 39 comments · 252 points · Aug 16, 2021
Aug 09, 202135 stories
Classical Data Structures That Can Outperform Learned Indexes↗ dawn.cs.stanford.edu · HN · 0 comments · 5 points · Aug 15, 2021
Why Observability Requires a Distributed Column Store↗ honeycomb.io · HN · 0 comments · 3 points · Aug 14, 2021
Aug 02, 202134 stories
The Oxymoron of “Data-Driven Innovation”↗ chelseatroy.com · HN · 48 comments · 95 points · Aug 02, 2021
Jul 26, 202139 stories
The Baked Data architectural pattern↗ simonwillison.net · HN · 23 comments · 70 points · Jul 29, 2021
Evolution of tree data structures for indexing↗ erthalion.info · HN · 12 comments · 101 points · Jul 26, 2021
Jul 19, 202143 stories
The Impedance Mismatch Test: Is Your Data Layer a Complex Mess?↗ thenewstack.io · HN · 21 comments · 24 points · Jul 22, 2021
Putting the power of AlphaFold into the world’s hands↗ deepmind.com · HN · 4 comments · 13 points · Jul 22, 2021
Jul 12, 202135 stories
Show HN: Work with CSV files using SQL. For data scientists and engineers↗ superintendent.app · HN · 61 comments · 99 points · Jul 18, 2021
Show HN: All Vision – Find dynamic patterns in data with no code↗ all.vision · HN · 7 comments · 12 points · Jul 14, 2021
If you want to transform IT, start with finance↗ zwischenzugs.com · HN · 150 comments · 293 points · Jul 13, 2021
Jul 05, 202143 stories
Building a data team at a mid-stage startup↗ erikbern.com · HN · 89 comments · 607 points · Jul 08, 2021
Datalore enterprise – Jupyter environment for data science teams↗ blog.jetbrains.com · HN · 42 comments · 89 points · Jul 06, 2021
The Rise and Fall of the OLAP Cube (2020)↗ holistics.io · HN · 91 comments · 211 points · Jul 05, 2021
Jun 28, 202138 stories
Tackling Technical Debt, Big and Small, in Wikidata and Wikibase↗ addshore.com · HN · 0 comments · 6 points · Jul 03, 2021
Hiring Data Scientists and Machine Learning Engineers↗ leanpub.com · HN · 4 comments · 11 points · Jun 30, 2021
Jun 21, 202140 stories
Ask HN: How do you run long computations without interruption?↗ news.ycombinator.com · HN · 14 comments · 6 points · Jun 26, 2021
Tired of creating test data by hand, we've built an open source data generator↗ getsynth.com · HN · 3 comments · 14 points · Jun 22, 2021
Jun 14, 202142 stories
Ask HN: Best practices for organizing geospatial data from different sources?↗ news.ycombinator.com · HN · 7 comments · 12 points · Jun 20, 2021
Notebooks are for data science, not only data scientists↗ medium.com · HN · 0 comments · 13 points · Jun 17, 2021
Jun 07, 202142 stories
Wildland – open data management protocol↗ wildland.io · HN · 47 comments · 128 points · Jun 11, 2021
Show HN: A flow based data processing editor↗ datablocks.pro · HN · 20 comments · 188 points · Jun 10, 2021
Perl Data Language: Scientific Computing with Perl↗ pdl.perl.org · HN · 108 comments · 144 points · Jun 08, 2021
May 31, 202133 stories
Scalable metadata: the new breed of file systems (em)powering big data companies↗ logicalclocks.com · HN · 0 comments · 11 points · Jun 02, 2021
May 24, 202140 stories
Data isn’t oil, whatever tech commentators tell you: it’s people’s lives↗ theguardian.com · HN · 24 comments · 80 points · May 30, 2021
May 17, 202143 stories
How to build a custom data connector in 2 hours↗ airbyte.io · HN · 32 comments · 99 points · May 19, 2021
Launch HN: Mozart Data (YC S20) – One-stop shop for a modern data pipeline↗ news.ycombinator.com · HN · 37 comments · 106 points · May 17, 2021
May 10, 202161 stories
Good data scientist, bad data scientist↗ ianwhitestone.work · HN · 91 comments · 159 points · May 11, 2021
Game theory as an engine for large-scale data analysis↗ deepmind.com · HN · 14 comments · 203 points · May 10, 2021
May 03, 202143 stories
Game theory as an engine for large-scale data analysis↗ deepmind.com · HN · 0 comments · 8 points · May 06, 2021
Apache Druid Turns 10: The Untold Origin Story↗ medriscoll.com · HN · 1 comments · 9 points · May 03, 2021
Apr 26, 202134 stories
Practical advice for analysis of large, complex data sets (2016)↗ unofficialgoogledatascience.com · HN · 22 comments · 109 points · Apr 30, 2021
Show HN: Spine automatically imports data from DBs or tools in Google Sheets↗ spine.dev · HN · 2 comments · 14 points · Apr 28, 2021
Beating Spark with a Better Map Reduce↗ vladfeinberg.com · HN · 1 comments · 24 points · Apr 26, 2021
Apr 19, 202142 stories
Data Curation Tools for Computer Vision in 2021↗ siasearch.io · HN · 1 comments · 13 points · Apr 23, 2021
The Data Engineering Study Guide↗ betterprogramming.pub · HN · 1 comments · 15 points · Apr 22, 2021
Apr 12, 202144 stories
What’s the origin of the phrase “big data doesn’t fit in excel”?↗ shkspr.mobi · HN · 74 comments · 125 points · Apr 17, 2021
Show HN: Personal Data Management Dashboard↗ volmarg.github.io · HN · 53 comments · 176 points · Apr 12, 2021
Apr 05, 202150 stories
Version Controlled Data Pipelines: Metaflow and Dolt↗ dolthub.medium.com · HN · 5 comments · 25 points · Apr 08, 2021
NoSQL Data Modeling Techniques (2012)↗ highlyscalable.wordpress.com · HN · 13 comments · 127 points · Apr 08, 2021
Mar 29, 202131 stories
Ask HN: What are the best data science bootcamps?↗ news.ycombinator.com · HN · 56 comments · 70 points · Mar 31, 2021
Everyone wants to do the model work, not the data work [pdf]↗ storage.googleapis.com · HN · 112 comments · 358 points · Mar 29, 2021
Mar 22, 202147 stories
Data Mesh: The Answer to the Data Warehouse Hypocrisy↗ blog.starburstdata.com · HN · 4 comments · 25 points · Mar 25, 2021
Show HN: Arc, an open-source Databricks alternative↗ arc.tripl.ai · HN · 36 comments · 175 points · Mar 24, 2021
Mar 15, 202140 stories
Data Science in Julia for Hackers↗ datasciencejuliahackers.com · HN · 51 comments · 240 points · Mar 19, 2021
Finding a standard dataset format for machine learning (2020)↗ openml.github.io · HN · 17 comments · 54 points · Mar 15, 2021
Mar 08, 202150 stories
Build your own “data lake” for reporting purposes↗ tech.fretlink.com · HN · 51 comments · 143 points · Mar 13, 2021
A Hybrid Apache Arrow/Numpy DataFrame with Vaex Version 4.0↗ vaex.io · HN · 16 comments · 58 points · Mar 09, 2021
Mar 01, 202150 stories
OmniSci launches free edition of platform for interactive visual analytics↗ omnisci.com · HN · 5 comments · 34 points · Mar 03, 2021
Feb 22, 202145 stories
Show HN: I wrote a book about using data science to solve “everyday” problems↗ andrewnc.github.io · HN · 96 comments · 632 points · Feb 24, 2021
Feb 15, 202154 stories
Launch HN: MindsDB (YC W20) – Machine Learning Inside Your Database↗ news.ycombinator.com · HN · 60 comments · 176 points · Feb 19, 2021
Feb 08, 202149 stories
Why Databricks Is Winning↗ cloudnativeenterprise.substack.com · HN · 75 comments · 85 points · Feb 14, 2021
The Atlas for the Aspiring Network Scientist v1.1↗ michelecoscia.com · HN · 0 comments · 17 points · Feb 09, 2021
Laver's Law Revisited: A Data-Driven Investigation of Fashion Trends↗ laverslaw.awardwinninghuman.com · HN · 4 comments · 21 points · Feb 08, 2021
Feb 01, 202155 stories
Weave.jl – Scientific Reports Using Julia↗ weavejl.mpastell.com · HN · 17 comments · 127 points · Feb 04, 2021
Solid Project: All of your data, under your control↗ solidproject.org · HN · 118 comments · 350 points · Feb 01, 2021
Jan 25, 202148 stories
Building a personal data warehouse in Snowflake for fun and no profit↗ thomasdziedzic0.com · HN · 31 comments · 166 points · Jan 31, 2021
Launch HN: Airbyte (YC W20) – Open-Source ELT (Fivetran/Stitch Alternative)↗ news.ycombinator.com · HN · 87 comments · 178 points · Jan 26, 2021
Jan 18, 202148 stories
Our infrastructure is being built based on past climate data↗ vice.com · HN · 44 comments · 107 points · Jan 24, 2021
Tree – a lib for working with nested data structures, open-sourced by deepmind↗ github.com · HN · 14 comments · 127 points · Jan 23, 2021
Why isn't differential dataflow more popular?↗ scattered-thoughts.net · HN · 120 comments · 228 points · Jan 22, 2021
Jan 11, 202149 stories
Elasticsearch and Kibana are now business risks↗ anonymoushash.vmbrasseur.com · HN · 22 comments · 94 points · Jan 14, 2021
We don't need data scientists, we need data engineers↗ mihaileric.com · HN · 353 comments · 718 points · Jan 14, 2021
How to Become a Data Engineer in 2021↗ khashtamov.com · HN · 137 comments · 264 points · Jan 11, 2021
Jan 04, 202140 stories
Large scale Industrial IoT data project: lessons learned in 2020↗ blog.quasardb.net · HN · 0 comments · 4 points · Jan 08, 2021
Show HN: Mljar Automated Machine Learning for Tabular Data (Explanation,AutoDoc)↗ github.com · HN · 8 comments · 33 points · Jan 05, 2021
Dec 28, 202033 stories
Idyll – toolkit for creating data-driven stories and explorable explanations↗ idyll-lang.org · HN · 8 comments · 168 points · Jan 02, 2021
The Elements of Statistical Learning [pdf]↗ web.stanford.edu · HN · 59 comments · 282 points · Dec 31, 2020
Dec 21, 202038 stories
Datasette.io, an official project website for Datasette↗ simonwillison.net · HN · 29 comments · 208 points · Dec 24, 2020
Stan is a state-of-the-art platform for statistical modeling↗ mc-stan.org · HN · 72 comments · 191 points · Dec 23, 2020
Dec 14, 202070 stories
An Overview of End-to-End Entity Resolution for Big Data↗ blog.acolyer.org · HN · 2 comments · 58 points · Dec 17, 2020
Dec 07, 202071 stories
How to Reliably Scale Your Data Platform for High Volumes↗ shopify.engineering · HN · 4 comments · 61 points · Dec 12, 2020
Datasette: An open source multi-tool for exploring and publishing data↗ datasette.io · HN · 22 comments · 285 points · Dec 11, 2020
Building a scalable e-commerce data model↗ resources.fabric.inc · HN · 25 comments · 129 points · Dec 08, 2020
Nov 30, 202058 stories
Kaitai: Describe the structure of data, not how you read or write it↗ kaitai.io · HN · 65 comments · 245 points · Dec 05, 2020
Amnesia – High-Accuracy Data Anonymization↗ amnesia.openaire.eu · HN · 96 comments · 150 points · Dec 02, 2020
Nov 23, 202055 stories
The path to learning SQL and mastering it to become a Data Engineer↗ medium.com · HN · 1 comments · 21 points · Nov 28, 2020
CrateDB: Purpose-built to scale modern applications in a machine data world↗ crate.io · HN · 13 comments · 37 points · Nov 27, 2020
Show HN: 1000 GitHub Data Science projects, now executable in 1 click↗ cloud.blobcity.com · HN · 32 comments · 125 points · Nov 27, 2020
Nov 16, 202064 stories
A brief history of enterprise data challenges↗ hassenchaieb.com · HN · 2 comments · 23 points · Nov 17, 2020
Show HN: Covid-19 Time Series Workbench (OWID Data)↗ info.qaware.de · HN · 3 comments · 18 points · Nov 16, 2020
Nov 09, 202072 stories
Ask HN: Am I too late for the “Data Science” wave?↗ news.ycombinator.com · HN · 143 comments · 199 points · Nov 15, 2020
Personal Data Warehouses: Reclaiming Your Data↗ simonwillison.net · HN · 59 comments · 436 points · Nov 14, 2020
AWS releases Glue Databrew, a visual ETL tool↗ aws.amazon.com · HN · 70 comments · 169 points · Nov 12, 2020
Nov 02, 202042 stories
Pro Rata and User Centric Distribution Models: A Comparative Study (2017)↗ digitalmedia.fi · HN · 86 comments · 84 points · Nov 08, 2020
Show HN: Deepnote – a real-time collaborative data science notebook in the cloud↗ deepnote.com · HN · 4 comments · 26 points · Nov 07, 2020
CSVs: The Good, the Bad, and the Ugly↗ alexgaynor.net · HN · 104 comments · 89 points · Nov 07, 2020
Oct 26, 202054 stories
Launch HN: Deepnote (YC S19) – A better data science notebook↗ news.ycombinator.com · HN · 70 comments · 316 points · Oct 30, 2020
Blueprints, making it easy to anonymize and balance datasets with a few clicks↗ gretel.ai · HN · 1 comments · 8 points · Oct 27, 2020
Oct 19, 202058 stories
Disney Animation data sets (2018)↗ blog.yiningkarlli.com · HN · 2 comments · 67 points · Oct 20, 2020
Oct 12, 202062 stories
Data engineering learning path with recommended resources↗ awesomedataengineering.com · HN · 26 comments · 207 points · Oct 18, 2020
Emerging Architectures for Modern Data Infrastructure↗ a16z.com · HN · 105 comments · 431 points · Oct 18, 2020
Ask HN: What are good resources to learn system design?↗ news.ycombinator.com · HN · 65 comments · 246 points · Oct 13, 2020
Oct 05, 202054 stories
Dendrite (next-gen Matrix server) is entering beta↗ matrix.org · HN · 11 comments · 65 points · Oct 08, 2020
Machine Learning Engineer Guide: Feature Store vs. Data Warehouse↗ logicalclocks.com · HN · 56 comments · 166 points · Oct 08, 2020
Project Cambria: Translate your data with lenses↗ inkandswitch.com · HN · 26 comments · 101 points · Oct 06, 2020
Sep 28, 202056 stories
Launch HN: Narrator (YC S19) – a data modeling platform built on a single table↗ news.ycombinator.com · HN · 59 comments · 143 points · Sep 30, 2020
Numenta Platform for Intelligent Computing↗ github.com · HN · 23 comments · 82 points · Sep 28, 2020
Sep 21, 202060 stories
New Schema.org support for retailer shipping data↗ webmasters.googleblog.com · HN · 32 comments · 88 points · Sep 22, 2020
Sep 14, 202062 stories
Sep 07, 202046 stories
AresDB: Uber's GPU-Powered Open Source, Real-Time Analytics Engine↗ ubere.ng · HN · 83 comments · 191 points · Sep 13, 2020
Data science interview questions with answers↗ github.com · HN · 71 comments · 242 points · Sep 13, 2020
Ask HN: Resources to teach myself Statistics?↗ news.ycombinator.com · HN · 12 comments · 22 points · Sep 10, 2020
Aug 31, 202062 stories
Pachyderm Hub: data science without the hassle of managing infrastructure↗ pachyderm.com · HN · 31 comments · 107 points · Sep 03, 2020
Exalting Data, Missing Meaning (2017)↗ blog.andymatuschak.org · HN · 7 comments · 19 points · Sep 03, 2020
Designing a Security Data Lake on AWS and Snowflake↗ blog.runpanther.io · HN · 0 comments · 10 points · Sep 01, 2020
Aug 24, 202047 stories
Ask HN: How does your data science or machine learning team handle DevOps?↗ news.ycombinator.com · HN · 41 comments · 86 points · Aug 30, 2020
Problem Solving with Algorithms and Data Structures [pdf]↗ cs.auckland.ac.nz · HN · 17 comments · 180 points · Aug 26, 2020
Talk to Your Data: One Model, Any Relational Database↗ blog.einstein.ai · HN · 18 comments · 67 points · Aug 26, 2020
Aug 17, 202054 stories
The Splitgraph Data Delivery Network – query over 40k public datasets↗ splitgraph.com · HN · 95 comments · 297 points · Aug 21, 2020
Launch HN: Hubble (YC S20) – Monitor data quality inside data warehouses↗ news.ycombinator.com · HN · 42 comments · 125 points · Aug 20, 2020
Aug 10, 202060 stories
Metaplane – A metadata catalog for modern data teams↗ metaplane.dev · HN · 20 comments · 81 points · Aug 13, 2020
Show HN: dstack – an open-source tool to build data applications easily↗ news.ycombinator.com · HN · 29 comments · 134 points · Aug 12, 2020
Show HN: Orchest – Data Science Pipelines↗ news.ycombinator.com · HN · 30 comments · 64 points · Aug 12, 2020
Aug 03, 202051 stories
Launch HN: Datafold (YC S20) – Diff Tool for SQL Databases↗ news.ycombinator.com · HN · 76 comments · 189 points · Aug 06, 2020
Simdjson: Parsing gigabytes of JSON per second↗ github.com · HN · 151 comments · 503 points · Aug 06, 2020
Jul 27, 202062 stories
Data isn’t just being collected from your phone. It’s being used to score you↗ washingtonpost.com · HN · 4 comments · 28 points · Aug 01, 2020
You Want to See My Data? I Thought We Were Friends↗ nautil.us · HN · 134 comments · 489 points · Jul 30, 2020
Jul 20, 202059 stories
Best Data Science Books According to the Experts↗ builtin.com · HN · 60 comments · 284 points · Jul 24, 2020
Jul 13, 202062 stories
Data structures and algorithms I actually used while working at tech companies↗ blog.pragmaticengineer.com · HN · 524 comments · 1025 points · Jul 15, 2020
Launch HN: Aquarium (YC S20) – Improve Your ML Dataset Quality↗ news.ycombinator.com · HN · 19 comments · 167 points · Jul 13, 2020
Jul 06, 202054 stories
A gentle introduction to HDBSCAN and density-based clustering↗ towardsdatascience.com · HN · 9 comments · 55 points · Jul 12, 2020
Jun 29, 202045 stories
The Triplebyte data download doesn’t give you all your data↗ linestarve.com · HN · 0 comments · 86 points · Jul 05, 2020
Scaling Pandas: Comparing Dask, Ray, Modin, Vaex, and Rapids↗ datarevenue.com · HN · 33 comments · 92 points · Jul 05, 2020
An Introduction to Conflict-Free Replicated Data Types↗ lars.hupel.info · HN · 13 comments · 165 points · Jul 05, 2020
Jun 22, 202052 stories
Ask HN: What Is the Value of CockroachDB?↗ news.ycombinator.com · HN · 29 comments · 47 points · Jun 24, 2020
Show HN: DVC 1.0 release. 5 lessons from 3 years of building open-source ML tool↗ news.ycombinator.com · HN · 4 comments · 19 points · Jun 22, 2020
Jun 15, 202069 stories
Hash: A free, online platform for modeling↗ joelonsoftware.com · HN · 59 comments · 208 points · Jun 18, 2020
GoPlus – The Go+ language for data science↗ github.com · HN · 61 comments · 143 points · Jun 17, 2020
Jun 08, 202060 stories
Show HN: We wrote a book to help data people build scalable analytics stacks↗ holistics.io · HN · 16 comments · 33 points · Jun 12, 2020
Large-Scale Transactional Data Lake at Uber Using Apache Hudi↗ eng.uber.com · HN · 31 comments · 66 points · Jun 10, 2020
(CMU talk) Building Materialize: Streaming Database powered by Timely Dataflow↗ materialize.io · HN · 0 comments · 12 points · Jun 08, 2020
Jun 01, 202054 stories
PANDAcap: A framework for streamlining the capture of PANDA execution traces↗ github.com · HN · 1 comments · 15 points · Jun 06, 2020
Ask HN: What do I need to learn how to write a scalable app?↗ news.ycombinator.com · HN · 17 comments · 28 points · Jun 02, 2020
May 25, 202044 stories
Beyond Pandas: Spark, Dask, Vaex and other big data technologies↗ medium.com · HN · 2 comments · 15 points · May 30, 2020
Analysis of how Covid-19 impacted product analytics usage↗ amplitude.com · HN · 0 comments · 16 points · May 27, 2020
May 18, 202063 stories
May 11, 202061 stories
Faster and more flexible pipelines with a Directed Acyclic Graph↗ about.gitlab.com · HN · 38 comments · 73 points · May 12, 2020
Launch HN: Data Mechanics (YC S19) – The Simplest Way to Run Apache Spark↗ news.ycombinator.com · HN · 42 comments · 131 points · May 11, 2020
May 04, 202072 stories
Bayesian Data Analysis, Third Edition [pdf]↗ users.aalto.fi · HN · 48 comments · 466 points · May 06, 2020
Apr 27, 202054 stories
Show HN: Census – Export for data warehouses↗ news.ycombinator.com · HN · 7 comments · 34 points · Apr 30, 2020
DataJet – Fully managed Apache NiFi as a service. Visual Data flows without code↗ datajet.dev · HN · 23 comments · 60 points · Apr 29, 2020
Apr 20, 202072 stories
Galaxy Brain: Data Science Workflows with Prefect and Dask↗ symphonyrm.com · HN · 0 comments · 14 points · Apr 22, 2020
Show HN: ProphecyHub: Metadata Re-Invented with Git,GraphQL for Data Engineering↗ medium.com · HN · 0 comments · 22 points · Apr 22, 2020
Causal Inference and Data-Fusion in Econometrics (2019)↗ arxiv.org · HN · 14 comments · 61 points · Apr 20, 2020
Apr 13, 202077 stories
Visualizing technological inventions using text mining↗ blog.researchly.app · HN · 1 comments · 27 points · Apr 19, 2020
Statistical process control after W. Edwards Deming↗ 2uo.de · HN · 8 comments · 53 points · Apr 18, 2020
DustMite: A General-Purpose Data Reduction Tool↗ dlang.org · HN · 6 comments · 66 points · Apr 13, 2020
Apr 06, 202074 stories
Timely Dataflow in Fifteen Minutes [video]↗ youtube.com · HN · 17 comments · 136 points · Apr 09, 2020
Data Science: Reality Doesn't Meet Expectations↗ dfrieds.com · HN · 163 comments · 462 points · Apr 07, 2020
Mar 30, 202052 stories
Data Structures and Algorithms Modules Collection↗ opendsa-server.cs.vt.edu · HN · 2 comments · 40 points · Apr 04, 2020
Why Vertica Customers Adopt Apache Druid for Real-Time Analytics↗ imply.io · HN · 11 comments · 31 points · Mar 31, 2020
Mar 23, 202054 stories
Secure Data Discovery for ElasticSearch and Apache Kafka with SQL↗ lenses.io · HN · 0 comments · 31 points · Mar 27, 2020
Emerging jobs and the green economy are boosting demand for analytical skills↗ pewresearch.org · HN · 1 comments · 31 points · Mar 23, 2020
This month we've learned: fzf, Datadog and Teraform↗ blog.softwaremill.com · HN · 0 comments · 14 points · Mar 23, 2020
Mar 16, 202066 stories
OpenRefine – free, open source, powerful tool for working with messy data↗ openrefine.org · HN · 23 comments · 235 points · Mar 22, 2020
Mar 09, 202071 stories
How Netflix uses Druid for realtime insights↗ netflixtechblog.com · HN · 59 comments · 104 points · Mar 14, 2020
Launch HN: Castodia (YC W20) – Populate Google Sheets from databases↗ news.ycombinator.com · HN · 51 comments · 103 points · Mar 13, 2020
Mar 02, 202076 stories
Show HN: UniversalDataTool – open-source collaborative data labeling↗ github.com · HN · 10 comments · 70 points · Mar 08, 2020
Launch HN: Datasaur (YC W20) – data labeling interface for NLP↗ news.ycombinator.com · HN · 62 comments · 174 points · Mar 06, 2020
Feb 24, 202067 stories
Aleph: A suite of data analysis tools for investigators↗ docs.alephdata.org · HN · 21 comments · 240 points · Mar 01, 2020
How I Did It: From Psych Degree to VP of Data Science at Top Startup↗ eugeneyan.com · HN · 76 comments · 153 points · Mar 01, 2020
Feb 17, 202071 stories
Materialize: A Streaming Data Warehouse↗ materialize.io · HN · 100 comments · 262 points · Feb 18, 2020
Feb 10, 202080 stories
Data USA: explore, map, compare, and download U.S. data↗ datausa.io · HN · 11 comments · 112 points · Feb 16, 2020
Applied Thinking for Intelligence Analysis: A Guide for Practitioners (2014) [pdf]↗ airpower.airforce.gov.au · HN · 5 comments · 145 points · Feb 11, 2020
Is a Dataframe Just a Table? (2019) [pdf]↗ plateau-workshop.org · HN · 113 comments · 193 points · Feb 10, 2020
Feb 03, 202052 stories
Using ‘radioactive data’ to detect if a data set was used for training↗ ai.facebook.com · HN · 26 comments · 162 points · Feb 05, 2020
Jan 27, 202057 stories
What to do when you don’t trust your data anymore↗ laskowskilab.faculty.ucdavis.edu · HN · 20 comments · 89 points · Jan 30, 2020
Jan 20, 202073 stories
Understanding the Power of Data Types↗ postgres-data-types.pvh.ca · HN · 9 comments · 87 points · Jan 26, 2020
Discovering Millions of Datasets on the Web↗ blog.google · HN · 32 comments · 443 points · Jan 23, 2020
Testing Dataflow Analyses for Precision and Soundness↗ blog.regehr.org · HN · 2 comments · 38 points · Jan 20, 2020
Jan 13, 202065 stories
Ask HN: What tools do you use for data munging and merging?↗ news.ycombinator.com · HN · 72 comments · 114 points · Jan 13, 2020
Jan 06, 202063 stories
Big Prometheus: Thanos, Cortex, M3DB and VictoriaMetrics at Scale↗ monitoring2.substack.com · HN · 21 comments · 65 points · Jan 08, 2020
Copying data is wasteful, mutating data is dangerous↗ pythonspeed.com · HN · 153 comments · 230 points · Jan 08, 2020
Dec 30, 201927 stories
Show HN: Self-Published Book – “Data Science in Production”↗ news.ycombinator.com · HN · 18 comments · 172 points · Jan 04, 2020
No. 10 Downing Street hiring data scientists, physicists, weirdos↗ dominiccummings.com · HN · 34 comments · 84 points · Jan 02, 2020
Scroll through time ↕
The pace of conversation
Posts per month
2020-012026-11
Posts per month · first and latest months may be partial.
04 / GO DEEPER
Follow your curiosity
Find a particular story, revisit a year, or see what resonated.
Highest rated · the last 30 days
The Dawn of the Data Intelligence Engineer↗ motherduck.com · HN · 78 comments · 160 points · Nov 13, 2026
The U.S. Census Bureau and "Super Intelligence"↗ simonwillison.net · HN · 54 comments · 127 points · Nov 02, 2026
Keplearn: Extract Equations from Data in Seconds↗ github.com · HN · 19 comments · 108 points · Oct 29, 2026
MiniLake Is a Local Databricks API Emulator with SQL, Delta Lake and Spark↗ dmux.github.io · HN · 25 comments · 101 points · Oct 27, 2026
USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance↗ arxiv.org · HN · 8 comments · 49 points · Nov 03, 2026
From Spreadsheets to DuckLake: A Data Engineer's Road to DuckDB↗ duckdb.org · HN · 0 comments · 22 points · Oct 29, 2026
10 stories shown.