From c50d21291f5ebc867ee05eeddf238b046b734f49 Mon Sep 17 00:00:00 2001 From: Gabor Szarnyas Date: Mon, 10 Mar 2025 10:39:17 +0100 Subject: [PATCH 1/6] Nit --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index ba2f119..1a22ef3 100644 --- a/README.md +++ b/README.md @@ -120,7 +120,7 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [UniverSQL](https://github.com/buremba/universql) - An implementation of Snowflake API, enables running queries on Snowflake tables locally with DuckDB without a running warehouse. - [Whereabouts](https://github.com/ajl2718/whereabouts) - Fast, accurate, open-source geocoding in Python, using DuckDB. - [Phoenix Analytics](https://github.com/lalabuy948/PhoenixAnalytics) - Plug and play analytics for Phoenix applications, powered by DuckDB. -- [sqlglot](https://github.com/tobymao/sqlglot) - Python transpiler that translates between 24 different SQL dialects including DuckDB. +- [SQLGlot](https://github.com/tobymao/sqlglot) - Python transpiler that translates between 24 different SQL dialects including DuckDB. - [yato](https://github.com/Bl3f/yato) - The smallest DuckDB SQL orchestrator on Earth. - [SQLMesh](https://github.com/TobikoData/sqlmesh) - A next-generation data transformation and modeling framework with support for DuckDB connections for state, transformations & running unit tests locally. - [ADPivot](https://github.com/danilo-css/analytics-data-pivot) - No code tool built on top of DuckDB-Wasm and Pyodide that helps build pivot tables from databases of any size with a few clicks. From e0a35d22ece48114e3616bded2b09eee7f832d3a Mon Sep 17 00:00:00 2001 From: Gabor Szarnyas Date: Thu, 20 Mar 2025 11:58:33 +0100 Subject: [PATCH 2/6] Remove outdated tools --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 1a22ef3..fb3479b 100644 --- a/README.md +++ b/README.md @@ -131,10 +131,8 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [Online DuckDB Shell](https://shell.duckdb.org/) - Online DuckDB shell powered by DuckDB-Wasm. - [SQL Workbench](https://sql-workbench.com) - DuckDB-Wasm based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs. - [Sekuel Playground](https://sekuel.com/playground/) - Query your local Parquet, CSV, JSON. Your data will not be sent out of the device you are using. -- [CSVFiddle](https://csvfiddle.io/) - Free tool to explore and share insights from CSV files using SQL. Import data, write SQL, then instantly share it with anyone. - [Codapi](https://codapi.org/duckdb/) - Embed executable code snippets directly into your product documentation, online course or blog post. - [QuackDB](https://quackdb.com/) - Open-source online DuckDB SQL playground and editor. -- [WhatTheDuck](https://whattheduck.incentius.com/) - WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data. - [Sidequery](https://sidequery.ai) - Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data. ### Backends From 5468c6edf74ad290480a9f4bcab96e8ad13d801c Mon Sep 17 00:00:00 2001 From: Gabor Szarnyas Date: Thu, 20 Mar 2025 12:04:29 +0100 Subject: [PATCH 3/6] Move Duck-UI to Wasm clients --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index fb3479b..846b42b 100644 --- a/README.md +++ b/README.md @@ -124,7 +124,6 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [yato](https://github.com/Bl3f/yato) - The smallest DuckDB SQL orchestrator on Earth. - [SQLMesh](https://github.com/TobikoData/sqlmesh) - A next-generation data transformation and modeling framework with support for DuckDB connections for state, transformations & running unit tests locally. - [ADPivot](https://github.com/danilo-css/analytics-data-pivot) - No code tool built on top of DuckDB-Wasm and Pyodide that helps build pivot tables from databases of any size with a few clicks. -- [Duck-UI](https://demo.duckui.com/) - Duck-UI is a web-based interface for interacting with DuckDB with a SQL editor, data import/export, data explorer, query history, theme toggle and keyboard shortcuts. ### Web Clients (WebAssembly) @@ -134,6 +133,7 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [Codapi](https://codapi.org/duckdb/) - Embed executable code snippets directly into your product documentation, online course or blog post. - [QuackDB](https://quackdb.com/) - Open-source online DuckDB SQL playground and editor. - [Sidequery](https://sidequery.ai) - Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data. +- [Duck-UI](https://demo.duckui.com/) - Duck-UI is a web-based interface for interacting with DuckDB with a SQL editor, data import/export, data explorer, query history, theme toggle and keyboard shortcuts. ### Backends From db802bb792175618fe7300ecc05fa31467531f4d Mon Sep 17 00:00:00 2001 From: Gabor Szarnyas Date: Thu, 20 Mar 2025 12:52:29 +0100 Subject: [PATCH 4/6] Fix wording --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 846b42b..d9476cc 100644 --- a/README.md +++ b/README.md @@ -151,7 +151,7 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [@jetblack/duckdb-react](https://github.com/rob-blackbourn/jetblack-duckdb-react) - A context manager for React and DuckDB-Wasm. - [QuackOSM](https://github.com/kraina-ai/quackosm) - A Python library for downloading and transforming raw OpenStreetMap data into GeoParquet files. - [PyGWalker](https://kanaries.net/pygwalker) - A Python library that turns your dataframe into an interactive UI for data visualization. -- [flapi](https://github.com/DataZooDE/flapi) - API Framework heavily relies on the power of DuckDB and DuckDB extensions. Ready to build performant and cost-efficient APIs on top of BigQuery or Snowflake for AI Agents and Data Apps. +- [flapi](https://github.com/DataZooDE/flapi) - An API Framework that heavily relies on the power of DuckDB and DuckDB extensions. Ready to build performant and cost-efficient APIs on top of BigQuery or Snowflake for AI Agents and Data Apps. - [smallpond](https://github.com/deepseek-ai/smallpond) - A distributed data processing framework by DeepSeek built on DuckDB and 3FS. - [BemiDB](https://github.com/BemiHQ/BemiDB) - PostgreSQL read replica optimized for analytics, using DuckDB. From f64fb2c4b5af8e9fa916ca6548e35e84aa0b634a Mon Sep 17 00:00:00 2001 From: Gabor Szarnyas Date: Thu, 20 Mar 2025 13:53:56 +0100 Subject: [PATCH 5/6] Move Avro extension from community to core --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d9476cc..cd8a283 100644 --- a/README.md +++ b/README.md @@ -220,6 +220,7 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist Official DuckDB extensions, which can installed via `INSTALL ⟨extension_name⟩`. - [`arrow`](https://github.com/duckdb/arrow) - A zero-copy data integration between Apache Arrow and DuckDB. +- [`avro`](https://github.com/duckdb/duckdb-avro) - For reading Avro files. - [`aws`](https://github.com/duckdb/duckdb_aws) - For handling AWS credentials. - [`azure`](https://github.com/duckdb/duckdb_azure) - For using the Azure Blob storage. - [`delta`](https://github.com/duckdb/duckdb_delta) - For Delta Lake support. @@ -236,7 +237,6 @@ Official DuckDB extensions, which can installed via `INSTALL ⟨extension_name Community-contributed DuckDB extensions, which can be installed via `INSTALL ⟨extension_name⟩ FROM community`. -- [`avro`](https://github.com/hannes/duckdb_avro) - Enables reading Avro files. - [`bigquery`](https://github.com/hafenkran/duckdb-bigquery) - Integrates DuckDB with Google BigQuery, allowing direct querying and management of BigQuery datasets. - [`chsql`](https://github.com/lmangani/duckdb-extension-clickhouse-sql) - ClickHouse SQL Dialect macros for DuckDB. - [`crypto`](https://github.com/rustyconover/duckdb-crypto-extension) - Cryptographic hash functions and HMAC. From c33a3b6095abf0efa9a80b385a1419136116ecc4 Mon Sep 17 00:00:00 2001 From: Gabor Szarnyas Date: Thu, 20 Mar 2025 13:55:53 +0100 Subject: [PATCH 6/6] Shuffle around clients --- README.md | 39 +++++++++++++++++++++------------------ 1 file changed, 21 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index cd8a283..24ad661 100644 --- a/README.md +++ b/README.md @@ -125,19 +125,9 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [SQLMesh](https://github.com/TobikoData/sqlmesh) - A next-generation data transformation and modeling framework with support for DuckDB connections for state, transformations & running unit tests locally. - [ADPivot](https://github.com/danilo-css/analytics-data-pivot) - No code tool built on top of DuckDB-Wasm and Pyodide that helps build pivot tables from databases of any size with a few clicks. -### Web Clients (WebAssembly) +## Backends -- [Online DuckDB Shell](https://shell.duckdb.org/) - Online DuckDB shell powered by DuckDB-Wasm. -- [SQL Workbench](https://sql-workbench.com) - DuckDB-Wasm based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs. -- [Sekuel Playground](https://sekuel.com/playground/) - Query your local Parquet, CSV, JSON. Your data will not be sent out of the device you are using. -- [Codapi](https://codapi.org/duckdb/) - Embed executable code snippets directly into your product documentation, online course or blog post. -- [QuackDB](https://quackdb.com/) - Open-source online DuckDB SQL playground and editor. -- [Sidequery](https://sidequery.ai) - Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data. -- [Duck-UI](https://demo.duckui.com/) - Duck-UI is a web-based interface for interacting with DuckDB with a SQL editor, data import/export, data explorer, query history, theme toggle and keyboard shortcuts. - -### Backends - -- [DuckDB API](https://github.com/tobilg/duckdb-api) - a TypeScript-based Docker image containing DuckDB, and a Hono framework REST API with JSON or streaming Arrow responses. +- [DuckDB API](https://github.com/tobilg/duckdb-api) - A TypeScript-based Docker image containing DuckDB, and a Hono framework REST API with JSON or streaming Arrow responses. - [Mosaic DuckDB Server](https://pypi.org/project/duckdb-server/) - A Python-based server that runs a local DuckDB instance and supports queries over Web Sockets or HTTP, returning data in either Apache Arrow or JSON format. - [duckdb-server](https://lib.rs/crates/duckdb-server) - A Rust-based server that runs a local DuckDB instance and supports queries over Web Sockets or HTTP/HTTPS, returning data in either Apache Arrow or JSON format. @@ -154,23 +144,32 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [flapi](https://github.com/DataZooDE/flapi) - An API Framework that heavily relies on the power of DuckDB and DuckDB extensions. Ready to build performant and cost-efficient APIs on top of BigQuery or Snowflake for AI Agents and Data Apps. - [smallpond](https://github.com/deepseek-ai/smallpond) - A distributed data processing framework by DeepSeek built on DuckDB and 3FS. - [BemiDB](https://github.com/BemiHQ/BemiDB) - PostgreSQL read replica optimized for analytics, using DuckDB. +- [jsqltranspiler](https://github.com/starlake-ai/jsqltranspiler) - Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB-compatible SQL. + +## DuckDB Clients and UIs + +### Web Clients (WebAssembly) + +- [Online DuckDB Shell](https://shell.duckdb.org/) - Online DuckDB shell powered by DuckDB-Wasm. +- [SQL Workbench](https://sql-workbench.com) - DuckDB-Wasm based SQL Workbench for running queries on local or remote data, being able to show data as tables or visually as graphs, and sharing queries via URLs. +- [Sekuel Playground](https://sekuel.com/playground/) - Query your local Parquet, CSV, JSON. Your data will not be sent out of the device you are using. +- [Codapi](https://codapi.org/duckdb/) - Embed executable code snippets directly into your product documentation, online course or blog post. +- [QuackDB](https://quackdb.com/) - Open-source online DuckDB SQL playground and editor. +- [Sidequery](https://sidequery.ai) - Sidequery is a privacy-preserving DuckDB-powered query editor & data exploration tool for local & remote data. +- [Duck-UI](https://demo.duckui.com/) - Duck-UI is a web-based interface for interacting with DuckDB with a SQL editor, data import/export, data explorer, query history, theme toggle and keyboard shortcuts. ## SQL Clients and IDE that Support DuckDB -- [Harlequin](https://harlequin.sh) - The DuckDB IDE for your terminal. +- [Harlequin](https://harlequin.sh) - The DuckDB IDE (TUI) for your terminal. - [qStudio](https://www.timestored.com/qstudio/) - A free SQL tool specialized for data analysts. It runs on every operating system and allows easy browsing of tables and charting of results. - [DuckDB SQL Tools](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.duckdb-sql-tools) - Free DuckDB SQL Tools for VS Code IDE. [Premium version available](https://github.com/RandomFractals/pro-data-tools/blob/main/duckdb-tools.md#duckdb-pro-tools) with advanced features. -- [VSCode SQLTools (Free)](https://marketplace.visualstudio.com/items?itemName=Evidence.sqltools-duckdb-driver) - Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support. +- [VSCode SQLTools (free)](https://marketplace.visualstudio.com/items?itemName=Evidence.sqltools-duckdb-driver) - Free open-source VSCode extension to query and explore your DuckDB databases with latest DuckDB support. - [DBeaver](https://dbeaver.com) - DBeaver is a universal database access and development tool that can be used to connect almost any type of database. - [DataGrip](https://www.jetbrains.com/datagrip/) - Paid SQL IDE by JetBrains that supports many different database technologies, including DuckDB. - [Duckling](https://github.com/l1xnan/duckling) - A fast viewer for CSV/Parquet files and DuckDB/SQLite, based on Tauri. - [rsql](https://github.com/theseus-rs/rsql) - CLI for DuckDB, LibSQL, MariaDB, MySQL, PostgreSQL, SQLite3 and SQL Server. -- [jsqltranspiler](https://github.com/starlake-ai/jsqltranspiler) - Rewrite BigQuery, Redshift, Snowflake and Databricks queries into DuckDB-compatible SQL. -- [jOOQ](https://www.jooq.org/) - Type safe querying of DuckDB (and many other RDBMS) from Java. A [transpiler](https://www.jooq.org/translate/) from and to DuckDB is also available. - [SQL DATA LENS](https://sqldatalens.com/build-in-support-for-duckdb/) - A lightweight, commercial SQL IDE that supports different DBMS, including DuckDB. The focus is on performance and special DBMS features. - [Dataflare](https://dataflare.app) - Simple easy-to-use database manager, supports DuckDB, PostgreSQL, MySQL, SQL Server, SQLite etc. -- [manifold-sql (DuckDB for Java)](https://github.com/manifold-systems/manifold/blob/master/docs/articles/duckdb_info.md) - Use native DuckDB SQL of any complexity directly & type-safely in Java source with comprehensive IntelliJ support. -- [Termux DuckDB package](https://packages.termux.dev/apt/termux-main/pool/main/libd/libduckdb/) - DuckDB CLI client for the Termux Android terminal emulator. ## Projects Powered by DuckDB @@ -202,6 +201,8 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [xlDuckDb](https://github.com/RusselWebber/xlDuckDb) - Excel addin to run DuckDB queries in Excel. - [Hasura DuckDB Connector](https://github.com/hasura/ndc-duckdb) - Allows connecting to a DuckDB database or a MotherDuck-hosted DuckDB database through a GraphQL API. - [Ontop](https://ontop-vkg.org/guide/databases/duckdb.html) - Allows to create Virtual Knowledge Graphs directly from DuckDB. +- [jOOQ](https://www.jooq.org/) - Type safe querying of DuckDB (and many other RDBMS) from Java. A [transpiler](https://www.jooq.org/translate/) from and to DuckDB is also available. +- [manifold-sql (DuckDB for Java)](https://github.com/manifold-systems/manifold/blob/master/docs/articles/duckdb_info.md) - Use native DuckDB SQL of any complexity directly & type-safely in Java source with comprehensive IntelliJ support. ## Client-Server Setups @@ -213,6 +214,8 @@ You can chat with this page's content on [HuggingChat](https://hf.co/chat/assist - [pg_mooncake](https://github.com/Mooncake-Labs/pg_mooncake) - A PostgreSQL extension that adds native column store tables with DuckDB. - [SQLFlite - Arrow Flight SQL Server](https://github.com/voltrondata/sqlflite) - An example implementation of the [Arrow Flight SQL protocol](https://arrow.apache.org/docs/format/FlightSql.html) that runs in a client-server setup with DuckDB or SQLite as backends. +- [Termux DuckDB package](https://packages.termux.dev/apt/termux-main/pool/main/libd/libduckdb/) - DuckDB CLI client for the Termux Android terminal emulator. + ## Extensions ### [Official Extensions](https://duckdb.org/docs/extensions/official_extensions)