one of the more awesome things you can do with
@duckdb
query geospatial data straight from github
this is from the excellent Natural Earth Data, and contains the boundaries for all countries in the world
When I was a Chief of Staff, one meeting generated more stress than *all the others* in the week.
What was it?
Our Tuesday AM 📊Weekly Business Review📊
(1/x)
okay so how about this
@duckdb
's statistical functions have enough coverage that you can basically recreate facebook prophet in SQL
time series forecasting in SQL, here we come
it can be tricky to do forecasting in SQL, because SQL isn't really designed for operating on data that doesn't exist
in
@duckdb
it's simple, by combining generate_series() with unnest()
when you run a query in
@duckdb
, it outputs results in a nice box
but there are 20 other **really useful** formats you can use with .mode
here are 7 of my favourite
.mode duckbox (default)
one of the more awesome things you can do with
@duckdb
query geospatial data straight from github
this is from the excellent Natural Earth Data, and contains the boundaries for all countries in the world
underrated feature of
@evidence_dev
it creates a parquet catalog from your data sources and publishes them with the app
so you’re not waiting on the data warehouse for your dashboard to load
just released
@duckdb
VSCode SQLTools v0.0.5, with support for DuckDB v0.9.1
- database explorer, query runner, autocompletion
- connect to a local, in memory or
@motherduck
DB
NLP exploration with
@duckdb
...
DuckDB doesn't have libraries like nltk for NLP, but that doesn't mean you can't do NLP, you just have to implement it from scratch
this is really some cool innovation
it uses
@duckdb
webassembly to reissue SQL queries in the browser against data files
this means you can explore very large datasets, with extremely low latency
Evidence Universal SQL is coming!
This brings multi data source support, and fast interactivity to Evidence
Sign up for the community preview next week here:
continuing on my vscode extension tour, just published an open source extension for duckdb
supports connecting to local, in-memory, and
@motherduck
instances
X premium users get analytics on how much people are viewing their content
if you open the devtools network tab, you can see all that data gets delivered in a JSON blob
it's a pretty hairy nested JSON, but as always, only a query away from a nice table in
@duckdb
writing a
@duckdb
extension for the first time
here's a quick guide
DuckDB extensions allow you to add:
- custom functions (scalar, aggregate, table)
- custom data types
- custom output formats
X premium users get analytics on how much people are viewing their content
if you open the devtools network tab, you can see all that data gets delivered in a JSON blob
it's a pretty hairy nested JSON, but as always, only a query away from a nice table in
@duckdb
@PradyuPrasad
so if I look after my friends child for four hours and they pay me $100 and then tomorrow they look after my child and pay me $100 for 4 hrs, that increases GDP by $200
But if we just agree to swap, then it wouldn’t
There are lots of uncounted exchanges of services
okay so how about this
@duckdb
's statistical functions have enough coverage that you can basically recreate facebook prophet in SQL
time series forecasting in SQL, here we come
next step: turn the text into a "Bag of Words" representation
it's a sparse table with one row per tweet, one column per word, count of words
in most SQL dialects this would be impossible - you need to know the columns ahead of time to pivot
not with
@duckdb
's PIVOT syntax
one of the more interesting parts of this is you need a list of "stop words" - common words that aren't interesting when doing text analysis:
I, you, they, about, in, that
they aren't built into duckdb, but there are lots of great public lists, so just a SELECT away
one of the more interesting parts of this is you need a list of "stop words" - common words that aren't interesting when doing text analysis:
I, you, they, about, in, that
they aren't built into duckdb, but there are lots of great public lists, so just a SELECT away
NLP exploration with
@duckdb
...
DuckDB doesn't have libraries like nltk for NLP, but that doesn't mean you can't do NLP, you just have to implement it from scratch
US BLS publishes avg salaries for 1,100 jobs at a state / metro level
6M rows per year = child's play for
@duckdb
data scientist salaries by state, 2023
in most bi tools it’s hard to create layered maps
because drag and drop UIs for creating maps layers suck
in
@evidence_dev
you compose maps from layers with code
based on the wonderful, open source
@LeafletJS
if you missed any of the talks at dbt coalesce this year, I've published an "unofficial coalesce app" where you can find slides for ~30% of the sessions.
planning to add links to youtube vids after they are published too
Evidence Universal SQL is coming!
This brings multi data source support, and fast interactivity to Evidence
Sign up for the community preview next week here:
90% of data team-biz stakeholder interactions
Data: Hey, we're from the data team, and we're here to help. If there's a problem that we can solve with data - ask!
Biz: I have this excel, it's taken me 100 hours to build, and takes me 5 more each week. Can you automate?
Data:💀
Did a demo call with
@Oracle
MySQL Heatwave today, and must say the pitch is good:
1. One database for both transactional and analytical queries: Less ETL needed.
2. It’s 3x faster and 5-8x cheaper than [Snowflake / Aurora / BigQuery]
3. Support for querying CSV, parquet files
tables containing lots of KPIs in rows are really common, but somehow they are really hard in most BI tools
messing around with the UX in
@evidence_dev
thoughts?
Just arrived for our wedding in Lyon and
@AirCanada
has lost our bag, with my fiancée’s wedding dress in it.
Can you help get the dress to us before Friday and save the wedding? +1 647 207 7690
Flew AC7968, AC867
Have willing friends in MTL, NYC & Paris so can get creative!