Matt Wonlaw
@tantaman
Followers
1,596
Following
645
Media
213
Statuses
2,505
Explore trending content on Musk Viewer
ハロウィン
• 912109 Tweets
Valencia
• 570224 Tweets
DANA
• 458980 Tweets
ポケポケ
• 117953 Tweets
Esenyurt
• 59845 Tweets
Ahmet Özer
• 47494 Tweets
Mercadona
• 44719 Tweets
フレンド
• 43900 Tweets
AEMET
• 42189 Tweets
LINGORM X TIGERCRY
• 40010 Tweets
RTVE
• 39329 Tweets
Turia
• 38374 Tweets
ピカチュウ
• 34050 Tweets
LINGORM N2U LIVE
• 32034 Tweets
Albacete
• 29126 Tweets
自民党会派入り
• 28031 Tweets
橋本環奈
• 21317 Tweets
ジェイド
• 21190 Tweets
フロイド
• 21145 Tweets
ポケモンカード
• 20891 Tweets
中川大志
• 18001 Tweets
ポケカのアプリ
• 15350 Tweets
ミュウツー
• 14952 Tweets
コミコン
• 13193 Tweets
オースティン
• 12695 Tweets
#うちの本丸の厨番長
• 12555 Tweets
Pinned Tweet
My 4th appearance on a podcast! I'm becoming a better guest (interviewee?) each time, which means this one is the best one :)
This time we cover colocated queries, materialized views, sqlite & crdts.
4
3
43
tfw appeasing the type checker is more work than writing the actual code.
62
17
388
I never thought cross-device sync and multiplayer could be sooooo easy.
With CRDTs on SQLite I just declare which tables are synced and voila, my app is multiplayer.
7
20
336
One of the things I'm most excited for is partnering with the LibSQL folks to create a custom syntax for defining tables which are backed by CRDTs!
LibSQL is truly getting SQLite ready for the coming decades --
5
25
163
Found a hornets nest on tech twitter. Here's some TypeScript types from Prisma:
23
10
157
Interested (or know someone interested) in databases? Conflict-free data types? p2p?
Looking for help on
- ~130 companies signed up for the beta release
- $16,000 in paid sponsorships over the last 6 months
- 1.2k github stars
- Front page HN 3x
3
22
137
"Most databases were built when data was static and queries were dynamic. For applications, most queries are static and the data is dynamic" -
@matthew_linkous
I never thought about it this way before but this is spot on.
2
6
138
CRDT extension successfully compiled and running in the new official SQLite WASM.
Adios absurd-sql and sql.js
11
11
108
Working on a post but I present a React table of 2 million rows, dynamically filtered in real time.
tldr: the data structure backing the table is maintained via differential dataflow so we only need to process the single row that changed rather than the entire set when filtering
5
8
83
A little talk on using SQLite as a Collaborative & Multiplayer Application File Format
1
10
86
quite worth it for the perfect auto-complete. Even prevents you from selecting ambiguous fields.
12
0
66
I was blown away by
@aboodman
's talk and the Replicache model so I spent the last day experimenting with a variation on the idea:
"Creating CRDTs without specialized knowledge"
5
16
68
This confluence of things:
- Event sourcing
- CRDTs & Causal Graphs
- Incremental View Maintenance
is going to unlock some stellar new infrastructure.
1. Event source your entire app.
2. Incrementally maintain app state as a view of events.
3. Rebase events for multiplayer.
3
8
57
CRDTs -- design overview of conflict free sqlite
4
7
55
Would anyone believe me if I said I can get IndexedDB to read and write faster than a fully in-memory SQLite DB in the browser?
9
1
54
I've never been so excited to publish new documentation. Just another day or two of writing!
1
2
51
Round tripping the server each time. Imagine if this was local first 😃
Next.js and React have never been easier, and they've also never been harder 🧵
Here's the universal 'minimum viable app', a TODO list, implemented with the latest features in 45LOC
112
119
2K
5
2
50
This is the happiest day of my life! Well, definitely up there.
wa-sqlite is now MIT licensed!
3
6
48
What a clever way to automatically run a destructor when a variable goes out of scope in JavaScript. A generator that yields and has a `finally` block. 🤯
7
4
46
I realize I never really talk about the developer experience of having SQLite in the browser + sync.
Working on a series of 10 minute videos to cover this.
Part 1 Setting up SQLite in the browser: solving persistence, shared state & cross-tab sync:
1
6
46
In case anyone needs it - convert your PG database to a SQLite database:
Also emits the PG WAL LSN so you can logically replicate to SQLite after initial import.
3
6
44
is an amazing piece of work in terms of improving the reactivity story for SQLite.
1
6
42
I'll be a bit tied up for a while. New addition to the household.
9
0
41
I was accepted to give a talk about & local-first software at
#DWebCamp2023
! 🎉
See you there!
@GetDWeb
2
5
37
So many people in the crdt & data sync space now. Feels like it is time for me to move on to the next frontier of problems.
7
2
36
Listened to a number of talks and podcasts recently where the guests say that they don't need CRDTs b/c they're "too complex" but then go on to explain their custom solution which is... a CRDT 😅
Maybe this means I shouldn't mention CRDTs to some audiences?
8
3
32
Almost have the entire SQLite dialect covered for type generation. Preview of joins & sub-selects.
Builtin functions, conditionals, with clauses, etc. supported as well!
Next up: syntax highlighting and intellisense
Pairing with TreeSQL is going be 🔥
In collaboration with SkipLabs, Riffle and
@tantaman
, we sketched out a simple SQL query extension for GraphQL-like nested query results. 👀
It's already really nice to use and will allow me to simplify my application (e.g. by removing GraphQL on top of SQLite).
5
5
55
5
1
33
Can't think of a more ideal place to work and exciting set of problems to work on. Happy to be a part of the team.
3
0
32
TS <-> SQL Integration becoming real. Next up: generating types for `SELECT`.
3
3
32
Not having to ever think about cross-tab sync again is enough to make me pay for this if it wasn't open source.
3
0
31
I've got differential dataflow implemented in JS. What should I do with it?
8
3
31
Been pretty silent for a while. That's because useQuery(...), while a great dev experience, has scaling problems.
So I'm scaling `useQuery` style hooks to 60fps response time when working with millions of items and keeping those queries up to date as data changes.
3
3
29
Dependency tracking for TypedSQL!
Modifying a schema will go update the return types of queries using that schema.
0
2
30
@kentcdodds
@joeflateau
Rust has a ? operator which will automatically return an error if one is encountered. Eg: ‘()?’.
This is pretty close to exception ergonomics but typed and thus checked at compile time.
1
0
28
Seeing my libraries show up in other local-first & dist sys infra 🤩
1
1
27
I used to think forking sqlite (e.g.,
@libsqlhq
) was crazy but I'm on board now. While SQLite is great, I just don't think it is ready for the demands of the coming years.
Just one tiny tiny slice of the mounting problems --
2
3
28
It's been 1 year + 2 days since got started.
My
#LocalFirst
journey started way back in 2012 with
A presentation editor that ran entirely locally in the browser.
Its remote storage solution at the time was
2
3
28
The obligatory TodoMVC app.
This time collaborative and replicated p2p via CRDTs in SQLite.
Still a bunch of bugs, docs & packaging to clean up but sooo close to a stable release.
The client code/interface is clean 🧽
2
1
28
Instant and jank free ui is hard to pull off. It’s why we’re building
Oh my fucking god they’re not optimistically leaving the dropped appointment in place? Just reverting and waiting for the server to send new content down?
22
3
345
0
0
28
An alternate approach to syncing SQLite DBs --
Similar to this idea but simplified by going through a central server.
0
1
24
Nice to be working with a team again. I forgot how much you learn just by watching other people operate.
2
0
23
> we're approaching 600 nodes and ~40K changes per second
Exciting to see my work deployed into production but always makes me nervous 😅
3
1
24
Interesting realization that if you're syncing CRDTs through a central server you can apply whatever sort of validation logic you want on the server to preserve constraints. The validation logic just needs to write fixes as a new database version to sync back to clients.
6
4
24
React has graduated to PHP status. Everyone shits on it but it runs the web.
3
0
23
In the refinement phase of cr-sqlite -
- decreasing crdt metadata size by 5x
- improving crr creation syntax
- adding support for resurrecting deleted items
- adding list CRDTs to the DB so you can have collaborative text w/o yjs or automerge
- improving schema migrations
3
3
22
@VicVijayakumar
I guess Netflix includes RSUs in their calculation, the others are just base pay?
4
0
22
Spent a week optimizing `useQuery` to get hundreds of concurrent SQLite queries to return in a single frame in the browser.
And now realizing that implementing a fully reactive SQL layer actually isn't that far out of reach.
ty &
4
1
21
@jmhodges
Facebook handles most of these problems by converting all queries to persisted queries at compile time. That allows you to know exactly what queries can be executed and to what depth and so on. Authz is handled at the individual node level via fwrks like
3
0
21
Big release today. Little did I know the cr-sqlite sync layer previously didn't work in `Chrome for Android` 🫠It does now!
Thanks to everyone who provided feedback and reported bugs!
0
1
21
This is 100% the reason why I'm working on
The amount of work the linear team had to do to get sync working well is crazy.
0
1
20
@ssalbdivad
Yes, that is true. I'm explicitly opting into this so I can provide a rich API for users. `appease` was a poor choice of phrase.
3
0
16
Putting together some benchmarks and examples for Materialite (that's what I'm calling this incremental view maintenance in JS library) --
I always underestimate how long it takes to get something ready for public consumption 🤪 Just a few more weeks?
1
1
16
Spent the am building reactive code cells that can depend on one another for the new documentation site.
Like ObservableHQ but I didn't want to include their notebook interface.
5
3
15
So who is going to write about how local-first can be a progressive enhancement? I know this sounds like a contradiction in terms.
There are two ways that I love to build web apps:
Server First and Local First.
1. Server First (Remix today). HTTP+HTML are the base API, assume a connection, or talk to service workers as if you're talking to a server.
This has the potential for very fast page loads, "land
27
29
267
5
1
16
Looking forward to doing some TinyBase explorations backed by SQLite this week.
Could be a big week for TinyBase. I'm pretty excited about SQLite & CRDT synchronization in v4.0 - nearly there!
I just put up a fun TinyBase version of
@tantaman
's awesome cr-sqlite demo to give you an idea.
Let's go!
2
3
36
0
1
16
Notion and Slack have so many rough edges in terms of UX.
- Code blocks that become inescapable
- Inconsistent shift+enter behavior
- Lists responding to delete incorrectly when things like mentions are in them
vs Obsidian & Discord where everything works as expected.
2
0
15
@captbaritone
Yeah, they're railroad diagrams and fairly common in the compiler space. The JSON spec uses them too:
and a library to generate them if you're interested --
0
0
15
Haven't used Synergy since high school but coming back to it, it is pretty amazing.
Can move my mouse and copy-paste between all my machines as if they were a single machine.
2
1
15
The bright side of landing a perf regression is taking time to add more perf tests in the post-mortem and finding a 50x perf improvement.
Pulling change sets from cr-sqlite will be 50x faster in the next release 🚀
1
0
15
Not sure how valuable it is but getting React and all the things running the browser with no build steps was fun.
3
2
14
@pvh
All my intuition points to the opposite of this quote being true.
- Devices have tons of compute and storage waiting to be used
- Cloud costs are crazy
- Privacy matters to people
- Given how many devices people have, data actually lives longer when local and replicated to peers
1
1
13
Cool to see projects being built on cr-sqlite 🎉
1
3
14
@brian_lovin
Just let my apps interoperate. I don’t want ten calendar, email, doc, sheet apps.
2
0
14
Do any databases have direct bindings into a host language that, rather than creating SQL, creates the query program directly?
E.g., `({...}).where(...)` would not have to go to the SQL step and would go directly to a compiled statement.
12
1
14
On the debate between raw SQL & ORMs, I find myself flip-flopping over the years.
Is there a final answer at the end of the curve? Or just an unsatisfying "it depends"?
17
1
14
If you've been wondering what I've been working instead of cr-sqlite the last several months, tune in to the talk!
On my way to
@localfirstconf
. Really excited to talk about taking sync mainstream:
Thank you very much to
@schickling
for organizing and the endless energy to bootstrap this community.
@rocicorp
is proud to sponsor, and looking forward to many more
3
0
65
1
0
14
If we made houses like software: we'd have proprietary electrical sockets, our homes would only work with appliances made by the manufacturer of the home, we'd be charged a fee for exchanging money in the house.
@brian_lovin
Just let my apps interoperate. I don’t want ten calendar, email, doc, sheet apps.
2
0
14
3
1
13
There are so many backend as a service companies because it’s the easiest problem to solve that doesn’t actually have a solution.
2
1
13
Type-level programming melted tech twitter's brains. Not because the programs were difficult but because they look different.
Type-level programming 101 by analogy to JS constructs.
1
0
13
Noodling on replicating a subset of the CRDTs stored in a database --
1
2
13
A nice property of a query language where every query is a subscription is that you can globally optimize across all queries.
Multiple queries doing the same join? Share it between all queries. One query is a more constrained version of another? Re-use the first query's output.
4
1
13
I find, now that my projects have traction, I'm moving at a glacial pace :(
- Messages to follow up on
- Integrations to consider
- Design input to provide
- Meetings with interested contributors
- Job offers to consider
Is muting everything and grinding a valid option?
4
0
13
@enlightenedcoop
lots of repliers are misunderstanding how intention is used here. They’re reading it as “good intentions” rather than being intentional and crafting one’s life like a work of art. Fascinating.
0
0
11
The biggest benefit of migrating a project from C to Rust is people actually starting to contribute. The second is not spending an inordinate amount of time reviewing code for memory issues. And lastly, being able to make large changes with confidence.
2
2
13
Nearing the v1 release of conflict-free sqlite!
0
2
12
Print books have a serendipity that I often don't experience in the digital world. Stuck on a problem, opening a massive book and somehow landing on the exact page I need.
0
0
12
Getting tired of seeing so many query builders trying to solve the TS integration problem while introducing their own problems.
Did you know SQLite lets you encode your own type hints into create table definitions?
We can even make TypeScript aware of these w/o a query builder.
2
1
12
So the official WASM build for SQLite is still rather rough. Mainly due to OPFS's coarse grained file locks.
Back to wa-sqlite and indexeddb-vfs for the time being.
1
1
11
@kriswuollett
@jmhodges
more or less. The upside is you can ship quickly and then optimize when there's actual problems.
Of course there should still be _some_ up-front thinking in terms of expected query patterns and modeling the data in a way to serve those kinds of queries.
1
0
12
💯. Transactions, mvcc, rollback, persistence, queries over objects should all exist in your language.
"Database features should be language features"
Alexander Stigsen from
@realm
shares his insights into how with local-first you want to bland the gap between the database and the language.
0
6
48
0
0
12
💯. Just make a modular monolith and let automation figure out deployment details. E.g., automatically splitting pieces that don’t talk (or barely talk) into separate deployments.
0
0
11
I'm excited about pulling in the more complicated operations () and using differential dataflow to maintain the views for an application that is entirely event sourced. Having a fully event-source app unlocks new things --
I've got differential dataflow implemented in JS. What should I do with it?
8
3
31
1
1
11
@DoomChild2
@Grady_Booch
mono vs many is an incidental detail.
You want, regardless of repo structure:
- atomic commits across _all_ your code
- atomic deployments based on a single sha
- ability to bisect the entire codebase to automatically find bugs
Monorepos are the only tool for this currently.
1
0
11
This is just full of gems. I've found my kindred spirit?
@CoderHi
Using SQLite as your reactive data store and single source of truth ~36 min in
Also eschewing ORMs and doing everything via store procedures ❤️
1
1
11