MapReduce is one of the foundational paradigms of Big Data processing, pioneered by Google!
In this blog post, I review the original paper, and add some diagrams to make it easy to understand!
Bengaluru friends, would y'all be interested in a monthly~ meetup that covers lower-level {distributed,} systems, databases, compilers, and dataflow systems?
Let me know 👀
This compiler is possibly my new best friend - have you ever seen such a comprehensive error message? it even suggests changes that you can make 😭
@rustlang
🙏🙏🙏
Skiplists today with
@therealdatabass
and MIT OCW!
An extremely interesting data structure, though I'm a little rusty with the math. i need to do the proofs + implement it once to be really comfortable.
Probabilistic data structures rock!
Using tarpc for my raft implementation in rust. As far as I know, this is a new take on how RPC libraries should be - Big win for rust-only projects to not have to mess with DSLs, tarpc relies only on traits and makes the magic happen 🚀
And we're finally live with "Understanding and Implementing Skiplists" - explaining the intuition, the learning, and all the nitty gritties of implementing a skiplist in C++.
Skiplists it is. public declaration: I'm going to write a blog post titled "understanding and implementing skiplists" and publish it on the 1st of October, and It's going to be good!
I'm going to implement Raft in Rust this month, with the aim of making a distributed in-memory key-value store with it. The goals are the same; hack, until I get a functional implementation. Not much care about anything else!
blog and code will be out by the 1st of November 🚀
I'm super stoked to share that I'm now on the Storage team at
@couchbase
, and I'll be working on Magma, our awesome LSM Tree-based (and then some!) storage engine.
I look forward to working with these really awesome folks and learning as much as I can from them 🚀
It's such a beautiful thing to see systems-oriented projects back at hackathons: (all 1st year students!) one team is attempting to add primitive gc to C, one team is attempting to implement a p2p collaborative text editor... This kinda stuff puts a smile on my face
Organizing this and listening to these talks was the most fun I've had in a while! Thanks to everyone who brought life to this meetup :)
If you're in Bengaluru, into systems, and want to show off what you've been working on: DMs are open!
Aaand that's a wrap on the v1.0 of Bengaluru Systems!
We'd like to thank everyone for showing up, and especially thank
@HasuraHQ
for being such gracious hosts!
pictures follow :]
As a beginner, I can't help but appreciate how good
@rustlang
docs are, as well as The Rust Book. Thank you to
@steveklabnik
and
@Carols10cents
🙌 and the fantastic community - I'm already feeling the love :D
Skiplists it is. public declaration: I'm going to write a blog post titled "understanding and implementing skiplists" and publish it on the 1st of October, and It's going to be good!
It's been eight months since I've started working full-time and I've since struggled to work on side projects.
To fix this, I'm going to write a technical blog post every month (thanks
@eatonphil
!) and implement the thing I'm talking about in the blog post. I'm excited! 🚀
What they don't teach you about leaving university is that learning new things every day is something you gotta deliberately practice;
you get used to it in uni and expect it to continue, but you see a noticeable decline in mental flexibility once that practice starts to atrophy
Met
@vaibhaw_vipul
yesterday and had an extremely insightful conversation covering life, databases, distributed systems, personal projects, high agency,
@BengaluruSys
, career and a lot more - I definitely walked away with some invaluable advice too. Thank you so much, Vipul!
Met
@AnirudhRowjee
. He is organizer of
@BengaluruSys
.
This younger generation is far more enthusiastic, confident, and clear-headed than I was at their age!
Looking forward to attend their meetups soon.
I've officially ranked up and completed the "forget your lunch bag" quest -
"Forgot lunch bag in school" ✅
"Forgot lunch bag in college" ✅
" Forgot lunch bag at the office" ✅
Jumped back into reading research papers after a break, starting today with the Dostoevsky () paper, and small parts of the Calvin () paper.
Super pumped to get back to doing this more regularly🌞
@AnirudhRowjee
is a student at PES University where he's one of the core team lead of a student-driven community with an open-source initiative called Homebrew.
Tune in as he takes you through his experience of building it while encouraging students within the FOSS Culture.
Hello, Bengaluru Systems Enthusiasts! let's boot up our meetup :)
we're going to be:
1. Using Hasgeek for meetup registrations and talk submissions (link in bio)
2. Using Telegram to discuss and coordinate meetups -
1/
Building in public - Saaru is a (slightly opinionated) Static Site Generator written in Rust. It supports Eleventy-like Data Merge to let you organize content your own way, using either Tags or Collections.
(codebase is a mess rn, going to refactor soon)
"Wrote a tokenizer by hand because it was reentrant and threadsafe when nothing else was" 🙌
Thank you
@andy_pavlo
and the
@CMUDB
group for making these lectures public!
"Every plain text file has a hidden superpower: when it includes the right symbols, it can be converted into something that can be interpreted by a CPU. That is the magic of a programming language."
@timClicks
sir, this should be framed somewhere
It's really the best feeling when your college juniors do better than you (and you help them do it) - keep an eye on these folks, they have a knack for building awesome, performant software :D
My teammates, Adhesh Athrey,
@anirudh_sudhir
,
@polarhive
, and I presented our project Anna, a static site generator, at the 76th Go Bangalore meet.
This project was part of the AIEP program conducted by
@AcmPesu
.
TIL about SQLite's SchemaCookie: it's an integer that's incremented when the schema is changed, so that prepared statements compiled against a particular version of the schema know when to try to recompile!
this hits the nail on the head. Textbooks are extremely advantageous as a reference not just because the folks who wrote them likely discovered a few things in the field, but also because textbooks provide you with the necessary *context* a lot better than blogs/stackoverflow
me in college: textbooks? eww, i can just learn everything on blogs & stack overflow
me now: textbooks? omg finally a curated, authoritative reference on X topic
OpenGL is Fun!
1 - Sierpinski Gasket made entirely of points (DIY Primitives, because I like pain) with the color as a function of recursion depth
2 - Not adjusting for window resize creates this fun effect where re-renders keep happening
@logseq
is super-fun to use already! I'm loving the branch/leaf model, and templating is suuuuper helpful!
can't wait to see what new things I discover because of this tool. Kudos!
Since Meetups have been going through existential crises, my hope is that the future of tech meetups is not just *programming language communities* but oriented around *domains*: browser programming, systems programming, game development.
Nothing better than reading a research paper and thinking about the answer to a question that comes up, only to have your answer be the next line in the paper
Our third meetup page is live! 🎉
📣 If you’re a systems enthusiast and want to share what you’ve been tinkering with, submit a talk proposal. We’re looking forward to another round of amazing discussions!
See y’all on 28th September!
note to self: boxes are boxes, but should never be containers. lots of things are boxes, but it only becomes a container when you let it limit you in some way.
it is possible to be good at multiple things, to have multiple skill sets and competencies at the same time.
The HNSW algorithm is so cool - it combines probabilistic skip lists with Navigable Small-World Graphs to accelerate nearest neighbour search.
Really cool stuff and a great video by
@jamescalam
and
@pinecone
explaining it -
Can't miss this! Madhav's talks are some of the best out there, and everyone should grab this opportunity to catch him live.
Plus, CRDTs! Who doesn't like those?
If I wrote a blog post "walking through" my understanding of a systems/databases paper, would you read it?
Seems to me like it's a great way for me to get back to both reading and writing.
writing on paper is the best feeling ever; maybe this is a consequence of the slow normalization of dopamine levels you get by writing on paper due to a lack of easy to access distractions?
A good, simplified way to compare B-Trees and LSM Trees is to think of the problem this way:
you need to maintain a counter on a whiteboard. The B-Tree is just a number written on your board, in digits, and the LSM Tree is a series of lines that you use to keep tally
No better way to restart
@papersweloveblr
other than by a talk from
@iavins
on SQLite 💯
Fantastic talk, lots of learning, and the QnA session post the talk provided an amazing level of insight into what today's DB nerds are thinking about :)
So glad I got to attend!
Thank you to everyone who showed up, that too on the Friday eve. I had great fun discussing with you all.
I met many amazing people and had great discussions; please DM / email me to continue the conversation.
This was my first ever public talk; any feedback is welcome.
It's incredibly rewarding to just immerse yourself in a particular topic or a concept for a solid amount of time and see it to it's depth, without distractions, and without interruption.
Hello, Systems Enthusiasts!
the v1.0 release of the Bengaluru Systems Meetup is happening in Bengaluru on the 6th of July (11AM-1PM)!
CFP and Venue details coming soon. Stay tuned!
Apart from convention and tradition, why do we teach loops with i, j, k? There are so many confusions and problems that can be avoided with better naming - outer_count, inner_count make for much easier reasoning than i j k imo
Aaand that's a wrap on the v1.0 of Bengaluru Systems!
We'd like to thank everyone for showing up, and especially thank
@HasuraHQ
for being such gracious hosts!
pictures follow :]
Just finished reading "The Three-Body Problem" by Liu Cixin, and all I can say is... wow.
It'll take some time to fully realize the expansion of my imagination this book's responsible for. Truly an excellent, gripping read.
I wasted a bunch of time on a personal project trying to chase down what I thought was a CMake error, thinking I'd written the CMakeLists file wrong.
turns out I'd forgotten to namespace my implementation in my .cc file - I had `Insert(...)` instead of `Skiplist::Insert(...)`
Learnt more about Bloom Filters today from
@therealdatabass
💯
Open question: what happens on a k-way hash collision (all functions collide at once)? Is this statistically too improbable to be a concern?
❌ Exact
✅ Approximate
The rise of vector search has brought on a shift from exact to approximate results. Let's talk about why approximate results can become the new normal and how much approximation is acceptable 🔎
An update on this: I've read most of the paper so far, and I've made good progress on the implementation, and I've got a decent skeleton of the writeup. We're on track to still be done by 1st October, though I might not be able to get around to concurrent skiplist maintenance :(
TIL that Postgres has what's basically GC - MVCC that just copies entire rows, needs index write amplification, and a bunch of other backpressure. Awesome article by
@OtterTuneAI
!
Studying Operating Systems (especially through OSTEP) is giving me a renewed fascination for computer science as a whole.
It's mind-blowing, how much goes on under the hood.
Asking someone "what languages do you know" in Bangalore will get you two types of respones -
1. "Kannada, Hindi, Tamil, Telugu, Malayalam, Marathi"
2. "Java, Python, C++, Javascript, C#, Go"