astle dsa
@AstleDsa
Followers
956
Following
116
Media
112
Statuses
3,321
Living in complexity, formalism, mathematics and computer science
Joined March 2022
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#Mchoice2024
• 1834169 Tweets
#บัสซิ่งไทยแลนด์FinalEP
• 219712 Tweets
Texas -6.5
• 186576 Tweets
MCHOICE X ML
• 126915 Tweets
#Venue101
• 100171 Tweets
#liSTeningPARTY
• 94238 Tweets
#SixTONESANN
• 84702 Tweets
TOP ELITE ARTIST V
• 43087 Tweets
#ZMZMINZM
• 33589 Tweets
リスアニ
• 27449 Tweets
ROCKSTA SA CALAMBA
• 26718 Tweets
デスゲーム
• 19387 Tweets
紫耀くん
• 16492 Tweets
Hysteria
• 14384 Tweets
Lee Carsley
• 11772 Tweets
Wayne Rooney
• 11115 Tweets
アクキヨ
• 10841 Tweets
カンスト
• 10724 Tweets
ビーフシチュー
• 10558 Tweets
#LEO_HBD_26th
• 10253 Tweets
#HAPPYLEODAY_2024
• 10068 Tweets
Transformer in C (from scratch)
Finally took the time to complete this, the penultimate step for the project. Writing this was easier than I thought, added AddNorm and FFN layers as well.
A low hanging fruit would be to parallelize the attention mechanism (model parallelism)
15
78
842
From scratch projects are underrated. Here's a list of things that I want to build from scratch (also making lists are fun):
- Operating System/Kernel
- Compiler/Interpreter
- JS Framework
- Database
- Vector Database
- Deep Learning Framework
- Git
- LLM (can't run it ofc)
19
25
415
Nearly done with my deep learning framework in C. I have got:
- A matrix library
- Autograd engine
- Batch, Layer and Model abstractions (?blocks)
- Parallel/Concurrent Model Training (Data Parallelism)
After cleaning up, i'll be getting close to my final goal with this project
5
24
373
Implemented a numpy + autograd engine in C, and trained a simple MLP which learned the inverse function of a matrix. Fun, but much more left to do (which includes cleaning up).
8
9
331
While talking to an embedding systems engineer on C being an unsafe language, he mentioned that the way they avoid memory leaks is by simply not using malloc/alloc, which means their code to entirely deterministic at compile time. A reason why C cannot be replaced in embedded sys
10
6
231
Got it to 1.536kB.
Turns out I was allocating around 2 million % more memory than needed.
Had to really hack around to getting from
~300mb->~7mb->1536B.
The total overhead memory usage was finally 0% (3M%->400K%->0%).
Tracking memory was fun tbh
Learned a lesson in memory management. My previous code for training a single MLP in my autograd engine in C was taking around ~300Mb of memory.
Had to optimise a lot and finally managed to bring it down to ~6Mb. Figuring out how to bring it down further
2
4
72
4
5
180
@ludwigABAP
The problem is that it only extends to the very basic symbols. A little more abstraction and it's pandemonium
3
0
101
@zmkzmkz
I did something like this (I think ?) but mathematically:
Exploring architectures- Transformers II
A match made in heaven, keys and values
mathblog.vercel.app
1
10
97
Learned a lesson in memory management. My previous code for training a single MLP in my autograd engine in C was taking around ~300Mb of memory.
Had to optimise a lot and finally managed to bring it down to ~6Mb. Figuring out how to bring it down further
2
4
72
@JoshuaLelon
But that's a very GenZ thing I feel (maybe millennial) due to advent of mobiles and facetime
Otherwise I think all the defining moments in my previous generations, at least where I live, happened during broad daylight
1
0
56
Nearly forgot to commit :
0
5
53
@meatballtimes
Sort of a first world take. Though rest of the population has bigger issues to consult with than dealing with their emotions: they cannot afford the luxury to care for themselves. I do agree though if a person has that luxury, they should probably sort out their emotional needs.
3
0
48
@yacineMTB
Supposedly the best mathematician in the world right now, surprised you didn't know
I think he's been the best for quite some time now (for whatever the definition of best means in the mathematics world)
3
0
46
@KamerynJW
You can learn in a few minutes and still don't know them ? Kinda reflects the amount of effort you put in
0
0
30
It's not healthy for me to have realised that I can implement my own garbage collector in C .
It's hard af
1
0
28
> 3 hour youtube video on Electromagnetism and Gauge theory
> Know nothing about either topics
> Not familiar with any sub-topic/pre-requisites
> Proceeds to watch anyways
1
1
26
@eatonphil
Couldn't agree more. I found so many cool data structures and algorithms (not conventionally taught anywhere) while looking through dbs, and also lately vector dbs.
S3, TigerBeetle, Sqlite, Pinecone, Postgres have some really great structures underneath that noone talks about!
0
2
25
@allgarbled
3b1b videoes were never meant to be completely educational, they were more of getting your interests into the subject.
There are better ways to learn and retain things about a topic than watch videos on it: just read textbooks ?
Or watch other channels with more technical vids
1
0
23
@angryfermion
I'm an economist. We made inflation up so that central bank can pretend to have work.
1
1
23
Peak knowledge being distributed on LinkedIn. Twitter ppl lowkey missing out on some gems
4
1
24
I cannot express my regret of finding this YouTube channel this late.
1
0
22
If I ever decide I need to solve problems, I'd rather solve problems like these (and other math exercises) than leetcode
1
0
20
Oftentimes I feel people don't *over* estimate, but *under* estimate the difficulty of mathematics, by comparing it to coding problems
CS concepts and problems are hard, but their abstractness is no where near math level. Higher mathematics is just hard, not to mention vast
3
0
19
@TonyTheLion2500
I've not read many, but the one I liked was Gilbert Strang's Introduction to Linear Algebra.
1
1
19
> Sits in front of VS code
> Encounter a problem without write a single line of code
> Ponder on the solution for straight 30 min
> Finds a viable solution
> Takes a break due to exhaustion caused by the oxygen used by my brain
> 0 lines of code written
2
0
18
0
0
17
@OccupyingM
All like is just a computer simulation, and we're the neural nets finding our way
1
0
17
Programming LLMs isn't hard, it's just array manipulations tbh
The difficulty might lie in the math of the architectures and the reasoning behind certain techniques
It's a shame that though overall it's a very easy field to get into, it's quite hard to train world models
2
0
16
Things I theoretically *can* write a book on :
- Linear Algebra
- Deep Learning
Since these are the only two topics I've consumed a lot of resources on
Also I love textbooks
1
0
16
@VictorTaelin
So :
Interaction Combinators is a computational model
Interaction calculus is the language derived for it
HVM is the practical implement of the of Interaction calculus
And Blend is the final completely parallel programming language ?
2
2
16
@eatonphil
I do find databases so much more alluring than web dev, but the lack of resources for database development is astonishing. A beginner would be overwhelmed, and thus simply deflect to web dev.
I'm sure no beginner would read Designing data-intensive applications over some React
3
0
15
Wait wait, so Chris Lattner developed LLVM, Clang compiler, Swift AND Mojo ?
Crazy.
3
0
15
In my pursuit of memory management, I've become the garbage collector
1
1
15
Read >100 blogs by now, yet I still remember the contents of a majority of them and know exactly which one I need (in case of reference or anything else).
Simply paying attention while reading does wonders.
0
0
15
> Ran a program in C
> Ran a program in Python (numpy)
> C is faster
> Used printf() in C
> C is still faster
> Used print() in python
> Abomination
3
0
15
If I ever teach CS to anyone, this would be in the curriculum for sure
1
0
13
It's like watching
@thecodingtrain
or
@code_bullet
videos
This was what originally got me hooked on coding, great stuff
Implemented the actual genetic algorithm today, the rockets now reach the target, also made it look a bit pretty.
It's written from scratch in C using Raylib, I'll probably make the code public tomorrow and wrap up with this project, then get to building neural networks in C :)
17
62
837
2
1
13
@tailwiinder
My personal coping mechanism is simply to accept that these are the cards I've been dealt with, and I simply maximize whatever goal I've in life .
Blaming the country, people or even the government won't help. Yes they might be at fault, but we must look for a solution
0
0
12
Personal site
Really happy with what I got, looking for to maintaining and adding to it !
(Made by
@joodalooped
ofc)
1
0
12
Gonna recommend Java, Objective-C and Rust to any new beginners I meet.
Have to stifle the competition before it begins
1
1
12
Lately my reading >> thinking, leading to less number of thoughts.
I can't tell if it's a good or a bad thing, since I've been coding a little more than writing .
Writing something really makes me feel I'm making something original
0
1
11
Gotta write a db in Zig now
Probably a toy version of TigerBeetle
TigerBeetle got over 900 start on Github yesterday.
Zig is going to the moon.
0
6
47
0
0
11
People trying to make me feel bad about "wasting" my time by "not doing anything" after my bachelor's.
In their mind, time not spent in a formal institution or doing a job is time wasted.
Not to mention time not spend earning money or trying to earn money is not productive.
0
0
11
Not to mention any beginner would get a boost if they simply start implementing any of these in their preferred language. Would probably teach them almost everything important
1
0
10
Computer science is basically applications of Trees, Tables and Graphs . They are *everywhere*
1
1
11
Well here's the plan (unless I get distracted) :
1
0
10
Does reading a lot lead to being a good writer ?
My assumption is no, we know writing a lot leads to that (obviously), but is reading a lot a requirement as well ?
9
1
10
@ludwigABAP
I did mention it to the person, he also mentioned C code taking the least amount of memory to be stored ?
3
0
10
TIL how malloc works, and frankly it's a little different than how I imagined it.
3
0
9
Talking to someone who's passionate about a certain thing is the single best way to learn that thing.
Nothing beats a passionate person explaining the concepts they think about 24/7
A reason why I love blogs. They are almost always written by people who genuinely like the field
0
1
10
@VictorTaelin
Prompt Injections directly affecting the kernel would be disastrous ngl
I think security would be (and still is) the biggest concern
0
0
10
Programming languages probably the most interesting thing about programming
1
0
9
Kinda annoyed when I see a .ipynb file in a GitHub repo.
Just a pain to read anything on there
2
0
9
But memorization should happen through repeated exposure to the same underlying concept, *not* the same piece of text: that's just over fitting.
True understanding is roughly the ability to draw analogies and apply the concept, something repeatedly reading something won't do.
2
1
9
Read upwards of 20 blogs last week, and still got more .
Does this count as reading ? Not sure.
Though I have to start reading books again (have to comprehend how the current model of the universe is defined)
1
0
9
I can't believe I'm saying this but ... I've fallen into the anime/ML side of twitter
Which isn't necessarily bad, but damn every other post has some anime/manga pic going on (with programming happening as well)
I need some more "flavour", maybe something other than software
1
0
9
Just finished reading some blogs today, only to find another four . Idt I'll ever stop reading blogs/papers. Best habit I've picked up ever (maybe except exercise)
1
1
9
If you understand a text material after the first time reading it, there's no merit in reading it again. At all.
Wanna understand the concept?
Read some other material, consume some other resource. Memorization is extremely important in achieving any intellectual feat ...
2
1
8
@dothebit
Hmm if you're following a blog/tutorial, I think yeah it's kind of a waste (done that).
But implementing stuff you see by yourself, while trying to solve some of the engineering problems they pose can be great, and can actually point you towards something novel you can build
1
0
9
My lack of attention span is really showing tbh. I wonder if I'll ever master a single paradigm at this point. Went from:
game dev->Web dev->ML->DSA->low-level stuff->Index structures.
Not to mention writing as well
3
0
8
Ok no matter how I try to even read C++, the "::" syntax is just plain ugly .
Also I know it won't be that complicated, but I also hate when "<>" is used (in Rust or C++).
Readability takes a serious hit (Ik if I just practice and get used to it, it won't matter as much)
1
1
8
I really love with the syntax of a language is contained within a single page.
No bs, pure skills to build whatever the f you want
2
0
8
@kimdayers
Trade off between speed and toughness ?
You can solve extremely hard problems, but also take average time to solve easier ones (as opposed to grinding easier problems to solve them faster)
0
0
8
@wojakcodes
Speaking particularly or just be good in English? I can do the latter but struggle with the former
1
0
8
Assuming Complex Analysis takes Real Analysis one dimension higher, what's stopping us from going further ?
If real number line -> complex domain can happen with some concrete results, what's further >2 "dimensions" ?
Here I am taking a geometric view of analysis, hence dims
4
0
8
@BonesaiDev
Raylib was always daunting to me idk why, anything graphical in C is scary.
NNs in C would be a breeze for you tbh
1
0
8
@ludwigABAP
I'm kinda seeing the opposite around me actually
People be doing these AI/ML courses from college which deviate not only from CS concepts but from CS itself
They are taught data science, bit of analytics and think python is enough, alongside math
0
0
8
Reading too many things on LLMs. Nowadays I just make analogies for everything in terms of deep learning.
"Maybe I share this trait in the same personality space as my friend".
"Reading this concept again would lead to overfitting, gotta watch a video to generalise" .
1
0
8
@ludwigABAP
With the amount of resources you provide, you're gonna overload some nerds here (me)
1
0
9
Being a good listener is a skill that's highly underrated.
It immediately makes you seem smarter and allows you to give intelligent and thoughtful responses to people .
Probably the most important skill when it comes to socializing and inter-personal communication
0
0
7
Going out, riding a train is really surreal experience when I think about it.
Thousands of people, each with their own lives, stories and perspectives. Walking along the street, simply noticing the hustle and liveliness feels amazing.
Though I am privileged to be able to do so.
1
0
7