PyTorch
@PyTorch
Followers
419K
Following
820
Media
764
Statuses
2K
Tensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundation
Joined September 2016
Today marks 5 years since the public release of PyTorch! We didn't expect to come this far, but here we're🙂- 2K Contributors, 90K Projects, 3.9M lines of "import torch" on GitHub. More importantly, we're still receiving lots of love and having a great ride. Here's to the future!
59
446
3K
We’re excited to announce support for GPU-accelerated PyTorch training on Mac! Now you can take advantage of Apple silicon GPUs to perform ML workflows like prototyping and fine-tuning. Learn more:
72
684
3K
To help developers get started with PyTorch, we’re making the 'Deep Learning with PyTorch' book, written by Luca Antiga and Eli Stevens, available for free to the community:
35
906
3K
The full version of the Deep Learning with PyTorch book from Luca Antiga, Eli Stevens, and Thomas Viehmann is now available! New chapters include in-depth real-world examples and production deployment. Grab a free digital copy on:
26
711
2K
We’re excited to announce the release of PyTorch 2.0! . This version includes:.⚙️ 100% backward compatible.📦 Out of the box performance.📶 Significant speed improvements . Learn more 👇.
25
606
2K
Inside the Matrix: Use 3D to visualize matrix multiplication expressions, attention heads with real weights, and more. ⚡. Read our latest post on visualizing matrix multiplication, attention and beyond:
13
400
2K
We just introduced PyTorch 2.0 at the #PyTorchConference, introducing torch.compile!. Available in the nightlies today, stable release Early March 2023. Read the full post: 🧵below!. 1/5
23
514
2K
Want to make your inference code in PyTorch run faster? Here’s a quick thread on doing exactly that. 1. Replace ) with the ✨torch.inference_mode()✨ context manager.
8
358
2K
10x performance on LLaMa 7B, all in native PyTorch. No C++ needed. Check out our second blog post in a series on Accelerating Generative AI using Native PyTorch. 🔥.
14
259
2K
PyTorch is now installed by default on Google Colaboratory. Go ahead and `import torch`, `import torchvision`, `import torchtext`
26
447
2K
We're excited to release the torch.fft module in PyTorch 1.8. This module implements the same functions as NumPy’s np.fft module, but with support for accelerators, like GPUs, and autograd. Learn more 👉
17
268
2K
Microsoft VSCode integrates deeply with PyTorch out of the box. As @aerinykim highlights:. 1. It shows values inside tensors. (Left panel).2. By simply mousing over, you can see the variable's shape, weight, bias, dtype, device, etc.
21
423
1K
Announcing the alpha release of torchtune!. torchtune is a PyTorch-native library for fine-tuning LLMs. It combines hackable memory-efficient fine-tuning recipes with integrations into your favorite tools. Get started fine-tuning today!. Details:
19
297
1K
We are announcing that PyTorch will stop publishing Anaconda packages on PyTorch’s official anaconda channels. For more information, please refer to the following post on dev-discuss:
62
309
1K
Introducing torch.profiler! New PyTorch Profiler collects both GPU and framework related info, correlates them, performs automatic detection of bottlenecks in the model, generates recommendations on how to resolve these bottlenecks, and visualize. Read 👉
5
329
1K
PyTorch 1.8 is here! . Highlights include updates for compiler, code optimization, frontend APIs for scientific computing, large scale training for pipeline and model parallelism, and Mobile tutorials. Blog👇
1
269
1K
Learn PyTorch on GPUs for free via Google Colaboratory. each PyTorch tutorial now has a link to open it on Colab, where you can interactively execute and play with code. #thanksgoogle
4
280
1K
Introducing VISSL ( - a library for reproducible, SOTA self-supervised learning for computer vision! Over 10 methods implemented, 60 pre-trained models, 15 benchmarks, and counting.
10
255
1K
Understanding GPU Memory 1: Visualizing All Allocations over Time 👀. In part 1 of this series, we show how to use Memory Snapshot, the Memory Profiler, and the Reference Cycle Detector to debug out of memory errors and improve memory usage. Read more:
1
184
1K
PyTorch 2.2 is here 🎉. Featuring: .- SDPA support of FlashAttention-2.- New ahead-of-time extension of TorchInductor.- device_mesh, a new abstraction for initializing and representing ProcessGroups .- A standardized, configurable logging mechanism called TORCH_LOGS
13
187
1K
PyTorch 1.9 is here! Highlights include improvements for:.- torch.linalg, torch.special, and Complex Autograd.- Mobile Interpreter .- TorchElastic.- The PyTorch RPC framework.- APIs for model inference deployment.- PyTorch Profiler . See full details👇.
6
257
999
GPU Tensors, Dynamic Neural Networks and deep Python integration. Hello world!.
14
509
920
[v1.1.0] Official TensorBoard Support, Attributes, Dicts, Lists and User-defined types in JIT / TorchScript, Improved Distributed. Read more about the changes at As always, get the install commands on
2
314
912
[v1.0] : JIT Compiler, Faster Distributed, C++ Frontend, CUDA10.Read more about the changes at As always, get the install commands on
13
322
894
Introducing torchchat 🔥. A lightweight library to run LLMs locally across mobile, desktop and laptops powered by PyTorch. Learn more: #llms #mobilellms #localai #pytorchllm #edge #ondeviceai
16
190
886
Happy 3rd Birthday TensorFlow! Looking forward to competing with and complementing you over the years. #HappyBirthdayTF.
Happy 3rd birthday TensorFlow! We've come a long way since the first release in 2015 & TensorFlow wouldn't be the framework it is today without you. As we work on #TensorFlow20, look at all the features we've added over the years to make TensorFlow easier to use. #HappyBirthdayTF
12
114
872
torchvision 0.3.0: segmentation, detection models, new datasets, C++/CUDA operators.Blog with link to tutorial, release notes: Install commands have changed, use the selector on
9
272
863
Stochastic Weight Averaging (SWA) is a simple procedure that improves generalization in deep learning over Stochastic Gradient Descent (SGD). PyTorch 1.6 now includes SWA natively. Learn more from @Pavel_Izmailov, @andrewgwils and Vincent: .
5
206
847
The full hands-on tutorials, "Building Recommender Systems with PyTorch", are now available. We show how to build deep learning recommendation system and resolve the associated interpretability, integrity, and privacy challenges. See:
3
195
830
PyTorch Lightning 1.0.0 is now available. This is the final stable API to train and deploy models at scale, without the boilerplate. Read more about this release below:
5
163
801
Disney uses PyTorch for animated character recognition and to speed up its video processing pipeline. @DTCITechnology engineers also contributed new features to the Torchvision domain library.
0
184
788
If you installed PyTorch-nightly on Linux between Dec. 25 and Dec. 30, uninstall it and torchtriton immediately and use the latest nightly binaries. Read the security advisory here:
16
505
774
v1.6: native mixed-precision support from NVIDIA (~2x perf improvement), distributed perf improvements, new profiling tool for memory consumption, Microsoft commits to developing and maintaining Windows PyTorch. Release Notes: Blog:
5
234
781
The team @WadhwaniAI has built a multi-task network that detects pest infestations in cotton crops. This technology is being put directly in the hands of more than 18,000 farmers across India using #PyTorch Mobile, TorchServe, and Weights & Biases.
11
171
762
Today, we made usability and content improvements to PyTorch Tutorials including additional categories, a new recipe format for quickly referencing common topics, sorting using tags, and an updated homepage. Learn more:
11
191
761
Dear PyTorch users,.We're sharing the roadmap for PyTorch 1.0: production ready PyTorch. Coming later this summer.
9
285
758
DGL (Deep Graph Library) - Clean and efficient library to build graph neural networks including GCN, TreeLSTM and graph generative models. Includes auto-batching and other tricks for speed.
4
273
744
v1.7: CUDA 11 supported with binaries on updated profiling/performance for RPC, TorchScript and Stack traces in the autograd profiler, support for NumPy compatible FFT via torch.fft. Release Notes: Blog: 👇
10
185
741
Do you want to adapt an LLM on your own data and domain? 🤔. Learn how to finetune a 7B parameter model on a typical consumer GPU (NVIDIA T4 16GB) with LoRA and tools from the PyTorch and Hugging Face ecosystem in our latest post. Details 👉
7
175
738
Tensorboard-PyTorch plugin now includes graph visualization of your model. Live demo here: Github page:
0
268
731
PyTorch 2.3 is here 😎🔥. PyTorch 2.3 offers support for user-defined Triton kernels in torch.compile, allowing for users to migrate their own Triton kernels from eager without experiencing performance regressions or graph breaks. Details:
7
130
738
PyTorch 1.10 is here! . Highlights include updates for:.- CUDA Graphs APIs updates.- Several frontend APIs moved to Stable.- Automatic fusion in JIT Compiler support for CPU/GPUs.- Android NNAPI now in beta. Blog: Release:
11
158
634
v1.5: autograd API for Hessians/Jacobians, C++ frontend stable and 100% parity with Python, Better performance on GPU and CPU with Tensor Format ‘channels last’, distributed.rpc stable, Custom C++ class binding. Release notes: Blog:
4
220
714
We are a year old :).A post looking back at our journey, the lovely community, the engineering and love.
13
204
725
Part 2️⃣: Understanding GPU Memory 🤔. In our latest post, we will use the Memory Snapshot to visualize a GPU memory leak caused by reference cycles, and then locate and remove them in our code using the Reference Cycle Detector. Read more:
5
139
707
vLLM Joins PyTorch Ecosystem 🎉 .@vllm_project has always had a strong connection with the PyTorch project. Tight coupling with PyTorch ensures seamless compatibility and performance optimization across diverse hardware platforms. Read more:
9
108
730
Announcing Flash-Decoding 🚀. Flash-Decoding makes LLM decoding much faster, and in particular allows to scale to very long sequence lengths (64k+) without slowdown!. Read more on our blog:
2
128
700
Today we’d like to highlight features from functorch, a beta PyTorch library that provides JAX-inspired function transformations like vmap. (. If you’re not sure what sort of cool new things vmap allows you to do, read on to learn more!.(1/n).
5
131
688
Bolts is a new Deep Learning research and production toolbox from PyTorch Lightning. Iterate faster with pre-trained models, components, callbacks, and data sets, all modular, tested, and optimized for GPUs/TPUs. Simply subclass, override, and train.
0
201
681
3x faster text-to-image diffusion models, all in pure PyTorch. No C++ needed. Check out our third blog post in the series on Accelerating Generative AI using Native PyTorch. 🔥.
8
116
670
✨ Low Numerical Precision in PyTorch ✨.Most DL models are single-precision floats by default. Lower numerical precision - while reasonably maintaining accuracy - reduces:. a) model size.b) memory required.c) power consumed. Thread about lower precision DL in PyTorch ->.1/11
2
129
674
Two creators of passion projects that transformed the landscape of how we code today — Linus Torvalds and @soumithchintala meet for the first time, sharing a smile and a love for the open source community. #PyTorchFoundation
10
72
656
RoMa: an easy-to-to-use, stable and efficient library to deal with rotations and spatial transformations in PyTorch. Read all about this PyTorch Ecosystem Tool in our latest Medium post ⚡
18
114
630
The Global PyTorch Summer Hackathon is back! This year, teams can compete in 3 categories:. 1. Developer Tools.2. Web/Mobile applications.3. Responsible AI Development Tools. Read more at: #PyTorchSummerHack
7
199
638
Today, Mark Zuckerberg announced the launch of the #PyTorchFoundation under the @LinuxFoundation & a board of @Meta @AMD @awscloud @googlecloud @Azure & @NVIDIAAI. With these leaders, learn how we're giving the AI community tools to accelerate innovation.
9
155
625
We’re happy to officially launch torchao, a PyTorch native library that makes models faster and smaller by leveraging low bit dtypes, quantization & sparsity. Our techniques are written in easy-to-read PyTorch code spanning both inference & training:
6
125
659
PyTorch BigGraph: a distributed system for learning large graph embeddings.- up to billions of entities and trillions of edges.- Sharding and Negative Sampling.- WikiData embeddings (78 mil entities, 4131 relations).- Blog: - Code:
2
198
631
Stochastic Weight Averaging: a simple procedure that improves generalization over SGD at no additional cost. Can be used as a drop-in replacement for any other optimizer in PyTorch. Read more: guest blogpost by @Pavel_Izmailov and @andrewgwils
3
181
596
Announcing PyTorch 2.1 Stable Release ⚡✨. PyTorch 2.1 offers automatic dynamic shape and NumPy compilation support in torch.compile, as well as torch.distributed.checkpoint for saving/loading distributed training jobs on multiple ranks. Learn more 👇.
6
135
594
Announcing ExecuTorch 🚀. ExecuTorch offers a compact runtime with a lightweight operator registry to cover the PyTorch ecosystem of models, and a streamlined path to execute PyTorch programs on edge devices. Details:
10
130
590
Hydra provides an ability to compose and override configuration from the command line and config files, helping PyTorch developers to more easily manage complex ML projects.
7
138
567
PyTorch v0.2 is out: Higher order gradients, Distributed PyTorch, Broadcasting, Advanced Indexing, New Layers, more!.
2
283
575
[v0.4.1] Spectral Norm, Adaptive Softmax, faster CPU ops, anomaly detection (NaNs, etc.), Lots of bug fixes, Python 3.7 and CUDA 9.2 support and more. Full release notes at As always, update via commands at or via PyPI.
5
177
554
[v1.3.0] Named Tensors, iOS / Android support, Quantization, Type Promotion and more. Also available: TPU device, Detectron2, Captum for model interpretability, CrypTen for PPML research etc. Read more here: [Link to the PyTorch 1.3 blog].
2
181
559
TorchServe v0.1.1 is available with new features and support for HuggingFace BERT, Waveglow, Model Zoo, SnakeViz Profiler, AWS CloudFormation Template, and Automated Integration regression test suite. Read the release notes for details:
2
136
558
Torchmeta is a collection of extensions and data loaders for few-shot learning and meta-learning. It won first place at the Global PyTorch Summer Hackathon last year. Learn more in the blog post from Tristan Deleu, the project author:
1
148
550
PyTorch Hub: reducing the friction in reproducing and building-upon research.- Pull models with 1 line of code, and a few more to use.- Curated models. Open with Google Colab and @paperswithcode .- Publish your models by sending a PR.Blog: #ICML2019
1
220
555
Happy to share an early version of float8_experimental, a library to accelerate training with float8 + torch.compile written in native PyTorch 🔥.
5
101
501
Universal Music Translation Network: translating music across musical instruments and styles using unsupervised training.
1
169
557
Announcing PyTorch 1.11, TorchData, and functorch!. Highlights:.- TorchData, a new library for common modular data loading primitives.- functorch adds composable function transforms.- DDP static graph optimizations in stable.Learn more👇
3
100
537
Catalyst is a PyTorch framework designed for reproducibility, faster experimentation and code/ideas reusing. This blog walks through MINST classifier tutorial and compare PyTorch and Catalyst code side-by-side:
4
111
538
v0.4: Trade-off memory for compute, Windows support, 24 distributions with cdf, variance etc., dtypes, zero-dimensional Tensors, Tensor-Variable merge, , faster distributed, perf and bug fixes, CuDNN 7.1. Read more at: Install via
10
216
536
The PyTorch team is excited to share that our paper on PyTorch 2 has been accepted for presentation at ASPLOS 2024! . The paper delves into the implementation of torch.compile and highlights the key technologies driving it, read it here 👇📝.
6
83
529
PyTorch BERT models are now 4x faster, thanks to @nvidia.
Always amazed by what people do when you open-source your code!.Here is pytorch-bert v0.4.0 in which.- NVIDIA used their winning MLPerf competition techniques to make the model 4 times faster,.- @rodgzilla added a multiple-choice model & how to fine-tune it on SWAG. many others!
1
121
525
Introducing @PyTorchLive, an easy to use library of tools for creating on-device ML demos on Android and iOS. With Live, you can build a working mobile app ML demo in minutes. Learn more at and post your demo with #PyTorchLive.
11
149
530
PyTorch Lightning V0.9 is available now featuring the final API with better data decoupling, shorter logging syntax, synchronized batchnorm, and more. Learn more:
4
119
502
Torch-TensorRT is now an official part of the PyTorch ecosystem and now available on PyTorch GitHub and Documentation. Torch-TensorRT is a TensorRT integration for PyTorch that accelerates inference up to 4x on NVIDIA GPUs with just a single line of code.
3
98
504
The Computer Vision Recipes repository by researchers at Microsoft provides examples and best practice guidelines for building CV systems. It supports classification, retrieval, tracking, action recognition, and more. Learn:.
1
172
505
Captum is a library for model interpretability. Its algorithms include integrated gradients, conductance, SmoothGrad and VarGrad, and DeepLift. Learn more:
1
162
488
PyTorch 1.9 extends PyTorch’s support for linear algebra operations with the torch.linalg module. The module has 26 operators, including every function from NumPy’s linear algebra module extended with accelerator and autograd support, and more. Read more:
3
99
476
Github Engineering experiments with Semantic Code Search: search for code snippets using natural language. Built on top of fastai + PyTorch, it's fully open-source. Read about their approach here: Online Demo:
4
154
477
This tutorial will introduce compute and data-efficient transformers and provide a step-by-step to create your own Vision Transformers. Through this guide, you'll be able to train state of the art results for classification in both computer vision & NLP.
2
123
487
Introducing “Learn the Basics" - a guide to a complete ML workflow with detailed explanations on concepts like Tensors, DataLoaders, Transforms, and more. Thanks @sethjuarez, @subramen, @Cassieview, @shwars, @pythiccoder for your contributions! Click👇.
1
124
480
FX-based feature extraction is a new TorchVision utility that allows access to intermediate transformations of an input during the forward pass of a PyTorch Module. Read more below on what makes TorchVision's utility more versatile than existing methods.
2
92
462
Introducing FlexAttention: a new API that lets you implement diverse attention variants in just a few lines of idiomatic PyTorch code. 🔥. Check out the blog post for more details:
3
92
487
Our community member from Japan, Yutaro Ogawa, shares the PyTorch Tutorials in Japanese. コミュニティメンバの小川さん（電通国際情報サービスISID AIトランスフォーメーションセンター）@ISID_AI_team より、日本語版PyTorchチュートリアルが公開されました.
1
112
478
Tensor Comprehensions are now integrated and interoperable with @PyTorch . Read our blog post to get started:
Tensor Comprehensions: einstein-notation like language transpiles to CUDA, and autotuned via evolutionary search to maximize perf. Know nothing about GPU programming? Still write high-performance deep learning. @PyTorch integration coming in <3 weeks.
2
197
459