Style transfer without a specific style image. I minimize only the content loss from neural style transfer and directly optimize brushstrokes via backpropagation. In this case, "style" is an intrinsic property defined entirely by the artistic medium.
I am on an H-1B, in the process of getting my green card and relocating my family to the US.
Me and many other colleagues in a similar situation have signed this letter.
I do not know what will happen next, but I am confident we will be taken care of.
the board should resign.
New blog post: Neural Style Transfer with Adversarially Robust Classifiers
I show that adversarial robustness makes neural style transfer work by default on a non-VGG architecture.
Blog:
Colab:
At
#NeurIPS2019
, talking to a guy from DeepMind about how I can't reproduce their results because it's so resource intensive.
"It's not too bad actually, we only use 16 GPUs, it's not like we use a cluster of TPUs or anything."
Some personal news: I moved to San Francisco and joined the OpenAI Fellows Program!
Super excited to learn from all the amazing people here. It's a dream come true to be able to do machine learning full-time!
My work is out on arXiv!
Neural Painters: A learned differentiable constraint for generating brushstroke paintings:
There are also
@GoogleColab
notebooks so you can reproduce all the experiments yourself:
Sharing my winter break project. I tried combining
@hardmaru
's world models and
@yaroslav_ganin
's SPIRAL to see if an agent can learn to paint inside its own dream. It can! These strokes are generated purely inside a world model, yet transfer seamlessly to a real paint program.
New blog post: Solving simple probability problems with intermediate steps and an external calculator
Link:
A transformer is used to map questions to intermediate steps, while an external symbolic calculator evaluates intermediate expressions.
I built a demo for arbitrary image stylization in the browser using
@TensorFlow
.js at . Instead of having a single network per style like previous algorithms for fast style transfer, only a single model is used for all style images. more details in replies
I haven't tweeted much about ML in a ~year, but happy to finally share what I've been working on with amazing collaborators - we trained GPT-3 to use a text browser!
We trained a research version of GPT-3 that can search the web, synthesize information, and cite its sources to provide more accurate answers to questions.
Not sure why this hasn't gotten more attention. Seems like a viable alternative to Colab that addresses a lot of my pain points e.g. no random restarts, easy upgrade to more powerful GPU, actually uses Jupyter UI instead of some weird closed-source fork
Directly optimizing brushstrokes using backpropagation makes it very easy to find a sequence that "paints" an image.
Ironically, the almost perfect reconstruction makes the end result feel less creative and more "robotic". 🤔
""ML frameworks don’t just enable machine learning research, they enable and restrict the ideas that researchers are able to easily explore.""
Perhaps the next big step forward is something completely inexpressible in the TF/PyTorch paradigm.
Today I had the opportunity to listen to Rich Sutton as he spoke to a few of us at the
#NeurIPS2019
RL Social.
He was incredibly nice and humble, patiently listening to all questions and responding thoughtfully.
Very short thread of his insights (sadly I arrived quite late)
The Joy of Neural Painting ()
Someone wrote an implementation of Neural Painters using PyTorch and .
They use NoGAN by
@citnaj
to speed up training of the GAN-based Neural Painter. Very cool work!
One of the most underrated
#Pandas
functions in
#Python
is `.query()`. I use it all the time.
data = data.query('age==42')
looks so much nicer than:
data = data[data['age'] == 42]
And it allows chaining like:
data = data.query('age >18').query('age < 32')
Happy to announce that this model has made its way into magenta.js! Now anyone can use this model in their own apps with a few lines of code. Also wrote a blog post about my experience porting the model!
magenta-js lib:
Blog post:
In any case, this is another followup to my work on neural painters on which I am currently sprinting as fast as I can to write a paper for. All code will be open-sourced as
@GoogleColab
notebooks.
Sharing my winter break project. I tried combining
@hardmaru
's world models and
@yaroslav_ganin
's SPIRAL to see if an agent can learn to paint inside its own dream. It can! These strokes are generated purely inside a world model, yet transfer seamlessly to a real paint program.
Want to learn how to deploy your Keras model in the browser? Rei
@ReiiYoda
and Kai are holding a
@__MLT__
workshop on TensorFlow.js on April 6 at Google Japan. Join us!
I personally think content loss is the best idea from the neural style transfer paper. We can preserve human-recognizable high level structure from a target image, without caring about exact pixel matching.
MNIST was easy, so I tried KMNIST. Simple chars like つ and ハ were easy but overall, the dataset is much harder😅. I'm confident it can be improved with more strokes + training steps, though. Stroke order is also all over the place.
@tkasasagi
@mikb0b
@hardmaru
@KitamotoAsanobu
At the end he smiled and said
"You guys have worn me out. It was great talking to all of you. It's nice to get a perspective of where the field is at." (from the person who basically invented the field)
Then we took this selfie.
Interesting
#NeurIPS2019
paper
Putting An End to End-to-End:
Gradient-Isolated Learning of Representations
They train a neural net without end-to-end backprop and achieve competitive performance.
Extremely grateful to
@distillpub
and
@ch402
for letting me be a part of their first experimental "discussion article"! Had a lot of fun interacting with the paper authors.
I think it's been a successful experiment so far. Looking forward to more of these in the future!
Happy Friday, Runway 0.5.9 is out!
New export and network panels and 3 new image-to-image models 🖼️➡️🖼️: CycleGAN, Pix2Pix and
@citnaj
’s DeOldify. All ported by
@ReiiYoda
!
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo.
We are collaborating to figure out the details. Thank you so much for your patience through this.
A single robustly trained classifier is all you need to perform image generation, image-to-image translation, inpainting, ML-assisted paint (and more)! Blog post (w/ paper and notebooks): (w/
@ShibaniSan
@tsiprasd
B. Tran
@andrew_ilyas
@logan_engstrom
)
I find the most amazing thing about this is how quickly it converges. The original SPIRAL paper shows it converges in 50M steps (using multiple multi-CPU/GPU instances). The MNIST-trained agent I show here used 11K steps and took ~10 hours to train on a free
@GoogleColab
GPU.
@gwern
Yup. One thing I want to try for the future is move from brushstrokes to something like paint splatters maybe? Anyway here's a grayscale one I tried real quick. As for regular style transfer, I did try but the results were just okay, not good enough to replace the original yet
Anyway, I have to state here that I was not the first to think of this. While planning this tweet thread, I decided to look at some citations of the World Models paper and saw . Apparently, I am some months late to the party 😅
I also trained a full color 15-stroke agent on CelebA. The outputs seem on par with SPIRAL (orig. version). Unlike SPIRAL, my agent doesn't make strokes that are fully covered by future strokes. I attribute this to the ease of credit assignment when full gradients are available.
@jeremyphoward
@github
Nice! I code for a living and wish I had this guide before starting my github blog. I blog like twice a year and everytime I do I have to relearn ruby and jekyll and all the little idiosyncrasies of a template I chose over a year ago.
My OpenAI fellowship has changed my life in so many positive ways! I encourage anyone interested to apply. It's a great way to get into ML, especially for folks from non-traditional backgrounds!
We're excited to announce the OpenAI Residency! This new program offers a pathway to a full-time role at OpenAI for researchers and engineers who don't currently focus on AI.
The program will focus on recruiting from underrepresented groups in technology.
I've turned into someone who prefers strongly-typed programming languages and am not sure if Google did this to me or if I've always secretly been this way
On theoretical guarantees:
"I don't care about theoretical guarantees. Show me the empirical results. Throw the chicken into the engine"
Presumably referencing the chicken gun used to fire chickens to test a jet engine's robustness
Python was the perfect beginner programming language for me. I couldn't understand memory management in C/C++, and Java's static types intimidated me.
Thank you Guido, and congrats!
It's bittersweet: I'm leaving
@dropbox
, and am now retired. I've learned a lot during my time as an engineer here -- e.g. type annotations came from this experience -- and I'll miss working here.
Had fun today visiting the Hokusai exhibit at Mori Art Museum. It was so crowded I arrived 10 minutes before opening but still had to wait 30 minutes to get in. Worth it!
@citnaj
LOL this is amazing. Basically what they want is coauthorship of Deoldify for the hard, hard work of entering it in a contest and describing it (apparently wrongly) in a pdf.
Adversarially robust models are going to be big for machine learning art.
Particularly interested in its effect on neural style transfer. Will better high level features result in better outputs?
Robustness goes beyond security: Representations induced by robust networks can align much more closely with human perception, and enable simple feature visualization and manipulation. See: (w/
@logan_engstrom
@andrew_ilyas
@ShibaniSan
@tsiprasd
B. Tran)
At
@EmbarkStudios
we're experimenting with a new physically-based animation system using reinforcement learning. No manual animation was involved making these robots move. Thanks to using physics, the 100 ton robot learns a heavier, more lumbering gait than the smaller robot.
Approach motivated by how a student might solve math problems: by setting up intermediate equations, using a calculator to solve them, and using those results to construct further equations.
The approach is simple. I train a world model that can predict the output stroke of a real paint program for a corresponding action, in effect creating a fully differentiable paint program. I then train an agent to paint using the same adversarial method detailed in SPIRAL.
Adversarially robust models are going to be big for machine learning art.
Particularly interested in its effect on neural style transfer. Will better high level features result in better outputs?
@GoogleColab
For anyone who wants to know more about this project, I wrote a blog post detailing my motivations, approach, and future work in this direction I'm excited about.
@jm_alexia
It might still be useful to put it out there! There might be insights that others can build on. I write blog posts for things I don't think are "paper-worthy".
I concur with this blog post. Independent research is not a good experience. The psychological toll failures take on you (rejection, failed expts, getting scooped) are magnified N times without other people to share it with.
After getting published in ICLR as an Independent Researcher, I have received nearly 100 messages from others who are looking to do the same. So I wrote a blog post on why I decided to do it and my advice to others.
@hardmaru
Seems like a perfect candidate for porting to the browser (maybe minus the glove embedding). The models are already in magenta.js.
Anyway, awesome job
@ljvmiranda921
@thinkdatasci
!
His top 5 most important problems in RL
1. Improving core RL algorithms. They should learn better.
2. State construction
3. Planning
He couldn't remember the other two 😁