I experimented with DomoAI to modify the style of a video.
I used small clips of a Tamil song and transformed to Anime style.
I am surprised with how nicely it transformed the original video and also interestingly, it retained the text.
Song: Maari 2 - Rowdy Baby (Wunderbar
Same prompt -- Six AI image generators:
Midjourney
Adobe Firefly
Leonardo
Dalle3
Clipdrop
Imagine with Meta AI
10 examples:
1/10
Prompt: 1960s movie still two soldiers talking in a busy street
Dalle3 + Runway
Two Steps:
1) Image: Dalle3 on ChatGPT
Prompt: wide image, 5 shirts hanging on a clothes line with one letter written on each "h" "e" "l" "l" "o"
2) Image to Video: Runway
Image + General Motion 1 only, no text prompt
Image to Video :
- Stable Video Diffusion (SVD)
- Runway
- Pika Labs
Images: Midjourney
Notes:
1) I used Stable Video Diffusion (SVD) on Replicate.
2) I tried a few times in each of the platforms and picked the result I liked the most.
3) By experimenting with different
How to blend images in Dalle3:
Two steps:
1) uniquely identify the images
2) Ask to blend the images
Step 1: How to uniquely identify the image in the chat window?
One method: by adding custom instructions
"In dalle3 always assign a unique identifier to each image. display
runway gen3
top view timelapse video of a pencil artwork drawn by a hand, it is an art of a girl with rabbit hair from beginning till the finish
*rabbit hair is a typo --I intended rabbit like ears, but gen3 got it
Dalle 3
Prompts:
1) embed text 'Grateful' as froth on top of milk, wide image
2) embed "Not alone" text on misty glass with a scary hand print, print wide image
(Inspired by Gothika)
3) embed text "hello" as pink clouds on sky, wide image
4) embed "halloween" text as neon
Image to Video:
- Pika Labs
- Runway
- Stable Video Diffusion (SVD)
Images: Midjourney
Notes:
For each platform, I tried few times, either by changing the settings or simply rerunning, and then chose one of the results.
For SVD, varying the motion value (Range: 1 to 255
Dalle3: One woman, different ages - Updated:
With the new changes to Dalle3 Seed feature, it's good to use "same seed" for ideal results.
The process is quite simple.
This is the entire chat request:
same woman, 4 different ages, 5 years, 20 years, 40 years and 60 years.
don't
Dalle3: one woman, many expressions
Steps:
1) First, I asked for image of a woman:
My prompt:
close up symmetrical photo of a middle aged chinese woman in her home garden, facing the camera, wide image
2) Then I asked for different expressions:
I like
#1
, can you create the
Dalle3 on Bing
Prompt:
top view of a snow covered ocean with opening shaped like something*, with real whales swimming
* Australia, Africa, Srilanka, Heart
Please note - some shapes work well and some don't.
Dalle3 + Pika Labs
Image: Dalle3
Illustration of the 3D Pixar-style animated couple engaged in a lively conversation. They are sitting on a park bench with trees and birds in the background. The woman is gesturing with her hands as she speaks, and the man is listening intently,
AI singing video
SunoAI, Midjourney, D-ID
Inspired by AI singing videos with lip-sync, I decided to give it a go.
3 steps:
Step1: Song creation
I used Suno AI which is a text to song tool for the first time, it generates songs based on text prompts.
Step2: Image creation
Dalle3 + PikaLabs
Steps:
1) Image creation: Dalle3 (on Bing)
Prompt:
a large flag flying, with something* logo on it
*burger, ice cream
2) Image to Video: PikaLabs
/animate + image only
(just repeated a few times and picked the result I liked)
Dalle3 Seed Number:
It appears Dalle3 now has a functional seed number.
Thanks to
@ai_for_success
for this discovery.
Previously, Dalle3 would always return a default seed# of 5000 for all the images, which wasn't helpful.
Now it returns a large number.
What is a seed#?
Dalle3 - Same character, different poses:
To a certain extent we can have the same character do various activities by simple chat requests.
First created the image with this prompt and then asked to make her sit, eat ice cream and close eyes.
The background and clothing are
[NEW] Runway Multi-Motion Brush
Gen-2 Motion Brush has got a new update called Multi Motion Brush.
It is now possible to paint up to 5 unique subjects or areas within your input image to control their motion.
This is an upcoming update, currently in early beta access.
I chose
Stable Video Diffusion (SVD) - Image to Video
I varied the motion values for the same input image.
Range: 1 to 255 Default: 127
Few observations:
1) Varying motion values has significant effect on the results: higher motion values lead to more movement overall, and also in
Google Deepmind launches Imagen 2.
I tried their prompts on Dalle3, Midjourney, and Imagine with Meta AI.
6 examples.
1/6
Prompt: A shot of a 32-year-old female, up and coming conservationist in a jungle; athletic with short, curly hair and a warm smile
Imagen 2 is not
Dalle3 Pixar-style couple dancing
Prompts:
1) 3D Pixar-style animation of a diverse couple dancing together in an open space. The woman, of Caucasian descent, is spinning joyfully, while the man, of South American descent, holds her hand and smiles.
2) 3D Pixar-style animation
Google VideoPoet, Runway, Pika & Genmo
Google recently announced Video Poet.
Google's VideoPoet is a large language model (LLM) that is capable of a wide variety of video generation tasks, including:
- text-to-video
- image-to-video
- video stylization
- video inpainting and
Midjourney + Runway
Midjourney prompt:
water flowing from tap --ar 2:3 --style raw
Runway:The image was the input; General Motion 5 was used, no text prompt.
Part 2: Google VideoPoet, Runway, Pika & Genmo
Image to Video.
In a previous post, I did text-to-video for these tools.
In this post, it is Image-to-video.
8 examples
1/8
A woman yawning
Emboss using Dalle3:
Thanks to
@PhiloYup
for Emboss token.
Prompts:
1) emboss text "soap" on a soap bar, product photography, flowers background, wide image
2) emboss text "chocolate" on a chocolate bar, product photography, wide image
3) emboss text "luxury" on a leather
Adobe Firefly structure reference:
I created these images using my kid's art as reference + text prompts like these:
- red aeroplane toy made with felt, appliqué stitch, clouds, blue background
- broken ship, flowing paint from a palette of yellow and green colors
Kept the
[New] Adobe Firefly Structure Reference.
First try
"Structure Reference enables you to easily apply the structure of an existing image to newly generated images."
I created an image with my name and then used that as Structure Reference for creating the images with simple
Aging in reverse
Dalle3 + Runway
Process:
Step 1:
Generate images using Dalle3.
I asked for the same individual at ages 10 to 100 in intervals of 10.
Given that it can generate only 4 images at once, it will ask to proceed.
Upon confirming with "yes," it will generate the
Dalle3
Prompt: 4 frame story board pencil illustration of a old man and squirrel
I really love what Dalle3 came up with for this prompt. I find it so deep and meaningful.
[NEW] Midjourney - Style References
First try.
I used the original image as style reference and created the remaining images in similar style.
Original image was created previously using prompt:
side view of a toy bus with a vibrant art of a lion, product photography --v
Dalle3 on Bing - Finger print on misty glass
Prompts:
1) fingerprint mark of dragon logo on misty glass, real fire from its mouth
2) fingerprint mark of a bears face logo on misty glass
3) fingerprint mark of logo of lion on misty glass
4) fingerprint mark of elephant logo
Viggle: This time I introduced a new character into an existing dance video.
Steps:
1) Chose a stock video (dance clip with one subject, Pexels)
2) Created a character image in Midjourney (Prompt: t pose of a woman character with antlers, red boots, 3d animation style)
3)
An experiment with Viggle:
I followed the tutorial shared by
@MaxEinhorn
Thanks to
@francolli
@Uncanny_Harry
for the inspiration.
Steps:
1) Chose a video clip (from the recent Sam, Lex interview)
2) Extracted a frame (using capcut export still frame )
3) Cleaned up image
Mootion text-to- 3D motion
It's free, and available on Discord.
Prompt format:
/motion + Text prompt + Preferred Character + render motion in place (True or False)
I used Text prompts like walking, running, shocked, dance etc
Characters choices available are Bot, Cat,
Image to Video: Dalle3 + Runway
Dalle3 (on ChatGPT)
My Prompt:
burning fire creating the word "fire" wide image
Dalle3 prompt:
A wide-format, visually captivating image depicting the word 'Fire' created out of vibrant, realistic flames. The flames are intense and lively,
Mootion: motion to stylized video
1⃣ text-to-3D motion using Mootion
Prompt: walking --creativity 0.1 --character Bot --inplace True
2⃣Motion to stylized video using Mootion
Prompt: man walking, sunny beach --style Anime
Then I just tried to place the subject in a different
Stable Video Diffusion on Replicate - First try
I just uploaded an image and experimented with the settings. The result is two clips combined.
Please check the post for more info.
Stable Video Diffusion (SVD) is now on Replicate:
- Make 14 frame or 25 frame videos
- Use any image, it'll be resized to make the best video
- Control degree of movement
Great work putting this together
@lucataco93
, and
@StabilityAI
for the fantastic
Pika text to video:
Experimenting with extending a generation.
First created a video and then extended 4s once.
Few observations:
1) Negative prompts helps with better results.
2) When extending 4s the negative prompt did not automatically carry over, we need to add it again
Dalle3 - Consistent two characters.
These two results came from an unexpected discovery from a typo, I will explore the prompt more and share more details if the results are consistent.
Go Dalle3!
Consistent 4 characters in different settings:
My prompt:
create images of same four people in four different settings, create all images in same realistic photography style: a dad, mum and their two little boys, in park, in the car, in the beach, in the garden
Dalle3 - Consistent characters - Update:
These are the results I got today, so when the results come in the grid layout itself the characters are very consistent.
Still so experimental and does not work all the time, just see it as a possibility for further exploration.
Dalle3 - Consistent two characters - Update:
Even after much effort i could not figure out a way to make this work consistently.
However, I will explain the process & learnings using the Mouse couple as example:
Step1: First created a set of mouse couple images
Prompt: cute
Pika 1.0 - Expand Canvas feature
I just got access to Pika 1.0
I created the first video using a text prompt, and then expanded the canvas few times to get the second video.
Initial prompt: A beautiful princess, standing on the castle wall, 3d animation
Pika 1.0 Text-to-video
Prompts:
1) a dancing galah, jungle background, 2d cartoon
2) a dancing giraffe, 2d cartoon
3) a dancing peacock, jungle background, 2d cartoon
4) a dancing lorikeet, kitchen background, 2d cartoon
(I flipped the last video to face the peacock on the
Dalle3 + PikaLabs
Image - Dalle3
Prompt: embed text "WOW" as clouds on sky
Image to Video - Pikalabs
/animate + Image + Prompt
prompt: time lapse of clouds moving gently in breeze -motion 2
Pika text-to-video: 15s video
First created the initial 3s video and extended 3 times using "Add 4s" feature.
Prompt: 1960s movie still two soldiers talking in a busy street, 4k, highly detailed, cinema
Motion: 1
I used the same negative prompts and also pasted the same
Pika text to video:
Experimenting with extending a generation.
First created a video and then extended 4s once.
Few observations:
1) Negative prompts helps with better results.
2) When extending 4s the negative prompt did not automatically carry over, we need to add it again
Sora
"Connecting videos
We can also use Sora to gradually interpolate between two input videos, creating seamless transitions between videos with entirely different subjects and scene compositions. "
The video in the bottom interpolates between the videos on the top.
Source:
Dalle3
First I created an image and then asked Dalle to change the lip colour of that image many times.
Prompt:
Black and white photo showcasing a breathtaking closeup view of a fashion model's face. The focus is on the eyes, lips, and contours of the face, highlighting the
Sora: How is this not real? 🤯
a woman wearing purple overalls and cowboy boots taking a pleasant stroll in Johannesburg, South Africa during a winter storm.
more examples:
Thanks to
@Mr_AllenT
for such a great idea and inspiration.
It turns out the silhouette works with text as well.
Sample prompt:
Illustration of the uppercase letter 'H' silhouette densely filled with vibrant and detailed images of pumpkins in various sizes and types, all
Let's create filled silhouettes 🔥
DALLE3 is incredible at understanding complex instructions. With this prompt, you can fill one object with multiple other objects 🖤
Prompt: Create a minimalist cartoon illustration of a prominent [Subject] silhouette with no outline. The
Runway Gen2 - Seed exploration:
For Image to Video
**Fixing the same seed of a result can help with further adjusting the intensity of motion for similar movement.**
I have tried this approach:
Round 1: I varied the general motion from 1 to 10 without a fixed seed & chose
Dalle3: How to reference previous generations within the same chat window?
n, n-1, n-2 generation etc works
This is how I tested:
1) create an apple
2) generate images of honey bee
3) generate images of a lion
4) can you just add text 'apple' to the
#1
of n-2 generation
An experiment with Viggle:
I followed the tutorial shared by
@MaxEinhorn
Thanks to
@francolli
@Uncanny_Harry
for the inspiration.
Steps:
1) Chose a video clip (from the recent Sam, Lex interview)
2) Extracted a frame (using capcut export still frame )
3) Cleaned up image
Midjourney vs Dalle3 vs Adobe Firefly vs Leonardo
Inspired by
@Marcel_A_Buth
's post I tested his prompt on these four platforms:
Photo of an elderly man from Siberia with a full beard during a cold day. The sunlight beams onto his face, emphasizing the ice that has formed from
Leonardo AI: Image2Motion
- First, created images using text prompts and then using Image2motion feature to create the videos.
- I used 3D animation style model for the images
- and used simple text prompts:
1) a cute little girl in uniform, in school bus
2) moving train
3)
Candle
Midjourney prompt:
a floral candle --ar 2:3 --style raw
(I did a pan up for some space on top)
Pika Labs Prompt:
timelapse candle burning -motion 0 -camera zoom in -gs 8
Finally upscaled video 2x in capcut