You could probably make your infra cheaper if you sent the generated slides to the web client and use some js ffmpeg wasm to compile the video in the browser. This way you could probably use the cheapest VPS to handle this.
For example, I have several posts that explain concepts in detail, with supporting images. They're written in Markdown, and I keep refining them until I'm confident that the average person can read them and immediately understand the concept.
The problem is that text is becoming a less popular medium for learning, while short form videos are increasingly being used to explain the same ideas.
From what I can tell, this tool only lets you provide a prompt. Can it turn my existing Markdown posts and images into videos, or is it mainly intended for generating videos from simple prompts? I'm looking for something that preserves the explanations I've already written rather than creating something entirely new.
These are the two styles that consistently keep me watching. I can't really describe them well in words, so I linked the videos directly to avoid losing important details in the translation from visual format to text.
Maybe someone with more experience in video production can explain what techniques are being used here and why they're so effective at holding viewers' attention.
They cut to the next scene a moment before the next sentence, and they cut to the next sentence before the last word finished (i.e. no periods between sentences).
So this is really just a kinda proof of concept, but I'd be pretty easy to turn it into what you want. It sounds like you've already got both basically a script and images to go with it?
Your project reminds me of something I built a while ago using Pollination.ai for images, Gemini Flash for script / captions / inserting transitions and all other editing, Google Cloud TTS for narration/voice, and FFmpeg drive by flash, back then Gemini Flash 2.5 (free) is what i used for these experiments. The result looked like this:
I wasn't happy with how it turned out, so after making 3–4 videos I dropped the idea.
Later, I came across several different video styles that consistently kept me watching. That's when I realized what my videos were missing they simply didn't have the same ability to capture and hold attention.
But i never managed to generate those styles after dozens of attempts and gave up.
Fun idea! Hope it doesn't cost you too much to keep up depending on how much traction this thread gets ;)
One small quirk I noticed with the generated short video: for each new slide/photo, there is a very gradual "zoom in" effect that occurs, which causes text to jitter a bit and is a bit disorienting. Not sure if that is specific to the "The Great Molasses Flood of 1919 in Boston" prompt I used or not.
Also, is Claude involved in the video generation itself? I noticed the generated video has "By Claude" in the top left corner, but the logs seem to indicate that only GLM-5.2, Gemini Flash, and GPT-4o are involved (just curious!)
Yeah, that's suppose to be a Ken Burns style zoom but I haven't figured out how to stop the shake the should probably just remove it. And I need to remove the "By Claude" too. That's a remnant from porting from my internal framework for the [By Claude)(https://www.youtube.com/@byclaude) Youtube, TikTok, etc channels.
Cool idea! Looks like my prompt is stuck, though. Might be an idea to provide a share link at the moment of hitting the button so you can check back later, because if I close the tab I will lose my prompt
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(self.args, *self.kwargs)
File "/home/ubuntu/byclaude/video/harness/pipeline/pipeline.py", line 829, in build_clip
clip = round(dur + TAIL, 3)
TypeError: unsupported operand type(s) for +: 'NoneType' and 'float'
Video build failed (pipeline exit 1) — please try again.
Powered by AI · by Claude · ~
I'm guessing its being hugged now - it says generate in under a minute but my short is taking ten minutes and counting; anyway I like the simplicity and am interested in seeing the results - though I do think it is quite depressing as we will see this sort of service mostly put to use for slop.
It’s an interesting concept, perhaps it could go more from ”narrated slide show” to ”video clip” in the near future if it doesn’t cost too much.
Yeah I'd been thinking of Gemini Omni for that but it adds about 10 cents per second of video, so not cheap.
You could probably make your infra cheaper if you sent the generated slides to the web client and use some js ffmpeg wasm to compile the video in the browser. This way you could probably use the cheapest VPS to handle this.
How would I create a video like this?
For example, I have several posts that explain concepts in detail, with supporting images. They're written in Markdown, and I keep refining them until I'm confident that the average person can read them and immediately understand the concept.
The problem is that text is becoming a less popular medium for learning, while short form videos are increasingly being used to explain the same ideas.
From what I can tell, this tool only lets you provide a prompt. Can it turn my existing Markdown posts and images into videos, or is it mainly intended for generating videos from simple prompts? I'm looking for something that preserves the explanations I've already written rather than creating something entirely new.
it works, but can u add different style of videos like this one:
Style 1: https://www.youtube.com/shorts/vGjrv2zjfsM
Style 2: https://www.youtube.com/shorts/6KoRHpAZ4zk
These are the two styles that consistently keep me watching. I can't really describe them well in words, so I linked the videos directly to avoid losing important details in the translation from visual format to text.
Maybe someone with more experience in video production can explain what techniques are being used here and why they're so effective at holding viewers' attention.
They cut to the next scene a moment before the next sentence, and they cut to the next sentence before the last word finished (i.e. no periods between sentences).
So this is really just a kinda proof of concept, but I'd be pretty easy to turn it into what you want. It sounds like you've already got both basically a script and images to go with it?
Your project reminds me of something I built a while ago using Pollination.ai for images, Gemini Flash for script / captions / inserting transitions and all other editing, Google Cloud TTS for narration/voice, and FFmpeg drive by flash, back then Gemini Flash 2.5 (free) is what i used for these experiments. The result looked like this:
https://www.youtube.com/shorts/IhGR8AC3a_g
Uploaded on sep 24, 2024.
I wasn't happy with how it turned out, so after making 3–4 videos I dropped the idea.
Later, I came across several different video styles that consistently kept me watching. That's when I realized what my videos were missing they simply didn't have the same ability to capture and hold attention.
But i never managed to generate those styles after dozens of attempts and gave up.
Fun idea! Hope it doesn't cost you too much to keep up depending on how much traction this thread gets ;)
One small quirk I noticed with the generated short video: for each new slide/photo, there is a very gradual "zoom in" effect that occurs, which causes text to jitter a bit and is a bit disorienting. Not sure if that is specific to the "The Great Molasses Flood of 1919 in Boston" prompt I used or not.
Also, is Claude involved in the video generation itself? I noticed the generated video has "By Claude" in the top left corner, but the logs seem to indicate that only GLM-5.2, Gemini Flash, and GPT-4o are involved (just curious!)
Yeah, that's suppose to be a Ken Burns style zoom but I haven't figured out how to stop the shake the should probably just remove it. And I need to remove the "By Claude" too. That's a remnant from porting from my internal framework for the [By Claude)(https://www.youtube.com/@byclaude) Youtube, TikTok, etc channels.
Cool idea! Looks like my prompt is stuck, though. Might be an idea to provide a share link at the moment of hitting the button so you can check back later, because if I close the tab I will lose my prompt
Yeah the disk on the instance filled up and it was down. Sorry. It's up again now.
Currently getting errors
---
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run result = self.fn(self.args, *self.kwargs) File "/home/ubuntu/byclaude/video/harness/pipeline/pipeline.py", line 829, in build_clip clip = round(dur + TAIL, 3) TypeError: unsupported operand type(s) for +: 'NoneType' and 'float' Video build failed (pipeline exit 1) — please try again. Powered by AI · by Claude · ~
Should be fixed now
I'm guessing its being hugged now - it says generate in under a minute but my short is taking ten minutes and counting; anyway I like the simplicity and am interested in seeing the results - though I do think it is quite depressing as we will see this sort of service mostly put to use for slop.
Hi! Sorry yeah, I'm been bumping up against rate limits, but it should be working again now!
I'm around to answer questions or receive feedback if anyone as any!
Please, pretty please, link some pre-generated examples.
On it!
https://instantvideos.org/video/6be78e5ea542
Quotations from Chairman Mao Tse-tung (Short form)
The audio narration is great and the documentary is accurate. Improvements could include more pictures or animations.
This is a very great tool and framework! Thank you so much.
No, thank you for trying it!
Ugh looks like I'm hitting my Gemini Tier 1 usage limits. Sorry!
Fixed!
Image not generating?
Working on it!
Fixed!
Did you really rent a 64 vCPU EC2 without exploiting ffmpeg GPU acceleration?
I'm no expert here and was working w/ GLM-5.2, but whatever rendering we were doing ended up being CPU bound.