I've been thinking of integrating pngquant as an ffmpeg filter, it would make it possible to generate even better pallettes. That would get ffmpeg on par with gifski.
The one good usecase I've found for AI chatbots, is writing ffmpeg commands. You can just keep chatting with it until you have the command you need. Some of them I save as an executable .command, or in my .txt note.
As pessimistic about it as I am, I do think LLMs have a place in helping people turn their text description into formal directives. (Search terms, command-line, SQL, etc.)
... Provided that the user sees what's being made for them and can confirm it and (hopefully) learn the target "language."
I agree apart from the learning part. The thing is unless you have some very specific needs where you need to use ffmpeg a lot, there’s just no need to learn this stuff. If I have to touch it once a year I have much better things to spend my time learning than ffmpeg command
But doesnt something like this interface kind of show the inefficiency of this? Like we can all agree ffmpeg is somewhat esoteric and LLMs are probably really great at it, but at the end of the day if you can get 90% of what you need with just some good porcelain, why waste the energy spinning up the GPU?
Because the porcelain is purpose built for a specific use case. If you need something outside of what its author intended, you'll need to get your hands dirty.
And, realistically, compute and power is cheap for getting help with one-off CLI commands.
I have a little script that I use on the CLI to do this kind of stuff (calls an LLM to figure out how to do CLI stuff) but you can just as easily now use any of the coding agents.
interesting approach, i solved similar problem by creating visual tool to generate ffmpeg commands but its not the same(it cant do conversion etc.)
I like that you took no AI approach, i am looking for something like this i.e. understanding intent and generating command without using AI but so far regex based approaches have proved to be inadequate. I also tried indexing keywords and creating index of keywords with similar meaning that improved the situation a bit but without something heavy like bert its always a subpar experience.
That's the problem ideally solved by typed data, i.e., some UI where instead of trying to memorize whether it's thumb/s/nails you can read the closed list of alternatives, read contextual help and pick one
Yeah, no, that's a pale imitation that only addresses the one specific example given. But, like, how would you even know what target formats are supported? Break the flow and look it up or simply read the drop-down list?
The free type-any-text interface with poor helpers is the worst in accessibility
Which format is the default if no argument is given?
Or more complicated contextual knowledge - if you cut 1sec of a video file, does fish autocomplete to tell you whether the video is reencoded or cut (otherwise) losslessly
Very cool idea since ffmpeg is one of those tools that has a few common tasks but most users would need to look up the syntax every time to implement them (or make an alias). In line with the ease of use motivation, you might consider supporting tab completion.
I was surprised that macOS (QuickTime/Preview, iMovie) can't read .mp4 files. Not sure if it was due to H.265 or the audio codec. I tried using ffmpeg to convert to .mov but that also failed to open, since I guess MOV is just another container format.
MP4 is container, not format, so if you have unsupported format packed into MP4 container it won’t be played. Example is trying to play AV1 video codec on devices with M2 chip or older. It won’t play. But it will play on devices with M3 chip and newer. Easiest solution is to use other player so that you can watch any MP4 file but with software decoding where hardware decoding is not available. Examples of such players are MPV or VLC.
Sometimes an idea comes along thats so obvious it makes me angry. I have been struggling with ffmpeg commands for over well a decade. All the time I wasted googling and creating scripts so I wouldn't have to regoogle and this could have existed literally from day one
When converting video to gif, I always use palettegen, e.g.
See also: this blog post from 10 years ago [1][1] https://blog.pkh.me/p/21-high-quality-gif-with-ffmpeg.html
I've been thinking of integrating pngquant as an ffmpeg filter, it would make it possible to generate even better pallettes. That would get ffmpeg on par with gifski.
The one good usecase I've found for AI chatbots, is writing ffmpeg commands. You can just keep chatting with it until you have the command you need. Some of them I save as an executable .command, or in my .txt note.
As pessimistic about it as I am, I do think LLMs have a place in helping people turn their text description into formal directives. (Search terms, command-line, SQL, etc.)
... Provided that the user sees what's being made for them and can confirm it and (hopefully) learn the target "language."
Tutor, not a do-for-you assistant.
I agree apart from the learning part. The thing is unless you have some very specific needs where you need to use ffmpeg a lot, there’s just no need to learn this stuff. If I have to touch it once a year I have much better things to spend my time learning than ffmpeg command
The "provided" isn't provided, of course, especially the learning part, that's not what you'd turn to AI for vs more reliable tutoring alternatives
One that older AI struggled with was the "bounce" effect: play from 0:00 to 0:03, then backwards from 0:03 to 0:00, then repeat 5 times.
But doesnt something like this interface kind of show the inefficiency of this? Like we can all agree ffmpeg is somewhat esoteric and LLMs are probably really great at it, but at the end of the day if you can get 90% of what you need with just some good porcelain, why waste the energy spinning up the GPU?
Requiring the installation of a massive kraken like node.js and npm to run a commandline executable hardly screams efficiency...
Because the porcelain is purpose built for a specific use case. If you need something outside of what its author intended, you'll need to get your hands dirty.
And, realistically, compute and power is cheap for getting help with one-off CLI commands.
Can't access the githup repo https://github.com/josharsh/ezff
I have a little script that I use on the CLI to do this kind of stuff (calls an LLM to figure out how to do CLI stuff) but you can just as easily now use any of the coding agents.
interesting approach, i solved similar problem by creating visual tool to generate ffmpeg commands but its not the same(it cant do conversion etc.)
I like that you took no AI approach, i am looking for something like this i.e. understanding intent and generating command without using AI but so far regex based approaches have proved to be inadequate. I also tried indexing keywords and creating index of keywords with similar meaning that improved the situation a bit but without something heavy like bert its always a subpar experience.
That's the problem ideally solved by typed data, i.e., some UI where instead of trying to memorize whether it's thumb/s/nails you can read the closed list of alternatives, read contextual help and pick one
This is why we have fish tab completions. Does exactly that; list of possible commands with contextual help. Fish rules.
Yeah, no, that's a pale imitation that only addresses the one specific example given. But, like, how would you even know what target formats are supported? Break the flow and look it up or simply read the drop-down list? The free type-any-text interface with poor helpers is the worst in accessibility
Which format is the default if no argument is given?
Or more complicated contextual knowledge - if you cut 1sec of a video file, does fish autocomplete to tell you whether the video is reencoded or cut (otherwise) losslessly
Also, what does fish complete to on Windows?
Somehow it seems ffmpeg has become the "Can it run crysis" of UX design
Very cool idea since ffmpeg is one of those tools that has a few common tasks but most users would need to look up the syntax every time to implement them (or make an alias). In line with the ease of use motivation, you might consider supporting tab completion.
GitHub repo link returns 404.
I was surprised that macOS (QuickTime/Preview, iMovie) can't read .mp4 files. Not sure if it was due to H.265 or the audio codec. I tried using ffmpeg to convert to .mov but that also failed to open, since I guess MOV is just another container format.
Is there an easier way?
IMHO the de-facto video player for macOS is [IINA](https://iina.io/).
That exists, but it’s still VLC.
It's based on mpv, not vlc.
MP4 is container, not format, so if you have unsupported format packed into MP4 container it won’t be played. Example is trying to play AV1 video codec on devices with M2 chip or older. It won’t play. But it will play on devices with M3 chip and newer. Easiest solution is to use other player so that you can watch any MP4 file but with software decoding where hardware decoding is not available. Examples of such players are MPV or VLC.
vlc
Sometimes an idea comes along thats so obvious it makes me angry. I have been struggling with ffmpeg commands for over well a decade. All the time I wasted googling and creating scripts so I wouldn't have to regoogle and this could have existed literally from day one