I worked on an interesting use case in generative AI that is creating publish-worthy short-form videos (YouTube shorts/ Instagram Reels) programmatically in one click.
Full code along with an easy-to-run Google Colab notebook is provided.
Script + Image Descriptions : ChatGPT API Image generation with SDXL 1.0: Segmind .com API Text to speech: ElevenLabs API Speech to text to get word timestamps: OpenAI's Whisper Overlay captions and stitch video: Moviepy (Python)
MoviePy is an incredibly powerful tool. However, for some of my projects, I kept running into performance bottlenecks, especially with compositions involving multiple layers, text overlays, and transformations.
So, I decided to build a solution for my own needs and wanted to share it with this community first to see if it might be useful to others. It's a performance-focused alternative called MovieLite.
The core idea was to keep a simple, chainable API similar to MoviePy, but to aggressively optimize the rendering pipeline using Numba (JIT compilation) for CPU-based tasks. The result has been pretty promising. These are my benchmark results for some examples (you can find them in my repo):
Task
movielite
moviepy
Speedup
No processing
6.92s
6.92s
1.00x
Video zoom (1.0x → 1.5x)
9.91s
31.44s
3.17x
Fade in/out
7.63s
8.44s
1.11x
Text overlay
8.98s
33.70s
3.75x
Video overlay
18.94s
72.44s
3.83x
Alpha video overlay
12.01s
40.31s
3.36x
Complex mix*
39.66s
166.88s
4.21x
Overall
104.04s
360.12s
3.46x
*Complex mix includes: video with zoom + fade, image clips with fade, text overlay, video overlay - all composed together.
Now, to be clear, this is not a full replacement for MoviePy. My focus was on making the common tasks significantly faster.
I'm still early in the journey, but it's reached a point where it's useful for my own projects. I'd genuinely love to get some feedback from MoviePy users like you.
Is this something that could be useful in your workflows?
Are there any obvious pain points or missing features that feel essential?
After seeing some people struggle with generating fancy and/or animated text clips (and wanting to produce some of my own), I've started putting together a system to do just that.
I've named it "camala" for "CAption MArkup LAnguage". In short, you specify the contents, look and feel and animations (with support for easing) of text elements in a .toml file and the system generates a high-quality moviepy VideoClip (or a .gif or .mp4) from those specifications.
The documentation contains some installation instructions as well as many examples to get you started.
How it works you ask? It generates .svg for every video frame, and then uses inkscape to render those .svg strings to .png frames and moviepy to collect all those frames into a VideoClp. It's not very fast (on my laptop running linux I can render about 5-6 fps), but produces very high quality clips. With inkscape as backend, the system supports many things that were not possible with e.g. the pango markup language that you could already use from moviepy today (like making outline text). Needless to say, to use the system you also must install inkscape from https://inkscape.org or using your favourite package manager.
This is alpha-quality software at best, and it has some limitations, but I've already used it to generate some animated text clips for a video of my own so it definitely works for me.
I'm working on a python script that takes a picture and a music file and create a video file using MoviePy library. Unfortunatly, I'm facing a problem I'm not able to resolve. When I try to define the AudioFile I get this error :
TypeError: 'float' object cannot be interpreted as an integer
I'm fairly new to Python so if someone could help me sort this problem it would be great :)
Here's the full error:
File "movietest.py", line 5, in <module> clip = clip.set_audio(AudioFileClip("music.mp3")) File "C:\Users\Julien_Dev\AppData\Local\Programs\Python\Python35-32\lib\site-packages\moviepy-0.2.2.11- py3.5.egg\moviepy\audio\io\AudioFileClip.py", line 63, in init buffersize=buffersize) File "C:\Users\Julien_Dev\AppData\Local\Programs\Python\Python35-32\lib\site-packages\moviepy-0.2.2.11-py3.5.egg\moviepy\audio\io\readers.py", line 70, in init self.buffer_around(1) File "C:\Users\Julien_Dev\AppData\Local\Programs\Python\Python35-32\lib\site-packages\moviepy-0.2.2.11-py3.5.egg\moviepy\audio\io\readers.py", line 234, in buffer_around self.buffer = self.read_chunk(self.buffersize) File "C:\Users\Julien_Dev\AppData\Local\Programs\Python\Python35-32\lib\site-packages\moviepy-0.2.2.11-py3.5.egg\moviepy\audio\io\readers.py", line 123, in read_chunk self.nchannels)) TypeError: 'float' object cannot be interpreted as an integer
I'm an experienced dev but new to video processing and I don't usually use Python. I need to be able to quickly process large volumes of 30 sec videos doing things like adding logos, concatenating and combining clips, and overlaying titles and graphics.
Functionally MoviePy seems to work well, it has all the features I need but it seems incredibly slow. e.g. in one of my tests, running on a 2016 MacBook Pro (2.9 GHz Core i7, 16GB RAM) it's taking around 258 seconds to process and write a 30 second video with just a few of the modifications I mentioned.
I've tried running it on several different types of AWS EC2 instances (Compute optimised and GPU instances) but it's actually considerably slower than on my laptop (360-380 seconds).
So my question is, what are the considerations for MoviePy rendering performance? What hardware/platform/instance type does it run fastest on? Are there techniques for speeding it up (or things I should avoid that might be slowing it down)?
Alternatively are there any other libraries (any language) or services that might be able to handle this task quicker than MoviePy?
Just wanted to say that moviepy has been a godsend for dumb creative projects for me. A lot of answers to the problems I ran into I found in this subreddit, so thanks!
Hello everyone, I was wondering if anyone knows if and how it is possible to get a glow like on the first picture on my text? I also wanna know how to make a background behind a text with rounded corners like in the second picture
Making these meme covers is a pain in the ass if you do it full manual. I tried using PyAutoGUI and Mido to control my mouse and keyboard, but it wasn't fast enough, so I tried looking into FFMpeg. Learning FFMpeg syntax and how it works is difficult, so I discovered MoviePy, and I decided to try it out. After a few experiments here and there it turns out that around 70% of the work in these memes could be fully automated using MoviePy so all I have to do is arrange these clips in Premiere Pro. Here's the code on PasteBin: https://pastebin.com/3MUi0vGt
Hey everyone! I've been working on an exciting project where I utilize a combination of Python libraries and AI technologies to generate automated music animations, and I thought I’d share my journey and some insights here.
1. The Core Idea: My main goal was to create an automated system that generates engaging music animations for YouTube Shorts. I wanted these animations to not only visualize the audio but also be visually appealing to enhance the overall viewer experience.
2. Technologies Used:
Moviepy: This library has been instrumental in handling video file operations. It’s great for setting up the video structure—everything from resolutions to concatenating audio and video.
Stable Diffusion: I used this to create dynamic and visually captivating backgrounds for each music video. By inputting specific prompts, I could generate unique artwork that reflects the mood and themes of the music tracks.
OpenCV: This tool came in handy for generating 'audio waves' that react to the music in real-time. It allowed me to create a visual representation of the music's dynamics, adding a layer of depth to the animations.
Suno AI: This is where the magic happens with the music. Suno AI analyzes the text automatically classifying them into different genres and moods to create music. but I did classified using some condition to tailoring the visual elements to better match the music style.
3. The Process:
I start by using Suno AI and make a analyze categorize for the music tracks.
Depending on the genre and mood, I generate appropriate pics using Stable Diffusion.
OpenCV is utilized to create audio waveforms that are synchronized with the music.
Finally, moviepy brings everything together into a cohesive video, ready for uploading to YouTube Shorts.
Humorous: Come to class with a poop face (chinese)
I think this music uses natural language processing (NLP) technology, combined with machine learning models to automatically generate humorous and clever lyrics.
2. EDM electronic music plus Audio Waves animation effect: Pop to your soul — Desire Lights (english)
When AI creates foreign language songs, first uses multilingual text generation models to write lyrics. These models are usually based on the Transformer architecture and can understand and generate poetry or lyrics in multiple languages.
4. Challenges and Learning:
Throughout this project, I faced several challenges, especially around syncing the visual effects with precise moments in the music. Getting the AI models to generate contextually appropriate images based on the music analysis also required a lot of tweaking and experimenting.
5. Conclusion:
This project has been a fantastic learning curve, and the results are incredibly satisfying. The combination of these technologies not only automates the process but also creates a product that is both artistic and technically impressive. I’d love to hear your thoughts or answer any questions about the workflow, the challenges, or anything else!
I just finished a small project using moviepy with Google Colab and had some problems along the way when using the ffmpeg tools in the library. To be more precise, I was using ffmpeg_extract_subclip which in Colab's version doesn't have an important fix (the first 2 to 6 seconds are frozen because of the way ffmpeg's command is called). So after finding the fix I went to add it to moviepy's github only to realize it was already there, just only on the most recent version which is not the one colab uses when you just call !pip install moviepy
You have to call specifically pip install moviepy==1.0.3 to get the 1.0.3
Anyways I just hope whoever needs this will find it before spending hours to fix something that's already been fixed.
If you have any ideas on how to make this more visible, I'd be happy to do it. I'm not entirely sure this is the right place but on the off-chance it helps someone I tried it out.
Hello, I have written a very short and easy to follow tutorial on video background removal using Python along with moviepy and mediapipe libraries. For a long time, I thought background removal would be a costly process in terms of money, time and commitment , but it turned out much easier.
I tend to document my experience for my future reference, but I thought some people might really benefit from my experience as well. So I am sharing my post here, let me also know if you have any questions, I would be happy to help/answer.
I'm wondering how I would use a film grain overlay, with a black background or green and use that as an overlay on a video clip? Can anyone help me out? Thanks in advance!
Instead of spending hundreds of dollars on software and tutorials, I'm literally doing everything for my YouTube channel programmatically for the low, low price of free. I plan to spend a week-long series teaching the things MoviePy can do, what it's done for me, and how it can be helpful.
Please let me know if the creator/maintainer of MoviePy has a tip jar. This is a powerful library that matches my needs perfectly and has an outstanding community.
i've just started using moviepy and i like it. this is a lot easier to use than gst or mlt.
is there an example how to concatenate clips with crossfades? i saw this: 'For more advanced fading, see composition.crossfade,' but i can't find that method in the documentation
I wanted to make reproducible videos for one of my apps so I can easily update the video when the app changes. So I made a "template language" for MoviePy.
I made a code that takes a podcast, tries to find some funny parts from the podcast, and then generates the parts into Youtube Shorts like videos, with subtitles and Brain Rot video (GTA Gameplay) below the Main Video. I basically just input the podcast and it pops out the final shorts.
It uses assembly ai transcriber, openai gpt4 api and of course moviepy and some Pillow
Since GPT4 has no sense of humor and i suck with prompts, so the shorts arent that funny or interesting, but this could be tweaked a bit with prompt engineering to work really well.
Here are 2 Examples, one is from podcast and one is from longer video (just random things i found)
As of my understanding the stroke is equally inwards & outwards. I've done some googling to find out that the workaround to this is to create two "layers". One with just the text and one with just the outline and to stack them on top of each other. I do this but they don't line up properly.
First I hade the issue with overflowing but I managed to compensate for that by increasing the size of the outlined text by the size of the outline but now I have some issues with the outlined text being shifted up and down randomly between each text snippet.This is what I have so far: