2023–2026 · AI music videos
MovieshMash.to: soundtrack-driven editing
The platform analyzes audio files: tempo, mood, and lyrics. It generates shots and edits them on beat. We released the editing engine as an open ComfyUI node under the Apache 2.0 license.
- Technologies
- Gemini 1.5 Pro, GPT-4, GPU rendering
- Project launch
- 2023
- Export
- 4K, 24, 30 or 60 fps, EDL and XML
- Address
- movieshmash.to
01Challenge
Cut to the beat
Music videos, trailers, and short-form social media content share a common workflow: syncing visuals to audio. Manually, this means listening through, placing timeline markers, and selecting shots.
With every new track, this work starts from scratch because markers from the previous one no longer apply.
02Solution
Track analysis, shots, and rhythm-synced editing
MovieshMash.to handles this part. Once a file is uploaded, the system detects tempo (BPM), mood, lyrics, and emotional structure, then generates shots and cuts them together. Product tagline: "Turn Audio Into Cinema".
Multimodal analysis is handled by Gemini 1.5 Pro and GPT-4. Over 14 video models are available, with rendering routed to distributed GPUs. Google Search Grounding verifies real-world accuracy, and the platform also detects lipsync.
Model selection remains with the user, as visual style is an artistic decision. Assemble the same track in multiple aesthetics.
-
A timeline instead of a single button
Scene-by-scene control: review, replace, or regenerate each shot, including assistant prompts. The initial render serves as raw footage for editing.
-
Professional editing export
Export up to 4K at 24, 30, or 60 frames per second. Alongside MP4 files, export EDL and XML cut lists for Premiere Pro and DaVinci Resolve.
-
"Self API" plan at $15 per month
Clients provide their own Gemini and Replicate API keys and pay providers directly for usage. The subscription covers access to the platform.
03Open source
The editing engine is open
Audio analysis and cut placement also work separately, as an open ComfyUI node Vibe Music Engine. The repository github.com/lazniak/videoclipgenerator has been public since February 12, 2025, licensed under Apache 2.0.
The node runs the audio track through OpenAI Whisper models and outputs word-level subtitles, an EDL cut list, beat detection, and tempo estimation. You can run it locally, without our platform.
The platform adds image generation, a GPU queue, and an interface. Both versions output cut lists in the same EDL format.
04History
From Vidgen.io to MovieshMash.to
Development started in early 2023. Led by Paul Lazniak and Grzegorz Łaźniak. The first prototypes ran on Deforum under the domain Vidgen.io.
They hit two obstacles: inter-frame flickering and shallow audio comprehension. The system identified the file, but not what was happening inside it.
In 2024, we redesigned the architecture and developed the i2v master shoot technique on lyrical tracks. The years 2025–2026 brought Gemini multimodal models and the production release, named Vidumidu until 2026.
It is the same project as today's MovieshMash.to. The name and architecture changed, not the intent.
Production footage
Do you have footage to sync with audio?
Describe what you work with. We will divide it into what the platform handles immediately and what is better assembled on site from an open node.
Schedule a workshop with Paul
Choose thirty minutes or a full workshop. Book a time slot in the calendar, receive immediate confirmation.