Video thumbnail scripts

[juliansader]

REAPER doesn't currently have the option to display video thumbnails in video items, so I uploaded two scripts that offer a (somewhat hackish) workaround:

* js_Video - Extract thumbnails of video items to empty, MIDI or video processor items

This script extracts thumbnail images from selected video items, and adds the thumbnails to selected empty items, MIDI items or dedicated video processor.

These empty/MIDI/processor items can be added manually at hitpoints, or alternatively, empty items can be added by the script "mrlimbic_ffprobe scene detect.lua", which creates empty items at scene (or camera/line) cuts.

(NOTE: For creating a zoom-responsive "thumbnail track", the "Create thumbnail items" script below is much faster than the "Extract thumbnails" script.)

On Linux, ffmpeg needs to be installed, and on WindowsOS and macOS, an ffmpeg executable is required.



* js_Video - Automatically adjust size of thumbnail items when zooming

If thumbnails are extracted and displayed in static empty items, zooming in will result in messy tiling when the items' pixel widths become longer than the width of the thumbnails, while zooming out will result in tiny, barely visible images when the item widths become shorter.

This script offers a workaround: It runs in the background, and automatically adjusts the width of items containing thumbnail images to the approximate width of the images.

All calculations are performed when zooming, not when scrolling, so the script doesn't interfere with playback or recording.

Although this script is mainly intended to create a zoom-responsive "video thumbnail track", it is also useful for preserving the on-screen aspect ratios of other empty items, such as text items or marker items.


EDIT: Third script:
* js_Video - Create thumbnail items in selected track for selected video items

This script creates thumbnail items at regular spacing along the entire lengths of selected video items, and is much faster than the first script, which adds images to pre-existing, irregularly spaced empty items.

[X-Raym]

As long as this isn't native, any workarround is welcome,


and this one seems pretty good ! Well done !



How efficient it is in practice ? Doesn't it take too many CPU power of disk space to generates all the images (let's say, for a 15 minutes video ?)

[Sexan]

On a scale from 1 to 10 thank you 500!!

[juliansader]

Originally Posted by X-Raym

How efficient it is in practice ? Doesn't it take too many CPU power of disk space to generates all the images (let's say, for a 15 minutes video ?)

It is surprisingly fast and CPU usage and well as disk space are negligible -- but RAM might be a problem for lower-powered DAWs.

On my current project, with a video item about 37 minutes long and one thumbnail per second, it uses almost 7GB RAM. Disk space is less than 100MB since each image is just 20kB. I don't know why RAM usage is so many orders of magnitude larger than disk space. EDIT: RAM problem solved!

A performance tip: lock the height of the thumbnail tracks, to prevent them from being re-calculated every time that you zoom other tracks vertically. (In fact, I think I should update the script to lock the heights automatically.)

[heda]

very clever and cool! even as workaround

a jpg can be 20 KB but I think what matters to REAPER would be the resolution. the total number of decoded pixels. maybe as a thumbnail it doesn't need large images specially if you are going to lock the track height, the images could be resized to that track height too

[X-Raym]

@juliansader
Thx for detailes bencharmking.


Indeed is is EXTRA heavy on RAM ! Maybe something to optimized on cockos end ? or maybe in the script Not sure what API functions you use for that... ^^


ANd heda is right maybe the JPEG is decoded and scaled to full rez and it become something larger than 20kn in RAM.

[X-Raym]

Maybe there would bea way to only load image in current arrange view section to avoid loading al of them in the ram ? More disk usage but less ram usage ?


(i know it is quite tricky under the hood, not even mure my idea make sense with the way it currently work).

[juliansader]

Originally Posted by heda

very clever and cool! even as workaround

a jpg can be 20 KB but I think what matters to REAPER would be the resolution. the total number of decoded pixels. maybe as a thumbnail it doesn't need large images specially if you are going to lock the track height, the images could be resized to that track height too

You are correct! When I scale the images to 100p, the RAM usage dropped from 7GB to just 150MB!

I will update the script tomorrow. Instead of asking the user to enter a number for JPG quality, it will ask for thumbnail height, with a default of 100p.

[Reflected]

superb work!

[X-Raym]

, the RAM usage dropped from 7GB to just 150MB!

Thats why I call optimization !

[juliansader]

I've added a new script, "js_Video - Create thumbnail items in selected track for selected video items", which creates thumbnail items at regular spacing along the entire lengths of selected video items, and which is much faster than the first script that adds images one-by-one to pre-existing empty items.

[EpicSounds]

very interesting script!

[Sexan]

Not sure what I'm doing wrong but I get only single thumbnail(frame) thru all the clip. Only 1 image is created in the folder.

Edit: Installed #1 script, works as expected (was on #3)

Edit2 : sorry other way around #3 works, #1 gets 1 image

[juliansader]

Originally Posted by Sexan

Not sure what I'm doing wrong but I get only single thumbnail(frame) thru all the clip. Only 1 image is created in the folder.

Edit: Installed #1 script, works as expected (was on #3)

Could you send me the script's Console output?

[Sexan]

1 thumbnail items found.

Processing thumbnails:

Done!

Successfully added: 1
Non-overlapping with video items: 0

[juliansader]

Was it script #1 or #3 that didn't work properly for you?

The output that you posted is from script #1, and it only found one empty item, so only one thumbnail will be created. Unlike #3, #1 doesn't create new items, and was originally intended as a companion script to mrlimbic's scene detection script, which performs the item creation.

[Sexan]

Oh, then sorry my bad. I thought you create empty item and it makes all pictures.

This was script #1 (I've also posted wrong above sorry)

Script #3 works fine

[Sexan]

BTW just wondering something, long ago when spectral view was in implementation stage, Xenakios managed somehow to write data into peak files. Is there any API/call from C++ that allows this (adding custom data to peak file)?

https://forum.cockos.com/showpost.ph...&postcount=112

[amagalma]

Thanks julian! Very nice!

[Sumalc]

Thank you so much for looking into this feature.

[Triode]

Does this work on Mac?
It’s asking me to select the folder with the ffmpeg executable.. and then it doesn’t accept me picking the user plugins folder..

[juliansader]

It is supposed to work on Mac. Does your user plugins folder contain the "ffmpeg" (no extension) file?

[Triode]

Hi Julian
It contains ffmpeg with the extension .dylib
What do I do ?

[juliansader]

The dylib file is an extension that REAPER can load, similar to SWS or ReaScriptAPI.

To run these scripts, you need the executable. If I remember correctly, you can get the executable in the "static" build from https://ffmpeg.zeranoe.com/builds/

[Triode]

Aha that seems to work!

Thanks

[magrotten]

anyone else getting the error - '...ail items in selected track for selected video items.lua:106: attempt to call a nil value (field 'JS_Dialog_BrowseForFolder')'

?

PS, I find that MrLimbic's scene detect works perfectly when the LUA colourspace threshold is set to 20% in the script

[juliansader]

That error message probably means that the js_ReaScriptAPI extension has not been installed. You can install the extension using ReaPack, or download directly from https://github.com/juliansader/ReaExtensions.

[Pink Wool]

Nice scripts! Can this script:

js_Video - Create thumbnail items in selected track for selected video items

be set up so I have 24 frames per second? And how?


Or actually 23.976!

[EpicSounds]

Originally Posted by Pink Wool

Nice scripts! Can this script:

js_Video - Create thumbnail items in selected track for selected video items

be set up so I have 24 frames per second? And how?


Or actually 23.976!

doesn't it follow your video settings in Project Settings?

[juliansader]

Originally Posted by Pink Wool

Nice scripts! Can this script:

js_Video - Create thumbnail items in selected track for selected video items

be set up so I have 24 frames per second? And how?


Or actually 23.976!

I have uploaded v0.91 of the script, which will accept decimals as thumbnail spacing.

To match with framerate, use a spacing of 1/fps, such as 1/23.976

[juliansader]

Originally Posted by EpicSounds

doesn't it follow your video settings in Project Settings?

Not at present. However, if other users also prefer to extract one thumbnail per frame, I can make the default spacing follow fps.

[Pink Wool]

Originally Posted by juliansader

I have uploaded v0.91 of the script, which will accept decimals as thumbnail spacing.

To match with framerate, use a spacing of 1/fps, such as 1/23.976

Thanks! I'm sure I'll start using this now!