Old 03-22-2019, 03:09 AM   #1
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Default Making an icon bob by sidechain audio?

I wanted to do a little video where there's a small icon (a face) overlaid as a layer on top of a video, and make it shake and bob according to an audio input.

It's almost possible. Almost.

To make it key with the alpha map correctly you need to insert a dedicated video processor (otherwise the underlying video will get moved as well). However the parameters of this dedicated video processor cannot be modulated or automated.

Then the icon needs to be added as another media item (an alpha mapped .PNG image), with its own video processor (pan/zoom) as an item FX. These parameters can be automated but only on envelopes on the item itself. Modulation can be added: you can add an LFO, but the LFO's parameters cannot be automated, only set permanently. If you add an audio sidechain, it will not listen to the signal on the track (presumably because the item itself doesn't really receive anything, it's a sender).

So, is there currently any way to achieve that "little icon that bobs up and down according to audio, on top of underlying video" goal in Reaper?

Last edited by Skaven252; 03-22-2019 at 03:16 AM.
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 04:51 AM   #2
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

Skaven252,

this should be possible but only with a workaround. Someone made a JSFX which allowed to translate audio to video parameter automation. I will see if I can find it.

Edit: Found it. Its called "audio to slider" and makes it possible to use a sidechain or main signal as an automated slider. You then can use this slider to automate the video parameter.



Search the forums for the plugin. Or compile it yourself. Here is the code for the audio to slider JSFX. (I hope this is allowed to post here as it is not my plugin.)

Code:
---------------------------------------------
--------------Audio to Slider----------------
------Outboarder   @2017  Reaper 5.50 -------
---------------------------------------------

slider1:0<0,1,0.000001>Left
slider2:0<0,1,0.000001>Right
slider3:M=3<0,10,0.01>Mul

@init
slider1 = 0;
slider2 = 0;
slider3 = 0;

@slider

@sample
    ampL = abs(spl0);
    ampR = abs(spl1);
    peakL = max(peakL, ampL);
    peakR = max(peakR, ampR);
    
    time_precise()-T > 0.02 ? ( 
       SL = 0;
       SR = 0;
       T=time_precise();
    );
    
    SL = max(SL, ampL);
    SR = max(SR, ampR);
   
    L = SL*M;  
    R = SR*M;
    
    slider1 = L;
    slider2 = R;
@block
Short overview of how to do it:
- Put you PNG file on a track
- Use the image overlay preset to make it transparent and proportional. (No dedicated VP needed)
- Put the background video on the track below the image
- Load the "audio to slider" JSFX before the video processor in the PNG FX chain
- Send audio to the PNG track but deselect the master send to avoid doubled volume
- The sliders of the "audio to slider" plugin should move now
- Go to the video processor image overlay preset, click once on the zoom parameter knob and go into the "param"-menu
- Choose Parameter modulation, MIDI link
- Open the drop-down menu and choose the left or right slider of the "audio to slider" FX
- Adjust the automation parameters (strength etc.) to fit to your expectations.

Hope that helps
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆

Last edited by Eliseat; 03-22-2019 at 05:16 AM.
Eliseat is offline   Reply With Quote
Old 03-22-2019, 04:57 AM   #3
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Default

Quote:
Originally Posted by Eliseat View Post
Someone made a JSFX which allowed to translate audio to video parameter automation.
Is it the JS: video sample peeker? That's meant to work with the Oscilloscope and Spectrogram video processor presets, but it can't be used to automate or modulate any parameter. Are you referring to something else perhaps?

It's a matter of being unable to link things together. I tried a ReaControlMIDI on the track, but you can't link a FX that is inside an item, to a MIDI parameter that's in a track, there's no connection.

Sidechain modulation listening to track channel 1-2 doesn't receive any audio, if the effect parameter to be modulated is nested inside an item. Because the item doesn't receive any audio from a track, the traffic goes the other way around.

So it's a bit of a fundamental problem of not being able to link things together "the other way around".
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 05:22 AM   #4
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Thumbs up

Awesome, thank you Eliseat!

So it exploits the workaround that parameters between different plugins in the same chain (or a different track) can be linked together. Clever!
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 05:45 AM   #5
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Arrow

Quote:
Originally Posted by Eliseat View Post
- Load the "audio to slider" JSFX before the video processor in the PNG FX chain
- Send audio to the PNG track but deselect the master send to avoid doubled volume
- The sliders of the "audio to slider" plugin should move now
I can see in that preview GIF that it's doing what I wanted - but I seem to be running into the same problem as before:
- If I put the Image Overlay as a Track FX on the PNG track, it offsets the background video as well. But the AudioToSlider modulation works.
- If I put the AudioToSlider FX -> Image Overlay as Item FX into the PNG item, the AudioToSlider doesn't receive any audio (item cannot receive from track?), thus the sliders don't move. But the zoom/offset applies to the image only, so that part works.
- If the Image Overlay FX is in the Item FX, and the Audio To Slider is in the Track FX, the Image Overlay cannot link to the Audio To Slider plugin's parameter - it has to be in the same chain apparently.
- If I use a Dedicated video processor item, its parameters cannot be linked or automated at all (no such menus or buttons available in dedicated script items).

How was the result you see in that GIF done? Because it's reacting to the audio level, yet not zooming the background video, only the overlay. I thought I did everything in the order you instructed. [scratches head]

Last edited by Skaven252; 03-22-2019 at 07:09 AM.
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 08:01 AM   #6
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

Uhm, for some reason the gif doesn't show an animation anymore. ¯\_(ツ)_/¯

Anyway. I did nothing special. If you use the latest image overlay it should allow to scale the PNG without affecting the background video. But if you want, I could recreate my project and send it to my file storage so you can test it on your system.

Are you using Mac or Windows? I'm on Windows.

Okay, I just make that same project and upload it somewhere. This should be the easiest way to show you how I made this.

Greetings
Eli
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-22-2019, 08:07 AM   #7
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Thumbs up

I'm using the latest Reaper (5.973) so the Video Processor should be latest as well (it includes the video sample peeker and the oscilloscope + spectrogram video presets). On Windows 10. And I have the latest FFMpeg DLLs, if it depends on that.

Did you put the Image Overlay effect into the track chain, or the item chain?

Quote:
Okay, I just make that same project and upload it somewhere. This should be the easiest way to show you how I made this.
Awesome, thank you!
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 08:09 AM   #8
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Lightbulb

All right, this is weird... instead of modifying an older project to those specifications, I created a new project from scratch. Now the Image Overlay as a Track FX does not affect the underlying video.

So... uh... it depends on in what order you do things? Hmmmmmmm.
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 08:19 AM   #9
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

No, I guess it loaded the code of the old Image overlay preset which did not take care of the background video. So this would have been the next question of mine if your preset code looks the same as mine.

Anyway. The upload is nearly done. I will post the links soon. If you don't need it after finding out what went wrong, maybe other people can use it to learn.

Here is the link for the example project: https://mega.nz/#!vuwjWIwQ!mEp884DkG...fCxdpqhCI9MlLY

Have fun

Edit: For testing purposes, let me know if the project works.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆

Last edited by Eliseat; 03-22-2019 at 08:24 AM.
Eliseat is offline   Reply With Quote
Old 03-22-2019, 08:29 AM   #10
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

By the way. Somewhere I asked Justin why we can't use audio as an modulation source for video parameters. He didn't really answer but told me it should not work. Maybe there was/is a limitation in doing so.
But I see no difference between using this workaround with a slider as the source vs. directly using audio.

As always in Reaper: You can do whatever you want if you know how to find a workaround.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-22-2019, 08:34 AM   #11
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Thumbs up

Thank you!

Yes, your example project is working. Thanks! I had to replace and re-link the Param To Slider plug-in as I had it with a different file name on my end, but after that I could make the sphere zoom with the kick drum.

On my end, it does seem like there are some oddities and dependencies happening. If the audio track is multi-channeled (5.1 surround in my case), the Audio To Slider doesn't work via a send. I'll need to switch the channel mode of the audio file itself to stereo - just setting the track channels won't help. Even if the Send only sends channels 1-2. This is a bit odd because Audio To Slider itself is 4-channeled.

So, as long as you create the tracks in the right order and use stereo audio only, it works. ^_^
Skaven252 is offline   Reply With Quote
Old 03-22-2019, 05:48 PM   #12
Bri1
Human being with feelings
 
Bri1's Avatar
 
Join Date: Dec 2016
Location: England
Posts: 2,373
Default

heh- take a peek at this lolz..>




Code:
desc:Disco-Lara
^if another js user would take this code justin wrote,and run with it--there could be a possibility of some better animations-monty python style..lolz.
just swap out the png's.. there's another with the bone guy=lolz..funny jsfx!
Bri1 is offline   Reply With Quote
Old 03-22-2019, 11:49 PM   #13
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

Bri1 ...

where the heck did you find something like this? ლ(´ڡ`ლ)

Its kind of disturbing to see such a weird combination of music and graphic. But then I realize that it was you who posted it and everything is as it is supposed to be.

Skaven252,

there is another version of the "audio to slider" JSFX, maybe this can manage multi track. I don't know. For some reason I can't find the thread where this got posted anymore. So here is the code for the multi channel version.

Code:
------------------------------------------------------
------------Audio to Slider  4X(stereo)---------------
------------------------------------------------------
----------Outboarder   @2017  Reaper 5.50 ------------
------------------------------------------------------

slider1:0<0,1,0.000001>Left(1)
slider2:0<0,1,0.000001>Right(2)
slider3:M1=2<0,10,0.01>Mul A
slider4:Time1=0.02<0,0.5,0.001>Time A

slider5:0<0,1,0.000001>Left(3)
slider6:0<0,1,0.000001>Right(4)
slider7:M2=2<0,10,0.01>Mul B
slider8:Time2=0.02<0,0.5,0.001>Time B

slider9:0<0,1,0.000001>Left(5)
slider10:0<0,1,0.000001>Right(6)
slider11:M3=2<0,10,0.01>Mul C
slider12:Time3=0.02<0,0.5,0.001>Time C

slider13:0<0,1,0.000001>Left(7)
slider14:0<0,1,0.000001>Right(8)
slider15:M4=2<0,10,0.01>Mul D
slider16:Time4=0.02<0,0.5,0.001>Time D

@init
slider1 = 0;
slider2 = 0;
slider3 = 2;
slider4 = 0;
slider5 = 0;
slider6 = 0;
slider7 = 2;
slider8 = 0;
slider9 = 0;
slider10 = 0;
slider11 = 2;
slider12 = 0;
slider13 = 0;
slider14 = 0;
slider15 = 2;
slider16 = 0;
@slider


@sample
    ///////////////////////A (1/2)///////////////////////////
    ampL1 = abs(spl0);
    ampR1 = abs(spl1);
    peakL1 = max(peakL1, ampL1);
    peakR1 = max(peakR1, ampR1);
    
    time_precise()-T1 > Time1 ? ( 
       SL1 = 0;
       SR1 = 0;
       T1=time_precise();
    );
    
    SL1 = max(SL1, ampL1);
    SR1 = max(SR1, ampR1);
   
    L1 = SL1*M1;  
    R1 = SR1*M1;
    
    slider1 = L1;
    slider2 = R1;
    ////////////////////////B (3/4)/////////////////////////////
    ampL2 = abs(spl2);
    ampR2 = abs(spl3);
    peakL2 = max(peakL2, ampL2);
    peakR2 = max(peakR2, ampR2);
    
    time_precise()-T2 > Time2 ? ( 
       SL2 = 0;
       SR2 = 0;
       T2=time_precise();
    );
    
    SL2 = max(SL2, ampL2);
    SR2 = max(SR2, ampR2);
    
    L2 = SL2*M2;  
    R2 = SR2*M2;
    slider5 = L2;
    slider6 = R2;
    ////////////////////////C (4/5)/////////////////////////////
    ampL3 = abs(spl4);
    ampR3 = abs(spl5);
    peakL3 = max(peakL3, ampL3);
    peakR3 = max(peakR3, ampR3);
    
    time_precise()-T3 > Time3 ? ( 
       SL3 = 0;
       SR3 = 0;
       T3=time_precise();
    );
    
    SL3 = max(SL3, ampL3);
    SR3 = max(SR3, ampR3);
    
    L3 = SL3*M3;  
    R3 = SR3*M3;
    slider9 = L3;
    slider10 = R3;
    /////////////////////////D (6/7)////////////////////////////
    ampL4 = abs(spl6);
    ampR4 = abs(spl7);
    peakL4 = max(peakL4, ampL4);
    peakR4 = max(peakR4, ampR4);
    
    time_precise()-T4 > Time4 ? ( 
       SL4 = 0;
       SR4 = 0;
       T4=time_precise();
    );
    
    SL4 = max(SL4, ampL4);
    SR4 = max(SR4, ampR4);
    
    L4 = SL4*M4;  
    R4 = SR4*M4;
    slider13 = L4;
    slider14 = R4;
    /////////////////////////////////////////////////////
@block
I'm also not sure why you can't get the 2 channel version working with multi track audio. Seems a bit weird to me. I will take a look into it if I find a multi track video somewhere.

In general I would suggest to place a second version of the multi channel audio on a separate track by exploding the audio of the video. As I remember right you have to right click the multi track file and choosing something like "item processing"-> Explode to single tracks.
You than get copies of every channel which you could easily use with the audio to slider JSFX.

__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 01:52 AM   #14
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Lightbulb

Thanks for the update Eliseat!

Now I'm getting ambitious. It should be possible to make multi-band bobbers by using a 3-Band Splitter, followed by 3 AudioToSliders on the bands, each bobbing a different image overlay layer.

... but there's no need to go more elaborate than that though, since beyond that you can just use the decorative spectrum analyzer video processor preset.
Skaven252 is offline   Reply With Quote
Old 03-23-2019, 03:53 AM   #15
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

You already are ambitious.

The problem is, that you will quickly run into preview issues as this takes a lot of software computation. But I see you finally managed to get into the Reaper philosophy: Its not a DAW it a big audio (and video) tool set to play with.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 06:58 AM   #16
Bri1
Human being with feelings
 
Bri1's Avatar
 
Join Date: Dec 2016
Location: England
Posts: 2,373
Default

Quote:
Its kind of disturbing to see such a weird combination of music and graphic. But then I realize that it was you who posted it and everything is as it is supposed to be.

taking the piss m8 ?

problem is not i-it's others lack of sensing the messages that are given here-in plain english (90% of time it's completely legible) unless blinded by ignorance.. =)
cockos justin wrote 2 jsfx,to do with png animating..go hunting if it pleases...the potential is further reaching than half_ass attacks may be..

there is the underlying thing with vids in reaper being able to play on top,or below each track---the ordering of effects is a pita tbh..videos could mix,as audio does,but that's just not happening right now is it? ..devs doing some half_ass baking,or,overcook=crumbling product. imo.
Bri1 is offline   Reply With Quote
Old 03-23-2019, 09:12 AM   #17
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

Quote:
Originally Posted by Bri1 View Post
taking the piss m8 ?

problem is not i-it's others lack of sensing the messages that are given here-in plain english (90% of time it's completely legible) unless blinded by ignorance.. =)
cockos justin wrote 2 jsfx,to do with png animating..go hunting if it pleases...the potential is further reaching than half_ass attacks may be..

there is the underlying thing with vids in reaper being able to play on top,or below each track---the ordering of effects is a pita tbh..videos could mix,as audio does,but that's just not happening right now is it? ..devs doing some half_ass baking,or,overcook=crumbling product. imo.
Bri, I don't understand your language. What is m8 and why do you write something like "taking the piss"? If this is a metaphor or picture, I don't get it. "Go hunting if it pleases"?
As a German its not easy to read everything here in English but your posts are even heavier to read as plain English. Sorry, this is no offend. Maybe I'm just to stupid to get your points.

And what I meant with "crazy" wasn't meant negative. As I just wanted to point out that no one else would have posted such a video here in this context. It seems everything you interested in is kind of abstract. That's fine. No insult. Its just obvious you. Nothing more.

Sorry, for all the misunderstandings.

If you are angry about the strange video handling aka - the effect of the very top video track affects all video tracks below, - then I can give you relief because since the last update some presets (like the image overlay we were talking here in this thread) really allow track dependent effects. Its a start. But a good one!

And I agree. The default video behavior seems kind of weird because one intuitively thinks everything works like audio. But as always, in most cases there is a workaround in Reaper.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 11:21 AM   #18
Bri1
Human being with feelings
 
Bri1's Avatar
 
Join Date: Dec 2016
Location: England
Posts: 2,373
Default

Quote:
As a German its not easy to read everything here in English......



If you are angry about.....

lolz-see m8,being a mate,friend buddy of iam=you. you must realize,this is an actual english typing forum.
there is a german forum i believe if your english is poorly developed...jus say'n bruv/sis.
+ you clearly have inserted this "anger" -when clearly,there is none =lol?
i encourage inventiveness+all sorts of expressions,but assumptions or unintentionally misleading ideas about a certain character,or a 1 sided perception of certain behaviours--is clearly,not in order..more 'out of order' =quite insulting when you look into the words,not @ them..


what's the topic again?

oh yeah icon bob by sidechain? did anyone here actually give a suitable response to that? nope?
hmmm.. could supply another reply to help further..but imo, it's best people work stuff out for themselves,in all honesty.. thanx.
Bri1 is offline   Reply With Quote
Old 03-23-2019, 11:38 AM   #19
Eliseat
Human being with feelings
 
Eliseat's Avatar
 
Join Date: Mar 2018
Location: Cologne
Posts: 1,151
Default

Do these "disco divas" belong to icon bob? Because I didn't find any valuable info about that. Where is the code of those presets? I just have no idea. (As always.)

And don't point me to the German forum, m8! <- please look at the cool British abbreviation someone taught me
I didn't mean that I don't understand English. Only yours seems to be encrypted. That's all.
So you suggest to look into the words instead of reading them? Uhm, do I need booze or drugs to find out what they mean or do I need more patience? Because this doesn't help at all! (☞゚∀゚)☞
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
Eliseat is offline   Reply With Quote
Old 03-23-2019, 12:15 PM   #20
Bri1
Human being with feelings
 
Bri1's Avatar
 
Join Date: Dec 2016
Location: England
Posts: 2,373
Default

Quote:
Do these "disco divas" belong to icon bob?
heh- yes,that's right m8,they belong to bob..try not to worry though,the nice young men with white jackets will be along to help,soon...soon.

someone else must know these older jsfx? i can't seem to remember the actual links,and think there are 2 folders for the 2 different js png files.

Code:
desc:Disco-Lara
desc:Disco-Slim
as i tried to suggest previously--if any js user could, (or more importantly) would, modify these 2 js- they could become little workshops in their own right= make them fully righteous+usable for all!! =please+ty.
Bri1 is offline   Reply With Quote
Old 03-26-2019, 01:12 AM   #21
Skaven252
Human being with feelings
 
Join Date: Nov 2011
Posts: 121
Default

Quote:
Originally Posted by Bri1 View Post
oh yeah icon bob by sidechain? did anyone here actually give a suitable response to that? nope?
I used the wrong term in the subject line.

I was just looking for a way to bob an icon by audio, and had a problem with having to place the "Image Overlay" preset into the item FX instead of track FX to keep it from moving the background video; so the item FX couldn't listen to the audio from the send, because you can't send to an item (!), you can only send to a track.

"Sidechain" in here could mean that it's using a send to another channel, and that send won't be audible in the final output, it's only used to produce the values for the bobbing.
Skaven252 is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 09:52 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.