11-05-2018, 07:00 AM | #1 |
Human being with feelings
Join Date: Oct 2015
Posts: 739
|
video processor: multiply with alpha, compositing
I'm looking for a way to crop video on one track with black/white image (or video) on a second track. Like this: https://en.wikipedia.org/wiki/Alpha_compositing
Two tracks get combined into one. Result would be partly seethrough video - transparent on black parts and fully visible on white parts. Video editors have this, but I cannot figure out how to do it in Reaper (apart from basic croping to rectangle). Has anybody already written preset for video processor that can do this, and can share it? Otherwise, which function in video processor would be appropriate for this task? Thank you for help. Last edited by sonicowl; 11-05-2018 at 07:21 AM. |
11-05-2018, 08:43 AM | #2 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
This would be a great addition.
I guess it could be kind of a combination off "chroma key" and "image overlay". Also PS layer modes like multiply, darken, lighten etc. would be a great addition. This stuff is standard in every video software. And i guess its not hard to do. Video processor and video in Reaper has such a huge potential. More people should code presets. I tried to write and change some presets but as a noob its pretty hard to put something together. Even the Reaper internals couldn't help me. |
11-05-2018, 07:05 PM | #3 | ||
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
This blends the next (or previous, depending what you have set in project settings) 2 video tracks after the track which has this video processor active: Code:
*snip* see below Quote:
Last edited by wwwmaze; 11-06-2018 at 03:36 AM. |
||
11-06-2018, 12:43 AM | #4 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Many thanks.
But where are the docs? Didn't even know there are docs for that. |
11-06-2018, 02:57 AM | #5 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Either press F1 when the mouse cursor is within the editor or use this online documentation: https://mespotin.uber.space/Mespotin...API_Video.html
On another glimpse I'm not even convinced if blending works the way I think (or the linux implementation of video processor maybe has a bug). 2 observations: 1. After I blend-in completely white pixels while adding both images/videos I still can see some non-white pixels. Same problem with subtraction, the picture should be black but I also see non-black pixels. 2. Multiplying makes the resulting image/video lose its color. |
11-06-2018, 03:10 AM | #6 | |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Quote:
And yes, I also discovered a very strange blend mode behavior. It could be because of the different color spaces. Have no idea actually. I only know those Photoshop blend modes and they react much different from that. It also does not blend drawn colors from other tracks. (Tried to blend in "solid color" from reaper blog.) Only video signals work. |
|
11-06-2018, 03:29 AM | #7 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Thx for that and for testing. Inserting colorspace='RGBA' helps:
Code:
//@param1:blend 'blend' 0.5 0 1 0.5 0.01 //@param2:mode 'blend mode' 0 0 19 0 1 //@param4:sub 'subtraction' 0 0 1 0 1 // blends the next 2 video tracks colorspace='RGBA'; gfx_blit(input_track(0)); gfx_a=blend*(sub?-1:1); gfx_mode=sub?1:mode; gfx_blit(input_track(1)); Though I'm not sure if I'm really supposed to change that, according to the docs: Quote:
|
|
11-06-2018, 03:32 AM | #8 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
There is definitely something going on. I loaded a video and some self made png files with color gradients into Reaper. Sometimes it doesn't blend anything. I even can't blend from one source to another.
I will go on and make some tests today. Edit: Was my fault. The track order was messed up. Works great with gradients and greyscale patterns. Nice work! Last edited by Eliseat; 11-06-2018 at 03:43 AM. |
11-06-2018, 03:40 AM | #9 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Strange, blending with images, videos and other FX's works great here now.
If you want I can take a look at your project. |
11-06-2018, 04:09 AM | #10 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
Not sure how to solve that. What i've tested so far: blend video-video: check blend video-image: check blend video-fx: check blend image-image: check blend image-fx: check blend fx-fx: not working |
|
11-06-2018, 04:09 AM | #11 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
wwwmaze,
this is just amazing. Really. Such a few lines make such a huge outcome. Just take some beautiful colorful gradients and add or multiply them over a video. Wonderful! |
11-06-2018, 04:18 AM | #12 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Video
Gradient Normal mode Adding ??? |
11-06-2018, 04:22 AM | #13 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
|
11-06-2018, 04:29 AM | #14 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
That's really cool Eliseat
Just be careful if any of the inputs is generated by a video-processor FX itself. I'm running into a bug in either my code (most probably) or because of a limitation of the video processor implementation. |
11-06-2018, 04:51 AM | #15 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
This one lets you specify RGBA color-space and set inputs manually (set both to -1 for next and second next video track):
Code:
//@param1:blend 'blend' 0.5 0 1 0.5 0.01 //@param2:mode 'blend mode' 0 0 19 0 1 //@param4:sub 'subtraction' 0 0 1 0 1 //@param6:in1 'in 1 (-1=next video track)' -1 -1 50 0 1 //@param7:in2 'in 2 (-1=second next)' -1 -1 50 0 1 //@param9:csp 'colorspace (1=RGBA)' 0 0 1 0 1 csp ? colorspace='RGBA'; gfx_blit(in1==-1 ? input_track(0) : in1); gfx_a=blend*(sub?-1:1); gfx_mode=sub?1:mode; gfx_blit(in2==-1 ? input_track(1) : in2); |
11-06-2018, 05:06 AM | #16 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
|
|
11-06-2018, 07:54 AM | #17 |
Human being with feelings
Join Date: Oct 2015
Posts: 739
|
wwwmaze, thank you for help.
I wish there was some kind of manual for video processor. F1 gives a list of functions, but no tips on how to use them properly. Cockos company, can you please hire someone to put together a manual for video processor? It is such a wonderful feature of Reaper, but it is such a pain to use it atm. Please... |
11-06-2018, 12:56 PM | #18 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
wwwmaze,
as I see how clear and simple the code for this presets looks like, would it be possible to take the same two inputs but using the second input as greyscale to make the first video transparent depending on the greyscale? This would be the principle of an alpha channel and would be a milestone. Imagine frames and blend modes of all kinds. The possibilities are endless for this. Just create a black and white pattern, use the alpha preset to blend the video depending of the pattern in. I have no idea how this could work but in my mind it should look like this. Input one is a clear screen, second is the video, third is the greyscale video or image. Maybe the "image overlay" preset or the "chroma key" could help in this case. Anyway. I'm really, really thankful for your efforts and for Owls contributions too. I think the video thing is way to underrepresented and it would be cool to have a section in the forum to gather information, presets and manual for the video interested people. Greetings Eli |
11-06-2018, 04:19 PM | #19 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
a) set up an example project, save an test preset and put in all functions to see what they output in certain cases (especially handy for all those input* functions). To plot to screen this could be a useful helper function: Code:
curX=0; function m(msg) local(s,h,id) ( id = match("%s",msg) ? "%s" : msg|0===msg ? "%d" : "%f"; sprintf(s=#,id, msg); gfx_str_measure(s,0,h); gfx_set(1,1,1,1); gfx_str_draw(s,0,curX); curX+=h; ); gfx_blit(0); // usage m(input_count()); m("spam:"); m(input_track_count()); m($pi); |
|
11-06-2018, 08:47 PM | #20 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
Code:
//@param1:byCl 'bypass clear screen' 0 0 1 0 1 //@param2:byVi 'bypass video' 0 0 1 0 1 //@param3:byMask 'bypass mask' 0 0 1 0 1 //@param5:alCl 'alpha clear screen' 0.5 0 1 0.5 0.005 //@param6:modeCl 'mode clear screen' 0 0 19 0 1 //@param8:csp 'colorspace (1=RGBA)' 1 0 1 0 1 // next (or previous, depending on project settings) video track: clear screen // track after next track: video // third closest track: mask gfx_set(0,0,0,1); gfx_fillrect(0,0,project_w,project_h); csp ? colorspace='RGBA'; !byVi ? (gfx_a=1; gfx_mode=0; gfx_blit(input_track(1));); // overlay !byMask ? (gfx_a=-1; gfx_mode=1; gfx_blit(input_track(2));); // subtraction !byCl ? (gfx_a=alCl; gfx_mode=modeCl; gfx_blit(input_track(0));); Last edited by wwwmaze; 11-06-2018 at 09:01 PM. |
|
11-07-2018, 12:38 AM | #21 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Hi wwwmaze,
that's again pretty cool. Many thanks But I guess "clear screen" was a bit weird as description, it was meant as the transparent layer. You did exactly what I suggested but one thing was the main intention: To hard mask video a over video b. At the moment video a blends in to video b thru blend modes AND alpha from the third layer. But its not possible to have 100% video a (white mask) over 100% video b (black mask). Would this be possible? I'll show what I mean: Video a video b Alpha Blend 0.5 Blend 0.5 blend mode add! <- corrected! (Was not blend 1.0) Last edited by Eliseat; 11-07-2018 at 06:59 AM. |
11-07-2018, 12:48 AM | #22 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Blend 0
This is blend 0 or 100% "video a" with mask. But it would be perfect to have the ability to hard blend this state over 100% of "video b". So all the black should be "video b". Why? Imagine a video with a view thru a window. You could then blend in a complete other video in the window frame to replace for example a wood with a beach. Or imagine a black mask with white moving clouds which then could layer "video a" as cloud over "video b". At the moment that's possible with image overlay. But only for still images in PNG format. Can you see the potential? |
11-07-2018, 05:19 AM | #23 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
So this would be video b completely blended in, aka blend=1.0. But what is the image showing you labeled "Blend 1.0" then? Specifically what is the difference between images "Blend 0.5" and "Blend 1.0". For me "Blend 1.0" just looks a bit brighter. Are your pictures showing the current state or how you'd like it to be? If the former, could you please create 4 or 5 mock-up pictures showing blend=0, blend=0.33, blend=0.66 and blend=1? Thank you. |
|
11-07-2018, 06:54 AM | #24 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
I understand your confusion as I used a blend mode not the normal blend. That's why the blend 0.5 looks a bit darker than expected.
And yes, your picture shows exactly what I meant. A complete masking of video a without blending. This would be great and extremely useful for every video composer. Many thanks Edit: I've corrected the picture title as it wasn't blend 1.0. Instead I changed the blend mode again. Blend 1.0 of course shows "video b" complete. Sorry, was in a hurry. Last edited by Eliseat; 11-07-2018 at 07:06 AM. |
11-07-2018, 11:10 AM | #25 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
can you please provide a couple of mockup pictures (done in photoshop/gimp/etc) how the end result of the new preset would look like (rather than what the preset is currently doing) using different blend amounts, all with the same blend mode (best would be normal mode = overlay/replace; no adding/subtracting/multiplying/etc)
sry really don't know how to go from here |
11-07-2018, 03:44 PM | #26 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Hi wwwmaze,
I didn't want to confuse you. I'm really happy with what you've shared already. And my English is maybe to bad to describe my intention as effective as it should be. So the preset you made is pretty amazing. We now have two videos which can get blended per mode AND alpha. The only thing that actually isn't possible is the mockup picture you posted yourself: Maybe its to complicated to achieve this inside your blend preset. Would it be easier to do it separately? To pin it down: we have three tracks with "Video a", "Video-b" and alpha. "Video a" covers "Video b" completely except if "Video a" gets transparent from alpha. Blacks make 100% transparent. Whites make 100% opaque. 50% gray blend 50% of "video a" in normal mode (or adjustable mode as you did). Here the example: Video a Video b Alpha Blend thru Alpha normal mode (Left and right 50%, middle 0% and 100%) Blend thru Alpha - mode multiply (100% shines thru because of blend mode) Practically this is what you already made but with one difference: The Alpha black doesn't turn "video a" to black instead makes "video a" completely transparent. If this is to complicated, please don't feel pushed. I don't want to hustle you. Many thanks |
11-07-2018, 03:49 PM | #27 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
I think my stupid "clear screen" did a great mess respective the understanding of how your blend mode preset works.
But how could it be called? |
11-07-2018, 04:28 PM | #28 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
|
11-08-2018, 05:45 PM | #29 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Give that a try Eli. It's probably not very efficient because it replaces the alpha of the foreground on pixel level.
I couldn't find another way, apart from chroma-keying with gfx_keyedblit maybe. PHP Code:
Last edited by wwwmaze; 11-09-2018 at 07:04 AM. Reason: enable multiprocessing |
11-09-2018, 02:14 AM | #30 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
It works!
Many thanks for all of your effort. Last edited by Eliseat; 11-09-2018 at 02:23 AM. |
11-09-2018, 02:34 AM | #31 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
cat video, a simple photoshop gradient and an alpha video of white smoke.
Wake up, cat! There's silly smoke around you! |
11-09-2018, 02:46 AM | #32 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Waving cat video, sleeping cat video, transition with clouds video.
Those color errors only occur because of the bad gif export. |
11-09-2018, 03:19 AM | #33 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
cat sleeping, cat waving, tint drops as alpha with automated zoom to blend over!
|
11-09-2018, 06:20 AM | #34 |
Human being with feelings
Join Date: Oct 2015
Posts: 739
|
Amazing. Great preset. Thank you both.
How to blend more things over this? What if I want to add some fancy animated sparkly text over it, rendered on black background, that also has animated alpha transition? Would you use folder tracks? Put cat videos+alpha in bottom and animated text in upper folder? Then somehow blend folders together? How would that work? With 3rd folder to include previous two??? Doable at all? |
11-09-2018, 07:11 AM | #35 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Very cool (and cute) demos Eli!
I somehow managed to disable multiprocessing for the main function in the latest preset. Fixed it now, so please copy/paste again. Sorry for the inconvenience. Quote:
|
|
11-09-2018, 08:46 AM | #36 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
sonicowl,
I put wwwmaze's Alpha preset on a folder and just sorted the videos inside. Works pretty well. To achieve what you meant you only need to create a new folder on top of it with the black screen etc. But keep in mind that this really sucks cpu and graphic power. Edit: Its not as easy as I expected. The problem is. If you fade out video A or B, the Alpha video stays and makes it impossible to blend altogether into another folder. I'll think about it. Maybe there is another way to achieve it. As you can see, the stage lightning which only showed around the text keeps going after fading out the alpha. But fading the stage lightning out lets the alpha channel show up because its not covered anymore. I made this simple text flickering with a black and white text title as video A, then I put a blank video processor with stage lightning as video B at the second position to blend it in via the alpha channel. |
11-09-2018, 09:16 AM | #37 |
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
How could we blend a whole folder? Put the alpha blending folder into another folder with just an opacity preset?
I will make a few tests. But before I have to feed my kids. Edit: wwwmaze, it works now wáy more fluidly for me. Many thanks. |
11-09-2018, 08:14 PM | #38 | |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
Quote:
Every track below a video-processor FX (or above, depending on project settings), containing a video item at the current play time, can be referenced by this processor by an index starting from 0. This index strictly increases as we go down no matter if a track is in a folder or not. A track with a video-processor FX is also considered a video track and gets its own index. This is how indexing looks for my "selective blender" preset (* denotes where the FX is placed) Code:
track* background index: 0 foreground index: 1 mask index: 2 Code:
parent track* ---background index: 0 ---foreground index: 1 ---mask index: 2 I.e. video processor (b) would like to use the output of (a) as foreground. Code:
parent track* b ---background b index: 0 ---parent track* a (= foreground b) index: 1 ------background a index: 2 ------foreground a index: 3 ------mask a index: 4 ---mask b index: 5 For the preset I'm not sure how to solve this other then maybe letting the user manually assign tracks (we would need 3 new params). Putting videos in a folder still makes sense imo to visually isolate stuff but one should keep in mind that it doesn't work like with audio. The best approach imo (and a good feature request) would be if video tracks/items would only be accessible if they have the same folder depth. Or some function to query the folder depth. |
|
11-09-2018, 08:57 PM | #39 |
Human being with feelings
Join Date: Jul 2009
Posts: 7,571
|
I forget who made the kaleidoscope code, but thank you! I used it in my last Q&A video.
__________________
REAPER Video Tutorials, Tips & Tricks and more at The REAPER Blog
|
11-09-2018, 09:41 PM | #40 |
Human being with feelings
Join Date: Oct 2009
Posts: 99
|
|
Thread Tools | |
Display Modes | |
|
|