Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Feature Requests

Reply
 
Thread Tools Display Modes
Old 03-16-2019, 01:47 PM   #1
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default Extension API: Link LICE bitmap to REAPER window, for seamless integration into UI

Cockos has an extensive arsenal of graphics-related code, ranging from LICE functions to WDL/swell's GDI emulation and REAPER's own gfx functions for script GUIs and video editing.

These functions are just *begging* to be used by C++ extensions and ReaScripts to draw inside REAPER's own windows. This will allow seamless integration of extensions and scripts into REAPER's UI.

Here are a few examples of what can be achieved:
(The problems that these examples encounter are described below.)

Scripts such the MIDI Inspector can display information inside the MIDI editor:



CC editing scripts can display their curves:



Extra buttons and other elements can added in open space for easy access:



All kinds of other fancy graphics that would make REAPER even more attractive and interesting to work with:



The js_ReaScriptAPI extension has already exposed most of the LICE and GDI functions to ReaScripts. Using GDI, extensions and scripts can draw in any of REAPER's windows -- but, this only works properly in static windows.

In windows that are continuously being refreshed (such as the MIDI editor when CCs are being drawn, or the arrange view when the project is playing and scrolling), the script-drawn GDI graphics will either flicker or disappear altogether. (In askjf.com, Justin confirmed that it would be tricky to avoid flickering with the existing GDI functions.)

I assume this problem is due to REAPER and the script drawing alternately and independently, so whichever is drawn last in each cycle, overwrites the other and is displayed on top. In Windows OS, it seems as if the script graphics are usually drawn last and is therefore visible (albeit with flickering), but in macOS and Linux, REAPER's own graphics are usually drawn last and the script graphics are therefore not visible at all.


Suggestion:

I suggest that this problem can be solved by adding a single new LICE function that "links" a LICE bitmap with a REAPER window:

* As source bitmap, the function takes a LICE bitmap as argument, and as destination, it takes one of REAPER's windows (or the HDC of the window, or a LICE sysbitmap that is somehow linked to the window). Similar to Alphablend and WDL/swell's StretchBlt, the function can specify source and destination rectangles within the bitmap and window. EDIT: If necessary to make the function easier to code, the function may even be limited to a handful of commonly used windows such as the arrange view and the piano roll.

* Similar to other LICE blitting functions, it should understand per-pixel alpha channel (RGBA color), so that the extension-drawn LICE bitmaps can be overlaid on top of the other window contents.

* Whenever a REAPER window is refreshed, REAPER should draw any bitmaps that were linked by an extension, on top.

* An extension can animate its graphics by simply redrawing the LICE bitmap at any time: The new bitmap will then be blitted to the window in the next refresh cycle. In order to force animation over a static window that isn't continuously being redrawn (such as most of REAPER's windows, excluding trackview and midiview), the extension can send an "InvalidateRect" to the window, to induce a refresh.

* And, of course, a function to UNlink a window and bitmap.

Last edited by juliansader; 03-20-2019 at 10:27 AM.
juliansader is offline   Reply With Quote
Old 03-16-2019, 02:20 PM   #2
nappies
Human being with feelings
 
nappies's Avatar
 
Join Date: Dec 2017
Posts: 198
Default

Already started to do a few projects using your JS API. And yes, this flicker is annoying! Great suggestion! Full support for Julian!

PS: To users who do not write scripts!!! Please support this too! Because JS API with LICE/CDI open just crazy opportunities with UI!

Last edited by nappies; 03-16-2019 at 02:37 PM.
nappies is offline   Reply With Quote
Old 03-16-2019, 02:52 PM   #3
ReaMike
Human being with feelings
 
ReaMike's Avatar
 
Join Date: Feb 2017
Location: Sydney Australia
Posts: 127
Default

+1

It will be very interesting to see what the scripting gurus can do with this.
ReaMike is offline   Reply With Quote
Old 03-16-2019, 03:40 PM   #4
Vagelis
Human being with feelings
 
Join Date: Oct 2017
Location: Larisa, Greece
Posts: 1,074
Default

Sounds nice +1
Vagelis is online now   Reply With Quote
Old 03-16-2019, 09:12 PM   #5
Tod
Human being with feelings
 
Tod's Avatar
 
Join Date: Jan 2010
Location: Just outside of Glacier National Park
Posts: 13,208
Default

+1 from me.
Tod is offline   Reply With Quote
Old 03-17-2019, 12:01 AM   #6
EvilDragon
Human being with feelings
 
EvilDragon's Avatar
 
Join Date: Jun 2009
Location: Croatia
Posts: 24,361
Default

Aye!

Devs please rid us of that annoying flicker and allow scripters to do more awesome things in scripts!!
EvilDragon is online now   Reply With Quote
Old 03-17-2019, 04:00 AM   #7
X-Raym
Human being with feelings
 
X-Raym's Avatar
 
Join Date: Apr 2013
Location: France
Posts: 7,204
Default

Interesting !



Are these functions accessible to C++ extension already ?
X-Raym is offline   Reply With Quote
Old 03-17-2019, 08:23 AM   #8
Andernik
Human being with feelings
 
Join Date: Apr 2016
Posts: 9
Default +1

+1 for me.
Andernik is offline   Reply With Quote
Old 03-17-2019, 09:53 AM   #9
Archie
Human being with feelings
 
Archie's Avatar
 
Join Date: Oct 2017
Location: Russia
Posts: 366
Default

from me +1.
__________________
=================================
ReaPack| Archie-ReaScript: Discussion | Donate | Donate2 | Donate3 | PayPal |
Archie is offline   Reply With Quote
Old 03-17-2019, 10:20 AM   #10
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default

Quote:
Originally Posted by X-Raym View Post
Are these functions accessible to C++ extension already ?
Which functions are you referring to?

The GDI and LICE functions are accessible to C++ functions, and also to scripts via the ReaScriptAPI extension. These work well for drawing in REAPER windows that are static (for example the rotating dice example), but not for drawing in animated windows.

The requested function will be new, and is not yet available for C++ extensions.
juliansader is offline   Reply With Quote
Old 03-17-2019, 10:37 AM   #11
BAYANBAYAN
Human being with feelings
 
Join Date: Nov 2018
Posts: 1
Default

+1 good idea
BAYANBAYAN is offline   Reply With Quote
Old 03-17-2019, 10:55 AM   #12
vitalker
Human being with feelings
 
vitalker's Avatar
 
Join Date: Dec 2012
Posts: 6,712
Default

I hope this API will allow using dropdown menus for buttons. +1 of course!
__________________
Quote:
Originally Posted by Sexan View Post
Make a thread "F%#! you Reaper! I quit!" to get more views and feedback since these kind of threads get much more traction
SoundCloudFR: True Non-destructive GLUEAll the REAPER names
vitalker is online now   Reply With Quote
Old 03-17-2019, 11:04 AM   #13
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default

Quote:
Originally Posted by vitalker View Post
I hope this API will allow using dropdown menus for buttons. +1 of course!
The way I do dropdown menus for buttons, is to open a tiny GUI window next to the button or mouse, and then draw the menu (which is larger than the GUI window) over the GUI. If necessary, you can even make the GUI invisible using JS_Window_SetOpacity. As an example of such a custom dropdown menu, see the MIDI Inspector example above.
juliansader is offline   Reply With Quote
Old 03-17-2019, 11:31 AM   #14
Yanick
Human being with feelings
 
Yanick's Avatar
 
Join Date: May 2018
Location: Moscow, Russia
Posts: 373
Default

+1, i agree
Yanick is online now   Reply With Quote
Old 03-17-2019, 12:19 PM   #15
YuriOl
Human being with feelings
 
Join Date: Sep 2018
Location: lugansk
Posts: 70
Default

Nice!+1 👍
YuriOl is offline   Reply With Quote
Old 03-17-2019, 12:26 PM   #16
vitalker
Human being with feelings
 
vitalker's Avatar
 
Join Date: Dec 2012
Posts: 6,712
Default

Quote:
Originally Posted by juliansader View Post
As an example of such a custom dropdown menu, see the MIDI Inspector example above.
So this is already possible? That's why I wrote this. Thanks!
__________________
Quote:
Originally Posted by Sexan View Post
Make a thread "F%#! you Reaper! I quit!" to get more views and feedback since these kind of threads get much more traction
SoundCloudFR: True Non-destructive GLUEAll the REAPER names
vitalker is online now   Reply With Quote
Old 03-17-2019, 12:57 PM   #17
_Stevie_
Human being with feelings
 
_Stevie_'s Avatar
 
Join Date: Oct 2017
Location: Black Forest
Posts: 4,127
Default

+1 from me, too!
__________________
My Reascripts forum thread | My Reascripts on GitHub
If you like or use my scripts, please consider donating to amagalma cfillion FeedTheCat HeDa Julian Sader lb0 nofish Sexan X-Raym
_Stevie_ is offline   Reply With Quote
Old 03-17-2019, 01:41 PM   #18
SNJUK2
Human being with feelings
 
SNJUK2's Avatar
 
Join Date: Feb 2017
Location: Ukraine
Posts: 131
Default

+++++
SNJUK2 is online now   Reply With Quote
Old 03-17-2019, 03:10 PM   #19
cfillion
Human being with feelings
 
cfillion's Avatar
 
Join Date: May 2015
Location: Québec, Canada
Posts: 3,413
Default

An extension can draw after REAPER (except in some cases where it appears to be repainting directly, eg. the playback cursor) by intercepting the WM_PAINT window message.


(Red=WM_PAINT, Green=js_ReaScript API redrawing at reaper.defer's 30Hz)

https://gist.github.com/cfillion/0d3...688175ec0026f1

Last edited by cfillion; 03-17-2019 at 04:36 PM.
cfillion is offline   Reply With Quote
Old 03-17-2019, 09:54 PM   #20
Aleksandr_Oleynik
Human being with feelings
 
Join Date: Jan 2013
Location: Kiev, Ukraine
Posts: 66
Default

+1 from me
Aleksandr_Oleynik is offline   Reply With Quote
Old 03-17-2019, 09:56 PM   #21
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default

Quote:
Originally Posted by cfillion View Post
An extension can draw right after REAPER (except in some cases where it appears to be repainting directly, eg. the playback cursor) by intercepting the WM_PAINT window message.
All hail our hero cfillion!

Using your approach, I've been able to get this to work beautifully! The user creates a LICE bitmap with transparent background, and then simply calls a function to "link" the bitmap to any window. Whenever the window is re-painted (or whenever the user calls InvalidateRect), the bitmap is blitted over the window. EDIT: Alas, as discussed below, after further experimentation I found that blitting LICE bitmaps doesn't work as well as drawing simple graphics such as filled rectangles.

Blitting is a very fast function, and updating the entire screen takes less than 1/10000 of a second, so I don't think it will affect performance.

It works in WindowsOS and macOS, but unfortunately blitting with transparency seems not to have been enabled in WDL/swell for Linux yet. I should check if I can replace the Linux function with my own.

I will upload some GIFs soon.

Last edited by juliansader; 03-19-2019 at 04:12 PM.
juliansader is offline   Reply With Quote
Old 03-17-2019, 10:37 PM   #22
deeb
Human being with feelings
 
deeb's Avatar
 
Join Date: Feb 2017
Posts: 3,221
Default

good luck on this! : )
deeb is offline   Reply With Quote
Old 03-18-2019, 03:28 AM   #23
Broojacker
Human being with feelings
 
Join Date: Apr 2016
Posts: 34
Default

+1 very useful
Broojacker is offline   Reply With Quote
Old 03-18-2019, 03:38 AM   #24
EvilDragon
Human being with feelings
 
EvilDragon's Avatar
 
Join Date: Jun 2009
Location: Croatia
Posts: 24,361
Default

Quote:
Originally Posted by juliansader View Post
All hail our hero cfillion!

Using your approach, I've been able to get this to work beautifully!
Meaning no more flicker in your Inspector? *_*
EvilDragon is online now   Reply With Quote
Old 03-18-2019, 04:36 AM   #25
Dragonetti
Human being with feelings
 
Join Date: Feb 2017
Posts: 487
Default

+1 from me.
Dragonetti is offline   Reply With Quote
Old 03-18-2019, 12:33 PM   #26
cfillion
Human being with feelings
 
cfillion's Avatar
 
Join Date: May 2015
Location: Québec, Canada
Posts: 3,413
Default

Quote:
Originally Posted by doppelganger View Post
didn't work for me until i made typecasting reinterpret_cast<WNDPROC>(&paintProc) in unique ptr wrapping.

I'm on Win7 32bit (VS2013) if it matters.

Interesting, why you guys use reinterpret_cast, i made send controls window not long time ago and used c-style and it seems working ok,
but i'm noob in all this stuff), as i understand, there may be some consequences, if c-style conversion used on c++ pointer types, right?
Ah, thanks. A WNDPROC is supposed to return a long, not an int. I've fixed the return type of paintProc to be correct.

Old C-style casts do whatever they can to do the conversion. C++ casts are more specialized, making the original intent clearer and helps catch mistakes.
http://www.stroustrup.com/bs_faq2.html#static-cast

Last edited by cfillion; 03-18-2019 at 12:42 PM.
cfillion is offline   Reply With Quote
Old 03-19-2019, 04:09 PM   #27
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default

Sadly, I must report that, after some further experimentation with cfillion's approach, there is still flickering when blitting images over an animated window.

I mentioned above that blitting is fast and shouldn't affect REAPER's performance, but apparently it's not fast enough to avoid glitching in the display. When drawing simple graphics such as a filled rectangle directly into the window with GDI, I see almost no flickering. But when blitting more complex LICE images such as the transparent rotating dice example, onto an animated window, flickering becomes very noticeable.
juliansader is offline   Reply With Quote
Old 03-19-2019, 04:44 PM   #28
nappies
Human being with feelings
 
nappies's Avatar
 
Join Date: Dec 2017
Posts: 198
Default

juliansader, So only the Reaper's devs can help with this?Or they can`t too?
nappies is offline   Reply With Quote
Old 03-19-2019, 04:56 PM   #29
vdubreeze
Human being with feelings
 
vdubreeze's Avatar
 
Join Date: Jul 2011
Location: Brooklyn
Posts: 2,183
Default

Quote:
Originally Posted by nappies View Post

PS: To users who do not write scripts!!! Please support this too! Because JS API with LICE/CDI open just crazy opportunities with UI!
OK, that's me!

+1 and however else I can support this thing that I have zero understanding of but am down for crazy opportunities with UI
__________________
The reason rain dances work is because they don't stop dancing until it rains.
vdubreeze is offline   Reply With Quote
Old 03-19-2019, 10:03 PM   #30
J Riley Hill
Human being with feelings
 
J Riley Hill's Avatar
 
Join Date: Jul 2014
Posts: 133
Default

this seems like a good thing to add. +1
J Riley Hill is offline   Reply With Quote
Old 03-20-2019, 09:42 AM   #31
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default

Quote:
Originally Posted by nappies View Post
juliansader, So only the Reaper's devs can help with this?
I dunno, perhaps cfillion and the other C++ gurus can find a perfect solution, so we don't have to wait for the devs?


Quote:
Originally Posted by nappies View Post
Or they can`t too?
I actually think it would be pretty straightforward for the devs to implement. REAPER probably uses some kind of multiple buffering for animated windows, so after updating an off-screen buffer, REAPER should simply blit whatever LICE bitmaps are linked to the window onto the buffer.

In my own experiments, this is a mere handful lines of code:
* A std::map of each "linked" window to its linked bitmaps and the sizes of the source and destination rectangles.
* Whenever WM_PAINT is received, iterate through the linked bitmaps and Alphablend (WindowsOS) or StretchBlt (macOS) each onto the window.

Last edited by juliansader; 03-20-2019 at 04:22 PM.
juliansader is offline   Reply With Quote
Old 03-20-2019, 03:13 PM   #32
nappies
Human being with feelings
 
nappies's Avatar
 
Join Date: Dec 2017
Posts: 198
Default

Thank! Then we need to get people's and devs attention!
nappies is offline   Reply With Quote
Old 03-20-2019, 03:41 PM   #33
Ozman
Human being with feelings
 
Join Date: Feb 2015
Posts: 664
Default

+1 from me as well.

More capabilities for scripters, more features can be realized.
Ozman is online now   Reply With Quote
Old 03-22-2019, 01:40 AM   #34
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: Berlin
Posts: 10,987
Default

+1
Maybe the devs can explain some of the inner workings to us to keep things at peak efficiency.
__________________
(Video)Using Latch Preview - Faderport16 CSI config
Airon Audio Dia/FX Edit&Mix
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 03-26-2019, 12:38 PM   #35
HDWind
Human being with feelings
 
HDWind's Avatar
 
Join Date: Apr 2016
Posts: 50
Default

A Good Thing +1
HDWind is offline   Reply With Quote
Old 03-28-2019, 05:32 PM   #36
AndrewFalcon
Human being with feelings
 
AndrewFalcon's Avatar
 
Join Date: Dec 2018
Posts: 31
Default

+1 Hey Guys ! Can anybody write area selection like script based on this ui Api?
AndrewFalcon is offline   Reply With Quote
Old 03-28-2019, 06:33 PM   #37
_Stevie_
Human being with feelings
 
_Stevie_'s Avatar
 
Join Date: Oct 2017
Location: Black Forest
Posts: 4,127
Default

IIRC Sexan was already doing some first tests.
__________________
My Reascripts forum thread | My Reascripts on GitHub
If you like or use my scripts, please consider donating to amagalma cfillion FeedTheCat HeDa Julian Sader lb0 nofish Sexan X-Raym
_Stevie_ is offline   Reply With Quote
Old 03-30-2019, 12:55 PM   #38
Arthur McArthur
Human being with feelings
 
Join Date: Sep 2016
Location: Toronto
Posts: 577
Default

Throwing an enthusiastic +1 on the pile
Arthur McArthur is offline   Reply With Quote
Old 04-03-2019, 10:31 PM   #39
Embass
Human being with feelings
 
Embass's Avatar
 
Join Date: Jan 2014
Posts: 519
Default

just workaround.. transparent gfx window example (windows os only).
https://stash.reaper.fm/manage_file/...%20example.lua
Embass is offline   Reply With Quote
Old 04-05-2019, 11:06 AM   #40
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,424
Default

I have added new "reaper.JS_Composite" functions to version 0.980 of the js_ReaScriptAPI extension, to implement cfillion's suggestion. These functions blend LICE bitmaps into REAPER windows.

Results may be affected by OS version (and perhaps even screen refresh rate?), but in my own tests:
* Compositing works perfectly on macOS, without any flickering;
* On WindowsOS, there is still a slight but noticeable glitching when the window is scrolling;
* Linux doesn't flicker, but per-pixel alpha blending isn't working.

An unconventional application of these functions would be for theming UI elements:

Last edited by juliansader; 04-05-2019 at 03:02 PM.
juliansader is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 06:28 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.