Go Back   Cockos Incorporated Forums > REAPER Forums > ReaScript, JSFX, REAPER Plug-in Extensions, Developer Forum

Reply
 
Thread Tools Display Modes
Old 11-06-2018, 02:18 AM   #1
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default Can extension-provided API functions return strings longer than 1024 characters?

REAPER itself seems to use a "NeedBig" modifier in the parameter name, if a long string is required:

Code:
C++: int GetProjExtState(ReaProject* proj, const char* extname, const char* key, char* valOutNeedBig, int valOutNeedBig_sz)
EEL: int GetProjExtState(ReaProject proj, "extname", "key", #val)
Lua: integer retval, string val = reaper.GetProjExtState(ReaProject proj, string extname, string key)
Python: (Int retval, ReaProject proj, String extname, String key, String valOutNeedBig, Int valOutNeedBig_sz) = RPR_GetProjExtState(proj, extname, key, valOutNeedBig, valOutNeedBig_sz)
When I tried to use "NeedBig", the API interpreter appears to recognize the modifier and, in the case of Lua, the modifier is removed from the parameter name in the API documentation and in the IDE help (as in the code above, "nameOutNeedBig" simply becomes "name").

However, the string size passed to the function is still only 1024.
juliansader is offline   Reply With Quote
Old 11-06-2018, 02:50 AM   #2
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,096
Default

Not an actual answer, just some info:
cfillion works around this limitation by passing a WDL FastString.

https://www.extremraym.com/cloud/rea...F_GetClipboard
https://www.extremraym.com/cloud/rea...etClipboardBig

though

https://forum.cockos.com/showpost.ph...2&postcount=22

But maybe you knew that already.

edit:
To clarify, I didn't post a link to Justin's comment to 'blame' cfillion, but only that I didn't want to suggest something Justin has concerns about.

Last edited by nofish; 11-06-2018 at 03:23 AM.
nofish is offline   Reply With Quote
Old 11-06-2018, 12:11 PM   #3
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,621
Default

CFillion mentioned at some point, that he knows now how to do it without using Faststring now, but I can't find the post anymore...
__________________
Use you/she/her.Ultraschall-Api Lua Api4Reaper - Donate, if you wish

On vacation for the time being...
Meo-Ada Mespotine is offline   Reply With Quote
Old 11-06-2018, 12:52 PM   #4
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default

Thanks for reminding me about SWS's FastStrings.

I checked out how GetFastString manages to return long strings, and it seems that API functions can return long strings if the C++ function has the char* as the return value, instead of a parameter with an "Out" suffix:
(EDIT: const char*, not simply char*.)
Code:
const char* myFunction()
instead of:
Code:
void myFunction(char* myStringOut, int myStringOut_sz)
When the return value is a const char*, the char* can point to a buffer of any size.

I was skeptical about returning a pointer to temporary data, but it seems that REAPER immediately converts the C string to a proper Lua string, so the buffer can be overwritten in the next function call without problems.


This technique has the limitation that the returned string must be the first return value. Usually, functions that return a boolean place the boolean first, similar to MIDI_GetAllEvts:
Code:
boolean retval, string buf = reaper.MIDI_GetAllEvts(MediaItem_Take take, string buf)

Last edited by juliansader; 11-07-2018 at 12:51 AM.
juliansader is offline   Reply With Quote
Old 11-06-2018, 02:37 PM   #5
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,096
Default

That's useful info thanks (as I remembered I also have some functions in SWS pull requests returning a FastString, will change to this technique).

Last edited by nofish; 11-06-2018 at 02:48 PM.
nofish is offline   Reply With Quote
Old 11-06-2018, 02:49 PM   #6
cfillion
Human being with feelings
 
cfillion's Avatar
 
Join Date: May 2015
Location: Québec, Canada
Posts: 4,937
Default

Also, Lua ReaScripts can override the default buffer size by setting the optional hidden *_sz parameter (like in Python, where it's mandatory).

Code:
reaper.myFunction('')       -- default (1024)
reaper.myFunction('', 4096) -- custom
Quote:
Originally Posted by juliansader View Post
I was skeptical about returning a pointer to temporary data, but it seems that REAPER immediately converts the C string to a proper Lua string, so the buffer can be overwritten in the next function call without problems.
Can't return the address of a temporary buffer though. It must still point to valid data when REAPER copies the string after the function has returned (and destroyed all temporaries). The string must be freed after that.

Depending on what your function does, another possibility could be to have it calculate and give the data by small chunks. Like how the various Enum* API functions give one row at a time (which is often the best solution when having to return a table).

Last edited by cfillion; 11-06-2018 at 04:14 PM.
cfillion is offline   Reply With Quote
Old 11-08-2018, 02:03 AM   #7
juliansader
Human being with feelings
 
Join Date: Jul 2009
Posts: 3,714
Default

Quote:
Originally Posted by cfillion View Post
Also, Lua ReaScripts can override the default buffer size by setting the optional hidden *_sz parameter (like in Python, where it's mandatory).
Great! I didn't know about this hidden parameter.

I will add this tip to my API functions' help text.
juliansader is offline   Reply With Quote
Old 11-08-2018, 07:47 AM   #8
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Internally, NeedBig uses a reallocator function to grow buffers up from 1k on demand. Extending it for the next builds so that you can use this function:

Code:
bool (*realloc_cmd_ptr)(char **ptr, int *ptr_size, int new_size);
In your function which has the NeedBig flag, if it is passed a char* bufNeedBig and int bufNeedBig_sz, and bufNeedBig_sz isn't big enough, you'd do this:

Code:
if (!realloc_cmd_ptr(&bufNeedBig, &bufNeedBig_sz, requested_size))
{
  //error!
}
assert(bufNeedBig_sz >= requested_size);
...

Last edited by Justin; 11-08-2018 at 08:19 AM.
Justin is offline   Reply With Quote
Old 11-08-2018, 02:29 PM   #9
cfillion
Human being with feelings
 
cfillion's Avatar
 
Join Date: May 2015
Location: Québec, Canada
Posts: 4,937
Default

Awesome!
cfillion is offline   Reply With Quote
Old 11-08-2018, 08:08 PM   #10
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Also making it so you can append Need8192 etc and it will try to make 8192 bytes available (if you don't need dynamically-sized).
Justin is offline   Reply With Quote
Old 11-09-2018, 05:26 AM   #11
Meo-Ada Mespotine
Human being with feelings
 
Meo-Ada Mespotine's Avatar
 
Join Date: May 2017
Location: Leipzig
Posts: 6,621
Default

Thnx Justin
__________________
Use you/she/her.Ultraschall-Api Lua Api4Reaper - Donate, if you wish

On vacation for the time being...
Meo-Ada Mespotine is offline   Reply With Quote
Old 11-13-2018, 06:49 AM   #12
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

This is in yesterday's +dev build, if you want to start on testing... (or I suppose I could get it working in sws...)
Justin is offline   Reply With Quote
Old 11-20-2018, 08:03 PM   #13
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Some slight changes to realloc_cmd_ptr in 5.962+dev1120 -- calling it will cause Lua to return the full resized length (so you can embed nul characters in there). There are other fixes/optimizations related to this, so if anybody is using this, please test thoroughly again! Thanks!
Justin is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 11:32 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.