|
|
|
10-24-2016, 06:00 AM
|
#1
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
How to store per-track user data?
Hi Reaper gurus,
I'm not able to find any APIs that allow one to associate user data to a track. If I've missed something, please administer a proper cluebatting.
Meanwhile, there are of course options to store either global state or project-level state, and one could store project state using the track UUID as the key.
There are two problems with this approach though:
1. User data is not carried over when the track is duplicated
2. The script must periodically sweep the stored data to make sure that data is properly cleaned up for deleted tracks.
#2 is a minor inconvenience, but #1 is something I don't have a solution for.
Are there any APIs to deal with this use-case, or maybe some common patterns that others use to solve the problem?
Thanks!
|
|
|
10-24-2016, 07:51 AM
|
#2
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
You can't add custom data attributes to a track, you have to deal with GUID and ProjectExtState
You can make custom actions to copy paste your particular data from one track to another, or most accurately, to associate a new track GUI to a certain ProjExtState data.
What are you trying to do ?
|
|
|
10-24-2016, 12:24 PM
|
#3
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
I have a long-running script that provides a UI related to the current track. The user is allowed to make changes in the UI that are per-track.
I wanted it so that if the user duplicates the track those changes are carried along. I could add a custom action to duplicate tracks to preserve the metadata associated with my script, but that does require users to switch their shortcuts to it, and of course only works for exactly one such script.
Thanks X-Raym!
|
|
|
10-24-2016, 12:26 PM
|
#4
|
Human being with feelings
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 8,062
|
It would sure be nice to be able to add custom properties into the various objects in Reaper : tracks, items, takes, fx instances...
__________________
I am no longer part of the REAPER community. Please don't contact me with any REAPER-related issues.
|
|
|
10-24-2016, 05:29 PM
|
#5
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
Yeah. It's not the first time I've wanted to do this.
Meanwhile, to handle the single track duplication use-case, I guess I can leverage the fact that duplicating the tracks automatically selects the newly created track. I can detect the track change, and if the new track GUID isn't found in the project state and it has the same name as the previously select tracked, then my script can clone the userdata to the new track GUID.
I think that's a workable pattern.
|
|
|
03-24-2018, 05:37 AM
|
#6
|
Human being with feelings
Join Date: Aug 2012
Location: Finland
Posts: 2,668
|
Quote:
Originally Posted by Xenakios
It would sure be nice to be able to add custom properties into the various objects in Reaper : tracks, items, takes, fx instances...
|
This would be so great! Has anyone made a feature request yet?
|
|
|
03-24-2018, 05:56 AM
|
#7
|
Human being with feelings
Join Date: Aug 2012
Location: Finland
Posts: 2,668
|
Can we "attach" anything (values/tables etc.) to Lua userdata? I mean, is it possible to treat userdata as it was a Lua table? (I can't explain it better )
Maybe by using Lua metatables/methods. Too difficult for me, though.
|
|
|
03-24-2018, 07:23 AM
|
#8
|
Human being with feelings
Join Date: Feb 2017
Posts: 4,812
|
|
|
|
03-24-2018, 08:03 AM
|
#9
|
Human being with feelings
Join Date: Aug 2012
Location: Finland
Posts: 2,668
|
Thanks deeb!
|
|
|
03-24-2018, 09:02 AM
|
#10
|
Human being with feelings
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,096
|
Quote:
Originally Posted by spk77
Can we "attach" anything (values/tables etc.) to Lua userdata? I mean, is it possible to treat userdata as it was a Lua table? (I can't explain it better )
Maybe by using Lua metatables/methods. Too difficult for me, though.
|
I think the way to go would be table serialization (if I understand correctly), haven't done it myself though.
E.g.
https://forum.cockos.com/showthread.php?t=170820
https://forum.cockos.com/showthread.php?t=191009
|
|
|
03-24-2018, 09:38 AM
|
#11
|
Human being with feelings
Join Date: Aug 2012
Location: Finland
Posts: 2,668
|
Thanks nofish!
By userdata, I mean "Lua userdata" (it's a Lua type), sorry for the confusion.
This is what I'd like to have
Code:
local tr = reaper.GetTrack(0,0) -- the type of tr is "userdata"
tr.tag = "Guitar"
This might be something related (I don't even know for sure) - see the test code/output at the end of this thread http://lua-users.org/lists/lua-l/2006-08/msg00245.html
|
|
|
03-24-2018, 10:12 AM
|
#12
|
Human being with feelings
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 8,062
|
Quote:
Originally Posted by spk77
Code:
local tr = reaper.GetTrack(0,0) -- the type of tr is "userdata"
tr.tag = "Guitar"
|
In that, "tr" is light userdata, which is just a dumb raw C pointer without any properties that could get, set or added.
__________________
I am no longer part of the REAPER community. Please don't contact me with any REAPER-related issues.
|
|
|
03-24-2018, 10:15 AM
|
#13
|
Human being with feelings
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,096
|
Oh, misunderstood, sorry...
|
|
|
03-24-2018, 10:21 AM
|
#14
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
I've a solution to this problem where I store serialized data to a custom JSFX using this hack. And then read it back and deserialize in the same way.
It'd definitely be nice to have native support in Reaper though, without the need for a custom JSFX.
|
|
|
03-24-2018, 10:22 AM
|
#15
|
Human being with feelings
Join Date: May 2015
Location: Québec, Canada
Posts: 4,937
|
Quote:
Originally Posted by spk77
Can we "attach" anything (values/tables etc.) to Lua userdata? I mean, is it possible to treat userdata as it was a Lua table? (I can't explain it better )
This is what I'd like to have
Code:
local tr = reaper.GetTrack(0,0) -- the type of tr is "userdata"
tr.tag = "Guitar"
|
Code:
local tr = {ptr=reaper.GetTrack(0,0)}
tr.tag = "Guitar"
-- tr.ptr
|
|
|
03-24-2018, 04:21 PM
|
#16
|
Human being with feelings
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 8,062
|
Quote:
Originally Posted by cfillion
Code:
local tr = {ptr=reaper.GetTrack(0,0)}
tr.tag = "Guitar"
-- tr.ptr
|
This doesn't in any way solve the problem of keeping that data around in the Reaper project files etc, though.
__________________
I am no longer part of the REAPER community. Please don't contact me with any REAPER-related issues.
|
|
|
03-25-2018, 01:00 AM
|
#17
|
Human being with feelings
Join Date: Aug 2012
Location: Finland
Posts: 2,668
|
Quote:
Originally Posted by Xenakios
In that, "tr" is light userdata, which is just a dumb raw C pointer without any properties that could get, set or added.
|
I guess I mixed light userdata and full userdata. - full userdata is an object
- light userdata represents a C pointer value
This is quite off-topic but interesting: ( https://www.lua.org/pil/28.5.html)
28.5 – Light Userdata
The userdata that we have been using until now is called full userdata. Lua offers another kind of userdata, called light userdata.
A light userdatum is a value that represents a C pointer (that is, a void * value). Because it is a value, we do not create them (in the same way that we do not create numbers). To put a light userdatum into the stack, we use lua_pushlightuserdata:
void lua_pushlightuserdata (lua_State *L, void *p);
Despite their common name, light userdata are quite different from full userdata. Light userdata are not buffers, but single pointers. They have no metatables. Like numbers, light userdata do not need to be managed by the garbage collector (and are not).
Some people use light userdata as a cheap alternative to full userdata. This is not a typical use, however. First, with light userdata you have to manage memory by yourself, because they are not subject to garbage collection. Second, despite the name, full userdata are inexpensive, too. They add little overhead compared to a malloc for the given memory size.
The real use of light userdata comes from equality. As a full userdata is an object, it is only equal to itself. A light userdata, on the other hand, represents a C pointer value. As such, it is equal to any userdata that represents the same pointer. Therefore, we can use light userdata to find C objects inside Lua.
As a typical example, suppose we are implementing a binding between Lua and a Window system. In this binding, we use full userdata to represent windows. (Each userdatum may contain the whole window structure or only a pointer to a window created by the system.) When there is an event inside a window (e.g., a mouse click), the system calls a specific callback, identifying the window by its address. To pass the callback to Lua, we must find the userdata that represents the given window. To find this userdata, we can keep a table where the indices are light userdata with the window addresses and the values are the full userdata that represent the windows in Lua. Once we have a window address, we push it into the API stack as a light userdata and use the userdata as an index into that table. (Note that the table should have weak values. Otherwise, those full userdata would never be collected.)
Quote:
Originally Posted by cfillion
Code:
local tr = {ptr=reaper.GetTrack(0,0)}
tr.tag = "Guitar"
-- tr.ptr
|
Thanks, that's how I'm doing it more or less at the moment.
|
|
|
11-25-2021, 10:43 PM
|
#18
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
GetSetMediaTrackInfo_String() added in 6.36
|
|
|
11-26-2021, 12:30 AM
|
#19
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@MonkeyBars
The function you mentioned isn't added in v6.36, it is a very old one.
What is new is extension prefix as parameters values.
Here is the changelog:
Code:
+ ReaScript: add P_EXT: prefix for extension-specific string state for GetSetMediaTrackInfo_String(), GetSetAutomationItemInfo_String(), etc
+ ReaScript: allow using various GetSet..Value() functions with strings, automatically converting numbers to/from string
I never used this, not sure what the limitations if any, thx for having bring me that to my attention :P
Last edited by X-Raym; 11-26-2021 at 01:19 AM.
|
|
|
11-26-2021, 06:06 AM
|
#20
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
I might have followed up on this thread and mentioned this. Yeah I've been using GetSetMediaTrackInfo_String() to store track metadata for some time in my dev branch of Reaticulate. It's been working great.
Code:
reaper.GetSetMediaTrackInfo_String(track, 'P_EXT:yourappnamehere', data, true)
Thanks for bumping, MonkeyBars.
|
|
|
11-26-2021, 07:06 AM
|
#21
|
Human being with feelings
Join Date: Jun 2012
Location: Spain
Posts: 7,239
|
I gave up on using P_EXT on tracks because it was causing
performance issues when used on tracks with big VST chunks data.. like Kontakt etc...
Maybe now it works well. I may try it again.
|
|
|
11-26-2021, 08:14 AM
|
#22
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Quote:
Originally Posted by X-Raym
The function you mentioned isn't added in v6.36, it is a very old one. What is new is extension prefix as parameters values.
|
That's the important one for this thread!
Quote:
I never used this, not sure what the limitations if any, thx for having bring me that to my attention :P
|
I can't believe I turned X-Raym onto something he didn't know in ReaScript
Quote:
Originally Posted by tack
I might have followed up on this thread and mentioned this. Yeah I've been using GetSetMediaTrackInfo_String() to store track metadata for some time in my dev branch of Reaticulate. It's been working great. Thanks for bumping, MonkeyBars.
|
That's awesome! Yes P_EXT is extremely useful, my pleasure!
Quote:
Originally Posted by heda
I gave up on using P_EXT on tracks because it was causing
performance issues when used on tracks with big VST chunks data.. like Kontakt etc...
Maybe now it works well. I may try it again.
|
Oh no, that's too bad! Were you storing those large chunks of data in P_EXT or were they otherwise present?
I am planning to store a (pickled?) table of item GUIDs that could get pretty big on a large project, but likely nothing as large as FX chunk data I imagine (unless you had a million items or something... you never know with these game devs and other specialized pros).
|
|
|
11-26-2021, 10:08 AM
|
#23
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
Quote:
Originally Posted by heda
I gave up on using P_EXT on tracks because it was causing performance issues when used on tracks with big VST chunks data.. like Kontakt etc...
|
I haven't seen any obvious performance issues. I just did a quick test on a track with a few VSTis, including Kontakt with 10 patches (section patches from Spitfire Chamber Strings and Cinematic Studio Strings), with a track state chunk totaling about 15MB, and the call to GetSetMediaTrackInfo_String() takes around 15-30μs on my system. And there is no obvious scaling of this time with respect to the number of FX or Kontakt patches.
So if you give it another try and still run into the performance issue you observed, track chunk size might be a red herring, or perhaps you're talking about track states much larger than 15MB? (Or I've entirely misinterpreted the nature of the performance issue you had. )
|
|
|
11-26-2021, 10:20 AM
|
#24
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Quote:
Originally Posted by tack
I haven't seen any obvious performance issues. I just did a quick test on a track with a few VSTis, including Kontakt with 10 patches (section patches from Spitfire Chamber Strings and Cinematic Studio Strings), with a track state chunk totaling about 15MB, and the call to GetSetMediaTrackInfo_String() takes around 15-30μs on my system. And there is no obvious scaling of this time with respect to the number of FX or Kontakt patches.
|
Thanks for testing, tack!
|
|
|
11-26-2021, 10:25 AM
|
#25
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Since I've got you guys here, just wanted to point out my question on this other thread https://forum.cockos.com/showthread....78#post2500578
I want to save project data that gets saved to the Undo History, but it seems there's no API hook to do so – both Project ExtState and Project Info Strings/Data are outside Undo History.
I'd rather not have to create a hidden dummy track just to save my script data, but at this point I'm at a loss as to where I could place script data that gets reverted/restored on undo/redo. Any ideas, or is track P_EXT the best concession here?
|
|
|
11-26-2021, 10:50 AM
|
#26
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
Quote:
Originally Posted by MonkeyBars
I want to save project data that gets saved to the Undo History, but it seems there's no API hook to do so – both Project ExtState and Project Info Strings/Data are outside Undo History.
|
I am commonly fighting against REAPER's undo system when it comes to APIs. I wish it offered more control over around when and whether API calls would modify undo history.
Quote:
Originally Posted by MonkeyBars
I'd rather not have to create a hidden dummy track just to save my script data, but at this point I'm at a loss as to where I could place script data that gets reverted/restored on undo/redo. Any ideas, or is track P_EXT the best concession here?
|
Could you store the P_EXT state on the master track?
|
|
|
11-26-2021, 10:56 AM
|
#27
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Quote:
Originally Posted by tack
Could you store the P_EXT state on the master track?
|
GREAT idea!! The master track is just track #0, right??
|
|
|
11-26-2021, 11:00 AM
|
#28
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
Quote:
Originally Posted by MonkeyBars
GREAT idea!! The master track is just track #0, right??
|
Call GetMasterTrack() to fetch the track object for the master track, and you can pass that to GetSetMediaTrackInfo_String().
|
|
|
11-26-2021, 11:02 AM
|
#29
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Roger that! Thank you so much! This is definitely the solution.
|
|
|
11-26-2021, 04:16 PM
|
#30
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
I just did a series of tests and it appears using the Master Track to store data whose deltas are stored in Undo History is NOT an option after all! Darn.
Hopefully this chart helps other script devs. Final results:
SetProjExtState
OUTSIDE Undo History
Master Track GetSetMediaTrackInfo_String P_EXT
OUTSIDE Undo History
Regular Track GetSetMediaTrackInfo_String P_EXT
INSIDE Undo History
GetSetMediaItemInfo_String P_EXT
INSIDE Undo History
Surely safe to assume envelope and automation item P_EXT are also inside the Undo History.
My new FR here: https://forum.cockos.com/showthread....52#post2500852
Last edited by MonkeyBars; 11-26-2021 at 11:58 PM.
|
|
|
11-27-2021, 01:54 AM
|
#31
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Looks like Master Track Tempo Envelope changes get entered into undo points, so I'm using its P_EXT!
|
|
|
11-27-2021, 01:54 PM
|
#32
|
Human being with feelings
Join Date: Jan 2014
Location: Ontario, Canada
Posts: 1,619
|
Quote:
Originally Posted by MonkeyBars
Looks like Master Track Tempo Envelope changes get entered into undo points, so I'm using its P_EXT!
|
Nice discovery. What modifies undo vs what doesn't feels a little ill conceived to me, so there's a risk this ends up being brittle. On the other hand, my intuition is that if something does change, it's likely to include master track updates via GetSetMediaTrackInfo_String() in undo history, rather than excluding tempo envelope.
Thanks for sharing!
|
|
|
11-27-2021, 02:16 PM
|
#33
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Quote:
Originally Posted by tack
Nice discovery. What modifies undo vs what doesn't feels a little ill conceived to me, so there's a risk this ends up being brittle. On the other hand, my intuition is that if something does change, it's likely to include master track updates via GetSetMediaTrackInfo_String() in undo history, rather than excluding tempo envelope.
|
Well put. That's also my worry, but then I am not able to deactivate the tempo envelope, so there's certainly no danger of the entire tempo envelope somehow getting eradicated.
Come to think of it, that's a bit of a UX glitch showing a box to deactivate the tempo env as if it were any other deactivtable env when it ain't.
My theory is that master track data is partially saved with project data (outside undo history), and partially as its own track (inside undo history).
My pleasure sir! H/t @Julian for the idea
|
|
|
11-30-2021, 11:55 PM
|
#34
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Yep this is working great for my project
|
|
|
12-08-2021, 10:03 AM
|
#35
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Mespotine and I found some small API bugs that affect using this technique.
Last edited by MonkeyBars; 12-09-2021 at 12:21 AM.
|
|
|
12-09-2021, 12:21 AM
|
#36
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Turns out P_EXT was never officially supported for Master Track which explains its inconsistent behavior.
However, schwa is working on it for the next release! Actually, in the latest dev version, changes in the Master Track (non-env) P_EXT are now being saved in undo points.
Last edited by MonkeyBars; 08-30-2022 at 09:28 PM.
|
|
|
01-19-2022, 05:07 PM
|
#37
|
Human being with feelings
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
|
Master Track P_EXT support was officially added in v6.43.
|
|
|
Thread Tools |
|
Display Modes |
Linear Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT -7. The time now is 07:09 PM.
|