Cockos Incorporated Forums EEL2 - string and table handling demystified
 User Name Remember Me? Password
 Register Search Today's Posts Mark Forums Read

 Thread Tools Display Modes
 07-06-2022, 02:16 PM #1 papagirafe Human being with feelings     Join Date: Aug 2020 Location: Brasil Posts: 267 EEL2 - string and table handling demystified Hi folks! As you most probably know, coding video preset is done in the EEL2. This is a powerfull programming language, quite easy to learn but characters strings handling is a total confusion. I am provinding here a piece of code to help understand the inner workings of strings. But first a bit of context: EEL documentation To add to the documentation: Apart from regular variables known as "named spaces", EEL has 3 sets of globally shared indexed memory slots (aka tables) for your convenience: 1) the main one that contains about 8 millions slots to store numbers that are adressable through this notation: base[index] 'base' and 'index' are regular name space variables. so the memory slot number is 'base + index' 2) the string table that contains 1024 entries. You address a string slot by assigning a value to a regular variable, ex: 'a=18' and strcpy() a value in it 3) gmem table: similar to #1 but shared among video presets and fx Code: ```//String and table operations demystified // by papagirafe // mutable strings operations a=5; // arbitrarily set string slot index to 5 strcpy(a,"memory slot #5"); //copy some stuff in there strcpy(a+1,"memory slot #6 (=5+1)"); z=10; //set base to 10th element in the main indexed mem space z[0]=5; //store a reference to 'a' in main indexed address space #z=z[0]; //copy string slot referenced by z[0] (i.e. 5) to named string #z z[1]=a+1; // z[0+1] = next string slot = 6 #z1=z[1]; // confirmed! == content of slot a+1 zz=z-5; //let's try an index base lowered by 5 #z2=zz[5]; //zz[5] == z[0] == string slot #5 = slot(a) strcat(z[0]," confirmed"); //append string to string slot #5 through z[0] #z0=a; //confirmed! we really modified slot refered by 'a' //litteral number = direct access to string slot #ms1000=ms1000=strcpy(1000,"memory slot #1000"); //but there is a maximum 1024 string slots, this operation fails: (empty string) #ms2000=ms2000=strcpy(2000,"memory slot #2000, maximum = 1024"); // immutable string operations im="immutable"; //if initialized this way, string is immutable. strcat(im," really???"); #im=im; //really immutable #x5=x5=#x[5]="x5"; //named strings have independent index slots but become immutable #check_a=a; //indeed slot #5 refered by a is untouched strcat(#x[5]," is mutable?"); #im2=#x[5]; // really not mutable #x2000=x2000=#x[2000]="x2000"; //x[n] is not limited to 1024 slots #str=str=str[5]="abcd"; //set a new immutable string with same index number as 'a' and 'x' #x5_is_untouched=x5_is_untouched=#x[5]; #a_is_untouched=a_is_untouched=a;``` Last edited by papagirafe; 07-07-2022 at 03:28 AM. Reason: code clarification
 07-06-2022, 05:03 PM #2 ashcat_lt Human being with feelings   Join Date: Dec 2012 Posts: 6,638 If you explicitly declare a gmem space, it can be separate from other gmems, and also has more slots. I have developed a few functions for array/table management which makes working with the memory buffers (local and gmem) a lot easier especially when trying to navigate and manipulate multiple multidimensional arrays. Other people have done very similar things, but their ways don’t really make sense to me for whatever reason, so I built my own. It’s not exactly exhaustive, but covers most basic functions and some specialty things which came up in my experiments. __________________ the Last Page --- Ash's Tube --- Join the Partnership for a Drum Free Amerika
07-06-2022, 05:55 PM   #3
papagirafe
Human being with feelings

Join Date: Aug 2020
Location: Brasil
Posts: 267

Quote:
 Originally Posted by ashcat_lt If you explicitly declare a gmem space, it can be separate from other gmems, and also has more slots. I have developed a few functions for array/table management which makes working with the memory buffers (local and gmem) a lot easier especially when trying to navigate and manipulate multiple multidimensional arrays. Other people have done very similar things, but their ways don’t really make sense to me for whatever reason, so I built my own. It’s not exactly exhaustive, but covers most basic functions and some specialty things which came up in my experiments.
Very interesting stuff you are describing there. So far, I only built a few test functions around arrays (like a range allocator) but I found alternative to gmem[] that allows multiple shared buffers in one script (albeit in read only mode).

 07-08-2022, 07:45 AM #4 Justin Administrator     Join Date: Jan 2005 Location: NYC Posts: 14,397 1) EEL2 memory is a flat space of memory, with millions of entries (the exact size can be queried by __memtop()). You can access memory by using the [] operators. So you can do: Code: ```a = 1; b = 2; a[] = 100; // write 100 to memory offset 1 a[b] = 100; // write 100 to memory offset 3 (1+2) b[a] = 100; // write 100 to memory offset 3 (2+1) (a+b)[] = 100; // write 100 to memory offset 3 (1+2), these are all the same :)``` There are not tables. You can emulate table syntax by setting up regions of memory as an array, but you need to make sure it doesn't overlap with any other memory and 2) Functions that work with strings use numbers to identify the string, which you can either pass directly or store in a variable. Examples: Code: ```a = 1; // use string index 1, 0..1023 are available for general use a = #; // generate a unique string index at compile-time (this will always be >=1024). If you have multiple # statements in code they will all get different unique numbers. a = "whatever"; // generate a (read-only) string index for "whatever". If you have multiple instances of "whatever" in your code, you may or may not end up with different numbers representing them. a = #name; // generate a unique string index that is associated with "#name" at compile time. If you have multiple #name instances they will all get the same number. ... strcpy(a,"foo"); // copy "foo" to whatever a points to``` That's it. If you read/write these numbers from/to normal EEL2 memory, they work just the same (they reference the string). Oh one more thing, there's a shorthand syntax (only applies to #strings): Code: `#foo = anything; // shorthand for strcpy(#foo, anything);`
 07-08-2022, 10:13 AM #5 ashcat_lt Human being with feelings   Join Date: Dec 2012 Posts: 6,638 The memory stuff trips a lot of people up, but it really is that simple. There is one big on dimensional array and we have to “manually” portion that out to hold however many arrays of however many dimensions as we want. We have to find ways to fold our multidimensional sets down to that one long string of slots and keep track of where they are. There are different strategies for achieving that. If you analyze a few different JS delay plugins you’ll see at least a couple different ways. It’s actually not that hard but it does take some planning and housekeeping. And a lot of redundant code, and… So I made a couple functions to make it a little cleaner and easier for my old object-oriented database driven brain to work with. I use namespaces and pseudoobjects to define my table/array and help find specific data indexes without having to type the same rather awkward math statement over and over. I’ve been meaning to post that work, but have plenty of lame excuses. Namespaces of course can also be global, and even shared between JS and VP and I’d imagine even scripts, though I don’t much mess with that side. But I think it really is best practice when using gmem, to use your own explicitly declared gmem space just to keep things from “leaking” and causing kind of invisible or at least opaque conflicts. Even then, though, we need to be careful. The video peeker plugin and VP presets which implement it use their own gmem space, but if you want to have a second set, you have to give them their own new space. That’s easy enough in VP, but on the JS side, you have to actually make a new copy of the plugin which starts to get awkward and is why I made the 64 channel version work. I haven’t worked a lot with strings. I wanted them to let me dynamically access namespaces and/or functions, but that totally doesn’t work. Otherwise I haven’t much seen the need until just recently when JS got that thing with the debug messages in the description line which makes…well, debugging…so much easier. Edit - Well I’m gonna keep babbling for a while on somewhat tangential subjects. My video delay presets don’t really use this memory array. In fact, it kind of goes the opposite direction. I’ve got a one dimensional array of frames that I am tiling into a grid on a much larger frame. For long delay times, we end up needing more than one of those buffer frames, so I end up projecting a one dimensional array onto a three dimensional space. Of course, the individual frame itself IS a multidimensional array. In fact, it’s kind of a two dimensional array of four element arrays, and if you’ve got data which can fit in 16 bit boxes, you really can use it as such. I have never really wanted this, but it is there, and one begins to wonder if some of our gfx functions could help us with certain table processing functions. Like, can an additive blit is a lot more elegant than looping through two tables just to add their values. And what kind of whacky stuff could we do with xformblit. Isn’t there one where we can literally pass in code as a string??? But actually I kind of lied. I did do a thing which translates “tables” to pixels and back, and even used that to pass video from JS to VP, which…. We’ve talked about this before and I honestly haven’t gone any further on it, but there is the issue of synchronization between asynched threads. I’m think that if we take a clue from the video peeker - it writes a buffer of samples and a sort of “current time” index to gmem - and combine it with the oversized offscreen “buffer” frame from my video delay… IDK for sure how that helps except that there is a frustrating disconnect between the gfx functions in JS and VP. (Why in the TF can’t I draw a circle in VP?!? F even a line might be nice once in a while) Also I suppose JS has a more direct connection to the audio stream. And if we could find a relatively easy way to adapt a JS plugin so that its display was available for our video projects… __________________ the Last Page --- Ash's Tube --- Join the Partnership for a Drum Free Amerika Last edited by ashcat_lt; 07-08-2022 at 10:49 AM.
07-09-2022, 05:16 AM   #6
papagirafe
Human being with feelings

Join Date: Aug 2020
Location: Brasil
Posts: 267

Quote:
 Originally Posted by Justin 1) EEL2 memory is a flat space of memory, with millions of entries (the exact size can be queried by __memtop()). You can access memory by using the [] operators. So you can do: Code: ```a = 1; b = 2; a[] = 100; // write 100 to memory offset 1 a[b] = 100; // write 100 to memory offset 3 (1+2) b[a] = 100; // write 100 to memory offset 3 (2+1) (a+b)[] = 100; // write 100 to memory offset 3 (1+2), these are all the same :)``` There are not tables. You can emulate table syntax by setting up regions of memory as an array, but you need to make sure it doesn't overlap with any other memory and 2) Functions that work with strings use numbers to identify the string, which you can either pass directly or store in a variable. Examples: Code: ```a = 1; // use string index 1, 0..1023 are available for general use a = #; // generate a unique string index at compile-time (this will always be >=1024). If you have multiple # statements in code they will all get different unique numbers. a = "whatever"; // generate a (read-only) string index for "whatever". If you have multiple instances of "whatever" in your code, you may or may not end up with different numbers representing them. a = #name; // generate a unique string index that is associated with "#name" at compile time. If you have multiple #name instances they will all get the same number. ... strcpy(a,"foo"); // copy "foo" to whatever a points to``` That's it. If you read/write these numbers from/to normal EEL2 memory, they work just the same (they reference the string). Oh one more thing, there's a shorthand syntax (only applies to #strings): Code: `#foo = anything; // shorthand for strcpy(#foo, anything);`
Thanks for the extra clarifications Justin. It is true that I abused the word "table" in there but for most programmers an indexed memory space or a one dimensional table are pretty much the same concept unfortunately.

07-14-2022, 01:14 PM   #7
Justin

Join Date: Jan 2005
Location: NYC
Posts: 14,397

Quote:
 Originally Posted by ashcat_lt IDK for sure how that helps except that there is a frustrating disconnect between the gfx functions in JS and VP. (Why in the TF can’t I draw a circle in VP?!? F even a line might be nice once in a while) Also I suppose JS has a more direct connection to the audio stream. And if we could find a relatively easy way to adapt a JS plugin so that its display was available for our video projects…
Thing is, the yv12/yuy2 colorspaces are subsampled in their U/V, so things like lines and circles are a mess to do well... so we've punted on that and made scripts do it themselves

07-14-2022, 01:22 PM   #8
papagirafe
Human being with feelings

Join Date: Aug 2020
Location: Brasil
Posts: 267

Quote:
 Originally Posted by Justin Thing is, the yv12/yuy2 colorspaces are subsampled in their U/V, so things like lines and circles are a mess to do well... so we've punted on that and made scripts do it themselves
At some point I considered creating a vector font with a few characters representing basic geometric shapes (like circle) ...

 07-14-2022, 01:36 PM #9 ashcat_lt Human being with feelings   Join Date: Dec 2012 Posts: 6,638 My “easy” solution for circles is to just draw one huge circle off screen at init (which of course we have to fake in VP) and then use blit tricks to get them where I actually want them. I have what I think is a decent “line between” function that I was working on, too. __________________ the Last Page --- Ash's Tube --- Join the Partnership for a Drum Free Amerika
07-14-2022, 06:35 PM   #10
papagirafe
Human being with feelings

Join Date: Aug 2020
Location: Brasil
Posts: 267

Quote:
 Originally Posted by ashcat_lt My “easy” solution for circles is to just draw one huge circle off screen at init (which of course we have to fake in VP) and then use blit tricks to get them where I actually want them. I have what I think is a decent “line between” function that I was working on, too.
clever!

 Thread Tools Display Modes Linear Mode

 Posting Rules You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is Off Forum Rules
 Forum Jump User Control Panel Private Messages Subscriptions Who's Online Search Forums Forums Home REAPER Forums     REAPER General Discussion Forum     newbieland     REAPER Q&A, Tips, Tricks and Howto     Recording Technologies and Techniques     REAPER Compatibility     REAPER Color Themes and Icon Sets     MIDI Hardware, Control Surfaces, and OSC     REAPER Non-English Speaking User Forums         Forum de REAPER en français         Foro de REAPER en Español         Fórum do REAPER em português         Forum di REAPER in italiano         Deutschsprachiges REAPER Userforum         Pyccкоязычный фopyм REAPER     REAPER Bug Reports     REAPER Feature Requests     Dstruct's Casa De Nitpicks     REAPER for Live Use     REAPER for Video Editing/Mangling     REAPER for Spatial Audio     ReaScript, JSFX, REAPER Plug-in Extensions, Developer Forum     REAPER for macOS     REAPER for Linux     REAPER Pre-Release Discussion     REAPER Music/Collaboration Discussion NINJAM Discussion     NINJAM User Discussion     NINJAM Developer Discussion Other Software Discussion     WDL users forum     LICEcap Discussion     OSCII-bot forum     Old Cockos Products Forum

All times are GMT -7. The time now is 08:21 PM.

 -- Cockos ---- REAPER 5 ---- Reaper 3 ---- Reaper 2 ---- Reaper 1 Contact Us - Çockos Incorporated - Archive - Top