Old 11-01-2015, 09:59 PM   #1
Solstice
Human being with feelings
 
Join Date: Oct 2015
Posts: 49
Default Defer() speed

I would like to run a function periodically(say every 10 seconds). I can use defer but how can I prevent wasting cycles?

Will time.sleep(10) work well?
Solstice is offline   Reply With Quote
Old 11-02-2015, 03:10 AM   #2
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 8,062
Default

Quote:
Originally Posted by Solstice View Post
I would like to run a function periodically(say every 10 seconds). I can use defer but how can I prevent wasting cycles?

Will time.sleep(10) work well?
I think doing a sleep would go horribly wrong. The deferred functions are run in the Reaper GUI thread, so if you sleep, I would expect Reaper's GUI to freeze. (In any case, stock Lua does not provide a sleep() function anyway.)

What you need to do is to yourself keep track of how much time has passed between the defer calls.
__________________
I am no longer part of the REAPER community. Please don't contact me with any REAPER-related issues.
Xenakios is offline   Reply With Quote
Old 11-02-2015, 05:36 AM   #3
kenz
Human being with feelings
 
Join Date: Aug 2013
Posts: 339
Default

I used sleep in EEL once for similar purpose, and yes it does freeze the entire GUI. Frankly this makes sleep completely pointless.

Would be cool to have the script be able to run some code in a different thread, at least in EEL it should be easier since they made it in-house.
kenz is offline   Reply With Quote
Old 11-02-2015, 06:30 AM   #4
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,751
Default

Don't call sleep() .. this will make the UI thread sleep, which is never what you want

Instead, either keep track of time or count calls yourself, like this:

Code:
starttime=os.time()
lasttime=starttime
loopcount=0

function runloop()
  
  local newtime=os.time()
  if newtime-lasttime >= 1 then
    lasttime=newtime
    reaper.ShowConsoleMsg("1 second\n")
  end
  
  -- or
  
  loopcount=loopcount+1
  if loopcount >= 100 then
    loopcount=0
    reaper.ShowConsoleMsg("100 loop calls\n");
  end
  
  
  if newtime < starttime+10 then 
    reaper.defer(runloop) 
  end

end

reaper.defer(runloop)

Last edited by schwa; 11-02-2015 at 07:35 AM.
schwa is offline   Reply With Quote
Old 11-02-2015, 08:24 AM   #5
kenz
Human being with feelings
 
Join Date: Aug 2013
Posts: 339
Default

Would it be possible to add some sort of timer function, like how AHK does it...? Like instead of 'defer' have a function that executes its function argument after X amount of time and so on without actually polling for the condition?

I really hate the idea of polling like we're forced to do here, I just think it's bad code design whenever I hear about it, when event-driven design exists. Reminds me of DOS era software.

i.e
Code:
function blah()
(
  // do something here

  defer_timer("blah()", 4.5);  // execute it again after 4.5 sec
);

blah();
kenz is offline   Reply With Quote
Old 11-02-2015, 08:43 AM   #6
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,751
Default

Polling is the really the canonical way to implement "wait without blocking" when you're not waiting for the result of some specific other calculation or process. It's not bad code design. We could add a timer argument to the defer function but we would be implementing it internally more or less the same way.


(AHK's timer is implemented via polling...)
schwa is offline   Reply With Quote
Old 11-02-2015, 08:55 AM   #7
kenz
Human being with feelings
 
Join Date: Aug 2013
Posts: 339
Default

Yeah but by polling I mean polling on every 'frame' (not sure how it's called?) for every single timer. I don't know how AHK internally implements it, but I imagine it's not hard to just not check every timer everytime. Like keep a sorted list of all timers but only keep the differences between one timer and the other (because adding a new timer is not as important; like when one timer expires and you execute the function). The overhead here is when you have tons of stuff in the background that executes rarely, not the list sort insertion.

Even though that may be a little complicated internally, but isn't it possible to use Windows API for it? IIRC back in the days when I used to code for it there were several event-based APIs available.


I have a better idea, with minimal API requirements. The simplest 'raw' way to do it would simply be to spawn a new thread that governs all the timers. This thread simply finds the 'minimum' timer and suspends itself for that amount. Then, it places that timer's function into a synchronized list for the main GUI thread to execute on the next frame... While the second thread re-adds that timer to the list and finds the next minimum timer again and suspends itself and so forth. Some simple atomic synchronization is needed for it to communicate with the main thread, but easy enough.

This is in a different thread. So when a new script is run with a new timer, it forcefully resumes that second thread (or terminates it and re-creates it) with the new timer added to the list, but subtracting the amount in suspension already in that thread from all other timers. (i.e in the second thread simply GetTickCount() before it is suspended, and when it's resumed check for the signal from the GUI thread -- if new timer was added, subtract the diff in GetTickCount from previous stored value before the suspension etc).

(obviously if it terminates the thread, the second thread needs to store its GetTickCount just before it got suspended/sleep mode in some global variable, so it can be accessed by main thread or the newer re-created thread)

(not sure about OS X but clearly it must have something similar, if not well it's a fail OS )

Yes there would be an overhead when adding a new timer from a new script, but really that's not a big deal compared to polling. You don't execute new scripts every frame and I'm sure that compiling those scripts is quite some overhead too! The polling overhead comes from having many scripts in the background, that don't even execute their deferred code often etc.

Anyway this is really longer to explain than I thought, but it's really no big deal, it sounds harder than it is IMO.

Also for synchronization, a Mutex would be more than enough to handle it.


EDIT:

Some "pseudo" code to get the picture cause the explanation seems lengthy:

Code:
void TimerThread(blah)
{
  while(num_timers)
  {
   uint64_t m = (uint64_t)(-1);  // assuming timers are 64-bit integers?

   int i;
   for(i = 0; i < num_timers; i++)
     m = min(timers[i], m);      // get minimum

   TimerThread_time = GetTickCount();  // global
   Sleep(m);

   // enter Mutex here, forgot API name, then add the timer's function to the list
   // so that UI thread executes it on next frame...

   for(i = 0; i < num_timers; i++)
     timers[i] -= m;

   // add the timer back to list (i.e 'm') and repeat
  }
}
Ok this is crude but something like that... TimerThread_time would be used in case a new script is added with a new timer -- in that case, the UI thread enters the mutex, terminates the TimerThread, then does:
Code:
m = GetTickCount() - TimerThread_time;
for(i = 0; i < num_timers; i++)
  timers[i] -= m;
to offset the timers with the time already passed, adds the new timer, and then it recreates the thread...

Of course this is the crudest way, I'm sure there's ways with APIs not requiring to re-create the thread. Anyway this only recreates it when a new script's timer is added -- so it's not a big overhead at all.



Lastly, I want to say that this only applies to scripts with a timer. Of course, for scripts that execute very many times a second (every frame) it's more overhead for no reason.

But those scripts won't even have timers so they will remain the same. I'm not saying to take out the current defer.

Last edited by kenz; 11-02-2015 at 09:16 AM.
kenz is offline   Reply With Quote
Old 11-02-2015, 09:50 AM   #8
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,751
Default

Lua isn't compiling on every defer call, it's just executing the bytecode. During the waiting state, the Lua code doesn't do anything but increment a variable or check the time.

Given that we are in the context of running some script periodically anyway, the cost of using Lua as a polling timer is insignificant compared to the cost of whatever the script is actually meant to do every second, or 10 seconds, or however long. Just for fun I ran 100 instances of the Lua script above and could not detect any performance effect.

In other words, if we are in some performance-critical situation, and you have a reascript that does something every 10 seconds, it's not the cost of waiting for 10 seconds that we need to optimize, it's the cost of whatever the script does every 10 seconds.

If this ever becomes a demonstrable problem we can look at it again, but trying to shave milliseconds off of the timer's waiting state seems like inappropriate optimization.

Last edited by schwa; 11-02-2015 at 10:13 AM.
schwa is offline   Reply With Quote
Old 11-02-2015, 09:57 AM   #9
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 8,062
Default

Quote:
Originally Posted by schwa View Post
but trying to shave milliseconds off of the timer's waiting state seems like inappropriate optimization.
Maybe kenz forgot to mention he is expecting ReaScript to do some task it isn't even meant to do...I get the impression he isn't after the same use case as the original poster with his 10 second timing requirement.
__________________
I am no longer part of the REAPER community. Please don't contact me with any REAPER-related issues.
Xenakios is offline   Reply With Quote
Old 11-02-2015, 10:16 AM   #10
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

defer() is very lightweight, and the script running can easily and quickly check the time and decide whether it is time yet to process some periodic task, and do so with little overhead.

IMO, it is preferable to do things this way than to have REAPER provide a more complex API trying to meet various needs.

Last edited by Justin; 11-02-2015 at 10:33 AM.
Justin is offline   Reply With Quote
Old 11-02-2015, 10:50 AM   #11
Solstice
Human being with feelings
 
Join Date: Oct 2015
Posts: 49
Default

Quote:
Originally Posted by Justin View Post
defer() is very lightweight, and the script running can easily and quickly check the time and decide whether it is time yet to process some periodic task, and do so with little overhead.

IMO, it is preferable to do things this way than to have REAPER provide a more complex API trying to meet various needs.
I agree with kenz. At least having something like defer(,time) allows reaper to internally control how the dispatching is handled.

It also reduces a lot of redundant coding and still allows the original use case.
Solstice is offline   Reply With Quote
Old 11-02-2015, 11:37 AM   #12
Solstice
Human being with feelings
 
Join Date: Oct 2015
Posts: 49
Default

Quote:
Originally Posted by schwa View Post
Don't call sleep() .. this will make the UI thread sleep, which is never what you want

Instead, either keep track of time or count calls yourself, like this:

Code:
starttime=os.time()
lasttime=starttime
loopcount=0

function runloop()
  
  local newtime=os.time()
  if newtime-lasttime >= 1 then
    lasttime=newtime
    reaper.ShowConsoleMsg("1 second\n")
  end
  
  -- or
  
  loopcount=loopcount+1
  if loopcount >= 100 then
    loopcount=0
    reaper.ShowConsoleMsg("100 loop calls\n");
  end
  
  
  if newtime < starttime+10 then 
    reaper.defer(runloop) 
  end

end

reaper.defer(runloop)

This code won't work!?

You have to call defer ever time in the loop so you can check if the function should be called.

In this case you only re-defer if newtime < start time


I'm looking for more of an interval timer... Which requires defer to be called ever time in the function. This seems slow, which was the point of the question.
Solstice is offline   Reply With Quote
Old 11-02-2015, 11:48 AM   #13
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,751
Default

This code always re-defers. It only prints stuff to the console every second, or every 100 calls. You would replace those print calls with whatever code you actually want to happen every second (or 10 seconds or whatever).

Code:
  if newtime < starttime+10 then 
    reaper.defer(runloop) 
  end
The "if" condition there just makes the whole test program exit after 10 seconds. You can remove the condition if you want the program to run forever.
schwa is offline   Reply With Quote
Old 11-02-2015, 11:51 AM   #14
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,751
Default

Maybe this is clearer?

Code:
lasttime=os.time()
loopcount=0

function runloop()
  
  local newtime=os.time()
  if newtime-lasttime >= 10 then
    lasttime=newtime
    reaper.ShowConsoleMsg("10 seconds\n")
    -- do whatever you want to do every 10 seconds
  end
  
  -- or
  
  loopcount=loopcount+1
  if loopcount >= 100 then
    loopcount=0
    reaper.ShowConsoleMsg("100 loop calls\n");
    -- do whatever you want to do every 100 loop calls
  end

  -- note, this program will run forever, until terminated by the action
  -- "close all running reascripts", which is possibly not ideal.
  reaper.defer(runloop) 

end

reaper.defer(runloop)
schwa is offline   Reply With Quote
Old 11-02-2015, 11:58 AM   #15
kenz
Human being with feelings
 
Join Date: Aug 2013
Posts: 339
Default

Quote:
Originally Posted by Justin View Post
defer() is very lightweight, and the script running can easily and quickly check the time and decide whether it is time yet to process some periodic task, and do so with little overhead.

IMO, it is preferable to do things this way than to have REAPER provide a more complex API trying to meet various needs.
On the scripting side there wouldn't be anything complex, if anything, it will simplify timer-based persistent scripts. (although it will be more complex in the program code of course, but I don't think you have to do something like I said if you ever wanted to implement this -- pretty sure Win API has some sort of threaded timer function)

To be fair, I do admit the use case is quite small so it's entirely up to you if you find it worth to implement or not.

My hate for polling is more idealistic than practical in most cases, I just find it sloppy coding in general, but in this case I reckon it's limited case so quite understandable if you are too busy with other things to worry about something smallish like this.

Oh btw, I still can't find a use for the sleep function, since it blocks the UI thread.
kenz is offline   Reply With Quote
Old 11-02-2015, 12:15 PM   #16
Solstice
Human being with feelings
 
Join Date: Oct 2015
Posts: 49
Default

Quote:
Originally Posted by schwa View Post
Maybe this is clearer?

Code:
lasttime=os.time()
loopcount=0

function runloop()
  
  local newtime=os.time()
  if newtime-lasttime >= 10 then
    lasttime=newtime
    reaper.ShowConsoleMsg("10 seconds\n")
    -- do whatever you want to do every 10 seconds
  end
  
  -- or
  
  loopcount=loopcount+1
  if loopcount >= 100 then
    loopcount=0
    reaper.ShowConsoleMsg("100 loop calls\n");
    -- do whatever you want to do every 100 loop calls
  end

  -- note, this program will run forever, until terminated by the action
  -- "close all running reascripts", which is possibly not ideal.
  reaper.defer(runloop) 

end

reaper.defer(runloop)
Yes, but don't you think this would eat up a lot of cycles by recursively calling defer as fast as possible? Just seems like it would make reaper significantly slower for absolutely no good reason. What if many scripts do this?

Could a sws solution be created? One that creates a thread to do the timing and dispatching? Something like the startup script action but for periodically executing a script?
Solstice is offline   Reply With Quote
Old 11-02-2015, 12:26 PM   #17
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,751
Default

Quote:
Originally Posted by Solstice View Post
Yes, but don't you think this would eat up a lot of cycles by recursively calling defer as fast as possible? Just seems like it would make reaper significantly slower for absolutely no good reason. What if many scripts do this?
No, it's fine. Please see posts 6, 8, and 10 above.
schwa is offline   Reply With Quote
Old 11-11-2015, 09:44 AM   #18
Solstice
Human being with feelings
 
Join Date: Oct 2015
Posts: 49
Default

Quote:
Originally Posted by schwa View Post
No, it's fine. Please see posts 6, 8, and 10 above.

I remember Bill Gates saying the same thing once. He was talking about 640k memory being the absolute maximum anyone could possibly ever need. Of course he's a genius so you can't argue with that. Good luck!
Solstice is offline   Reply With Quote
Old 11-11-2015, 10:04 AM   #19
Justin
Administrator
 
Justin's Avatar
 
Join Date: Jan 2005
Location: NYC
Posts: 15,721
Default

Quote:
Originally Posted by Solstice View Post
Yes, but don't you think this would eat up a lot of cycles by recursively calling defer as fast as possible? Just seems like it would make reaper significantly slower for absolutely no good reason. What if many scripts do this?
Defer() does not get called as fast as possible, it only gets called at about 30-50Hz. At this rate, even a hundred scripts executing a timestamp check each tick is not significant.

Quote:
I remember Bill Gates saying the same thing once. He was talking about 640k memory being the absolute maximum anyone could possibly ever need. Of course he's a genius so you can't argue with that. Good luck!
Whatever
Justin is offline   Reply With Quote
Old 11-13-2015, 05:46 PM   #20
Solstice
Human being with feelings
 
Join Date: Oct 2015
Posts: 49
Default

Quote:
Originally Posted by Justin View Post
Defer() does not get called as fast as possible, it only gets called at about 30-50Hz. At this rate, even a hundred scripts executing a timestamp check each tick is not significant.
Ok, thanks for the info! I can write a scheduler without worrying then.
Solstice is offline   Reply With Quote
Old 01-19-2022, 09:48 AM   #21
MonkeyBars
Human being with feelings
 
MonkeyBars's Avatar
 
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
Default

This is a really helpful, informative thread. Thanks so much!

I want to use defer for my script but don't need full speed and was thinking I would only do like 5 checks per second, but I don't even think my function will take any longer than comparing to the OS clock and waiting, so might as well just run it at 30hz default.
MonkeyBars is offline   Reply With Quote
Old 01-19-2022, 02:22 PM   #22
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,096
Default

I also use it in some of my defer scripts to slow down (thanks schwa), e.g.
https://github.com/nofishonfriday/Re...pooled.lua#L29
nofish is offline   Reply With Quote
Old 01-19-2022, 03:56 PM   #23
MonkeyBars
Human being with feelings
 
MonkeyBars's Avatar
 
Join Date: Feb 2016
Location: Hollyweird
Posts: 2,630
Default

Quote:
Originally Posted by nofish View Post
I also use it in some of my defer scripts to slow down (thanks schwa), e.g.
https://github.com/nofishonfriday/Re...pooled.lua#L29
Actually my script could get very slow in some cases so this is helpful after all. Thanks!
MonkeyBars is offline   Reply With Quote
Old 02-18-2023, 04:36 AM   #24
bFooz
Human being with feelings
 
Join Date: Jul 2010
Location: Slovakia
Posts: 2,588
Default

Quote:
Originally Posted by Justin View Post
Defer() does not get called as fast as possible, it only gets called at about 30-50Hz. At this rate, even a hundred scripts executing a timestamp check each tick is not significant.
Could there be a possiblity to run defer faster when needed? 30fps stable.

Because of the smooth scroling script https://forum.cockos.com/showthread....54#post2647854
bFooz is online now   Reply With Quote
Old 02-18-2023, 04:45 AM   #25
cfillion
Human being with feelings
 
cfillion's Avatar
 
Join Date: May 2015
Location: Québec, Canada
Posts: 4,937
Default

https://forum.cockos.com/showthread.php?p=2546458 TL;DR Sort of, but not really. Low-priority OS timers just aren't good at high rates.
cfillion is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 08:41 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.