Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Bug Reports

Reply
 
Thread Tools Display Modes
Old 10-04-2018, 04:30 AM   #1
meta
Human being with feelings
 
Join Date: Sep 2018
Posts: 17
Default Deferred Python loop dies due to Reaper using a file for error output

[EDIT] Found the cause of the bug, retitled the thread from 'Deferred Python loop dies inexplicably when running in multiple instances of Reaper'.

Specs: Python 3.7, Windows 10 x64, REAPER 5.95

I've got a python script like this:

Code:
def loop():
    # do things
    RPR_defer('loop()')

RPR_defer('loop()')
When this script is run inside a single instance of Reaper there are no problems. However, if I open any other instances and start running it in them too, it eventually crashes at random in some or all of them. Each time, there's no trace-back. Only a vague 'Deferred script execution error'.



My guess is that it's got something to do with multiple instances trying to access the script file at the same time and encountering some kind of lock. But instead of waiting to retry, Reaper is just throwing its hands up in the air.

Last edited by meta; 10-22-2018 at 08:51 AM. Reason: Retitled thread, added link to solution post
meta is offline   Reply With Quote
Old 10-04-2018, 04:57 AM   #2
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 8,154
Default

I'm more familiar with Lua than Python, but looking at your code it seems unusual to me that you start the defer loop with another defer call. This would be the way I'd expect it to be (I've just let it run in two Reaper instances in parallel and no error so far):

Code:
def loop():
    # do things
    RPR_ShowConsoleMsg("defer")
    RPR_defer('loop()')

loop()
(Sorry if there's a reason for the way you did I'm not familiar with...)
nofish is online now   Reply With Quote
Old 10-04-2018, 06:08 AM   #3
meta
Human being with feelings
 
Join Date: Sep 2018
Posts: 17
Default

In that example my '# do things' comment was just to represent other code doing work between the start of loop() and the RPR_defer at the end. The actual script itself is a fair bit longer. It's a TCP server that polls sockets every cycle and processes their requests. Here's the code: reaper_server.py

Here's a simple client to test it with: reaper_client.py

Both of these were written and tested using Python 3.7 in Windows 10 x64.

Once you run reaper_server.py you should get this:


And after running reaper_client.py:


After that you're free to send whatever expressions you like and see the interaction between them printed out. E.g.:




If you run multiple instances of reaper_server.py in the same instance or other instances of Reaper it'll automatically start the server on the next available port it finds.

EDIT: This bug occurs even if there are no clients connected to any of the servers. It's enough to just open multiple Reaper instances, run reaper_server.py in each and wait a while and then see the crashes happen.

Last edited by meta; 10-04-2018 at 10:55 AM. Reason: Minor edits to reaper_server.py and reaper_client.py
meta is offline   Reply With Quote
Old 10-04-2018, 09:18 AM   #4
meta
Human being with feelings
 
Join Date: Sep 2018
Posts: 17
Default

Got some sleep and now realized I misread your post, sorry. I don't have any particular reason for starting the loop with an RPR_defer call other than not wanting the first cycle to block the main thread. Either way, just checked and the bug still occurs for if the loop is kicked off without deferring the first call.
meta is offline   Reply With Quote
Old 10-04-2018, 01:37 PM   #5
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 8,154
Default

No problem, with your explaination I now also understand why you start the loop with defer.
No other idea though, sorry.
nofish is online now   Reply With Quote
Old 10-20-2018, 06:16 PM   #6
meta
Human being with feelings
 
Join Date: Sep 2018
Posts: 17
Default

I've found the cause of this bug, and a workaround.

Short story:
Every time a Python ReaScript is run or partially invoked via RPR_defer, a file called 'reascripterr.txt' is created in AppData\Local\Temp (or its equivalent per the user's OS). The interpreter's stderr is redirected into this file, which Reaper then accesses afterwards to report any ReaScript errors. Once the execution of the script (or RPR_defer code) is complete, this file handle is closed, and the file is deleted.

All scripts use this same file which results in access collisions as two processes try to close/open handles to it at the same time. This also explains why there's no traceback output when the bug occurs.

Aside from causing crashes, this behavior also means a huge amount of unnecessary disk access. A simple deferred loop means that 'reascripterr.txt' is being created and deleted dozens or possibly hundreds of times per second. I don't need to explain why this isn't great. This design choice would not be acceptable for the Lua API, and it shouldn't be for the Python API either.

There are other, better ways to do this. One easy solution would be to redirect to a custom object with write()/flush() methods defined. Example:

Code:
class ReaperConsole:
    def write(self, output):
        RPR_ShowConsoleMsg(output)

    def flush(self):
        pass

reaper_console = ReaperConsole()

import sys

sys.stderr = reaper_console

# with the added benefit of being able to redirect stdout as well:
sys.stdout = reaper_console

# meaning python's built in print() function will now be usable:
print('This will be printed in ReaScript console output')
The Python C/C++ API also exposes functions specifically for retrieving information about exceptions, for example:

PyErr_Fetch - https://stackoverflow.com/a/43372878/10444096
PyException_GetTraceback - https://stackoverflow.com/a/25167989/10444096 (only available for python 3 and up i think? so not suitable)

There's also a lightweight C/C++ project called PyLogHook (https://github.com/TinyTinni/PyLogHook). It will handle all the heavy lifting and let you redirect text sent to sys.stdout/stderr from Python to a user defined C/C++ function. In my opinion this is the best option. All it takes is one call to set up.

Then there's also boost.Python - https://stackoverflow.com/a/30155747/10444096 , but will require all of boost to be included as well.

I also found this example of someone redirecting stderr to a StringIO buffer (note that StringIO is part of the io module in python3):
https://mail.python.org/pipermail/py...ry/501944.html

Long story:

I was doing a bit of poking around to find info about the embedded interpreter, using the following snippet:
Code:
import ctypes
import inspect
ctypes_members = inspect.getmembers(ctypes)

for key, value in globals().copy().items():
    if key == 'ctypes_members':
        continue

    if not key.startswith('RPR_'):
        if not key in ctypes_members:
            console_msg(key, '=', value)
In the output of this I saw the following:
Code:
_errfn = <_io.TextIOWrapper name='C://Users//admin//AppData//Local//Temp//reascripterr.txt' mode='w' encoding='UTF-8'>
Which led me to google _errfn and 'reascripterr.txt', leading to this: https://totalhash.cymru.com/analysis...84a6ea84cd5d17
(do a ctrl+f search for _errfn and reascripterr.txt)

From here, I decided to take a look at reaper.exe inside a hex editor and found this:
Code:
import sys.sys.dont_write_bytecode=True._errfn=open('...','w').sys.stderr=_errfn....sys.path[0]='...'...sys.path.append('...')..........................def RPR_defer(p):..f=CFUNCTYPE(None,c_uint64,c_char_p)(0x%08X%08X)..f(0x%08X%08X,rpr_packsc(p)).................................def RPR_atexit(p):..f=CFUNCTYPE(None,c_uint64,c_char_p)(0x%08X%08X)..f(0x%08X%08X,rpr_packsc(p))................................def __RPR__add_globlist(p,q):..f=CFUNCTYPE(None,c_uint64,c_char_p,c_char_p)(0x%08X%08X)..f(0x%08X%08X,rpr_packsc(p),rpr_packsc(q))..............................def RPR_runloop(p):..RPR_defer(p)...............................def __RPR_get_glob():.  f=globals().  for key in f:.    val=f[key].    if type(val) == float or type(val) == int or type(val) == str:.      __RPR__add_globlist(key,val).  return...............obj=compile(open('..').read(),'.','exec').exec(obj)._errfn.close()..Script execution error......_errfn=open('...obj=compile('...\r..\n..\t..\n\n....','defer','exec').exec(obj)....._errfn.close();.Deferred script execution error.....RTLD_...DEFAULT_MODE....__name__....__RPR__add_globlist("%s",%s)....__RPR_get_glob()....atexit.String...defer.String code...runloop.String code.....pythonlibpath64.pythonlibdll64..%s%clibs....libs....%s%clib.lib.%s%cApp.App.PATH....lib%s...python..libpython...python%d....python%d.%d.libpython%d.libpython%d.%d..%s%c%s..No Python dynamic library found for any compatible version.., %.300s %.400s, %.300s %.400s..using custom path...custom library.., %.300s %.400s.using custom library....%.400s %.200s...is installed....No compatible version of Python was found...reascripterr.txt....File "<string>", line ..%.200s %.800s...Can't open..rescript....Py_Initialize...Py_IsInitialized....Py_NewInterpreter...Py_EndInterpreter...PyRun_SimpleString..PyThreadState_Swap..PyGILState_Ensure...PyGILState_Release..Can't initialize Python.from reaper_python import *.rpr_initft({.  .,.  ....'%s':0x%08X%08X..}).....hasrecentsec....recent%02d......................
After a bit of cleaning up:
Code:
import sys

sys.dont_write_bytecode = True

_errfn = open('...','w')
sys.stderr = _errfn
sys.path[0]='...'
sys.path.append('...')

def RPR_defer(p):
	f=CFUNCTYPE(None,c_uint64,c_char_p)(0x%08X%08X)
	f(0x%08X%08X,rpr_packsc(p))

def RPR_atexit(p):
	f=CFUNCTYPE(None,c_uint64,c_char_p)(0x%08X%08X)
	f(0x%08X%08X,rpr_packsc(p))

def __RPR__add_globlist(p,q):
	f=CFUNCTYPE(None,c_uint64,c_char_p,c_char_p)(0x%08X%08X)
	f(0x%08X%08X,rpr_packsc(p),rpr_packsc(q))

def RPR_runloop(p):
	RPR_defer(p)

def __RPR_get_glob():
	f=globals()
	for key in f:
		val=f[key]
		if type(val) == float or type(val) == int or type(val) == str:
			__RPR__add_globlist(key,val)
	return

obj = compile(open('..').read(),'.','exec')
exec(obj)

_errfn.close()

_errfn=open('..')
obj=compile('...\r..\n..\t..\n\n....','defer','exec')
exec(obj)
_errfn.close()

# wasnt really sure how to clean up the rest but this was enough to go off of

.Deferred script execution error.....RTLD_...DEFAULT_MODE....__name__....__RPR__add_globlist("%s",%s)....__RPR_get_glob()....atexit.String...defer.String code...runloop.String code.....pythonlibpath64.pythonlibdll64..%s%clibs....libs....%s%clib.lib.%s%cApp.App.PATH....lib%s...python..libpython...python%d....python%d.%d.libpython%d.libpython%d.%d..%s%c%s..No Python dynamic library found for any compatible version.., %.300s %.400s, %.300s %.400s..using custom path...custom library.., %.300s %.400s.using custom library....%.400s %.200s...is installed....No compatible version of Python was found...reascripterr.txt....File "<string>", line ..%.200s %.800s...Can't open..rescript....Py_Initialize...Py_IsInitialized....Py_NewInterpreter...Py_EndInterpreter...PyRun_SimpleString..PyThreadState_Swap..PyGILState_Ensure...PyGILState_Release..Can't initialize Python.from reaper_python import *.rpr_initft({.  .,.  ....'%s':0x%08X%08X..}).....hasrecentsec....recent%02d......................
So to counteract sys.stderr = _errfn I changed stderr to a custom object that would route errors to RPR_ShowConsoleMsg:
Code:
import sys
class ReaperConsole:
	def write(self, output):
		RPR_ShowConsoleMsg(output)

	def flush():
		pass

reaper_console = ReaperConsole()

def loop():
	sys.stderr = reaper_console
	RPR_defer('loop()')

if __name__ == '__main__':
	RPR_defer('loop()')
And eventually was confronted with the following error:
Code:
Traceback (most recent call last):

  File "<string>", line 1, in <module>

PermissionError
: 
[Errno 13] Permission denied: 'C:\\Users\\admin\\AppData\\Local\\Temp\\reascripterr.txt'
So to completely prevent reascripterr.txt being created at all I used the workaround below to replace the open() function entirely.

Workaround:
Code:
class ReaperConsole:
    def write(self, output):
        RPR_ShowConsoleMsg(output)

    def flush(self):
        pass

    def close(self):
        pass

reaper_console = ReaperConsole()

original_open = open
patched_open = lambda *args: reaper_console

def loop():
	open = original_open
	# do work
	open = patched_open
	RPR_defer('loop()')

if __name__ == '__main__':
	RPR_defer('loop()')

Last edited by meta; 10-20-2018 at 07:03 PM.
meta is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 09:02 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.