jsfx : debugging messages / facility to write text on screen without using graphics c

[RichCaloggero]

First, as a new user around here, I want to thank the reaper devs for making an amazing piece of software and offering it to the world for a reasonable price.

As a blind user, reaper has made it possible for me to do things that I could simply never do before. One of those things is exploring the wonderful world of DSP development. Eel2 (i.e. JSFX) is easy to write and does pretty much everything you'd ever want. I've also been using the cookDSP library found in the atk for reaper (ambisonic toolkit for reaper) distribution. It has many excellent filter implementations etc, all clearly documented in the code.

Now for my question: as a blind user, I have no access to the jsfx development environment. I can get syntax error messages via my screen reader as they show up in the reaper fx window. However, I cannot write out debugging info for myself (i.e. there is no print function as in just about every other language). There is a way using the graphics facility to write text to the screen, but this is completely inaccessible to screen readers as it draws bitmaps. Would it thus be possible to implement a simple way to write text to the screen in a way that is detectable by a screen reader (i.e. the screen reader can detect sliders and their values). One way to do it would be to have a slider whose value you set to the value of a variable to examine it's contents, but this is kind of a lame way to do it.

Thanx for any help / info ...


-- Rich

[IXix]

That would be really useful. I wonder if Justin could crowbar ShowConsoleMsg() into JSFX? I know he's been poking around in the code recently.

[Justin]

I will look at adding something for this!

[IXix]

Originally Posted by Justin

I will look at adding something for this!

REAPER devs are the best devs.

[souk21]

Originally Posted by Justin

I will look at adding something for this!

Thanks! That would definitely be a wonderful addition

[Justin]

Coming in a build soon, there will be a writable string (#dbg_desc, probably) that will update the description in the main plug-in UI

[IXix]

Originally Posted by Justin

Coming in a build soon, there will be a writable string (#dbg_desc, probably) that will update the description in the main plug-in UI

Cool! Any chance of a message log window like ReaScript at some point? Would be very handy for quick traces and such.

[Meo-Ada Mespotine]

I was about to request something like this as well and been thinking about this for some time.
But less for "just" some debug-messages, but for a more generalized approach, so JSFX-coders could send filterable accessibility-messages too(or output debug-messages, like RichCaloggero wants to do).

Basically, what you need is an additional way for messages to be send, that is completely independent of any gfx-stuff.

You also need a way to separate useful and important messages from less important ones, to reduce the information sent.
UI's can show much more at the same time, than a description can say, so a way to filter out messages by some criteria
would be important.

So here's what I have in my mind, that might be an attempt to do it(and it has similarities with Justin's approach).

1) On FX-side:
I think, there should be an internal messages-log, that could be filled by JSFX/Video Processor/ReaScript/Cocko's VST-extension.

In JSFX for instance, you fill it by setting the variables #acc_description and acc_priority(more on priority later) and acc_message_changed.

This adds a message to an internal logger.
This logger logs:
- the description sent(acc_description)
- the priority(acc_priority between 0(low) and 10(high))
- the time of the log(like reaper.time_precise())
- guid of the jsfx that logs.
And it logs it everytime, when acc_message_changed=true(so the log wouldn't get spammed with 15623 versions of the same message, unless wanted to).


2) On API-side:
The API gets a function to access this internal log:

Something like this:

string message, integer timecode, integer priority = reaper.GetFXMessage(string fx_guid, integer message_log_index)

This can be used by extensions(Osara) or ReaScripts to get the sent messages, show them, filter them by priority, specific jsfx, message-content, time-range, etc.
And when they know, which message is wanted, they put it onto screenreaders, etc.


3) Filtering messages, using the information in the log:
Now, as I said, reducing and filtering the messages is important to prevent possible information-overkill.
With a log like that, this filtering can be done by Osara itself, with an interface designed for reducing messages
to the most important ones.

For this, we have multiple things we can filter for:

First, the fx itself. Second the message itself. Third, the message-priority. Fourth, the time-codes.


a) The FX itself:
This is simple. If you want to get only the messages of the currently opened JSFX and ignore the rest, you get
the guid of the fx and just get the messages of it.
That way, all JSFX can send messages, but I just see the one visible or even in focus.

Or I want to monitor messages of a certain JSFX I'm currently testing, how it works in combination with other FX,
no matter if the JSFX is visible or not.
The same way: get the JSFX's guid and get its messages only.

Means: reducing messages by specific JSFX.


b) Priority:
But what if I want to monitor multiple JSFX at the same time? They might constantly send messages, some important,
others not.

For instance: you have a JSFX, that monitors, if at certain frequency-ranges you have clippings.
It might want to send a message of "everything is fine and under xx dBa" constantly and once in a while
a "OMG, CLIPPING ALERT!"-message.

Now, getting the constant "everything is fine"-message written on your screenreader is tedious. You might want
to check this once in a while to get the current dBa-value but not constantly, so such a message is of low priority.

However, a "OMG, CLIPPING"-message is more important and you might want to know this instantly, so such a message gets
a higher priority.
So JSFX-coders, who want to send messages, can use the variable acc_priority to give the sent message a priority.
That way, the accessibility-user can set a threshold of messages, that they want to know instantly and those, they just get when needed.

The threshold being the priority and therefore importance.

Reduces the amount of messages and seperates between "give me instantly" and "I get them, when I want to".


c) The message itself:
This can be used to sort by keywords etc. So if all sorts of messages are sent via JSFX, but you want to have only a
handful of them in your message-list to check for, you can enter filter-keywords, that allow you ignoring messages by
content.

That also reduces the information in the most custom way.


d) The timecodes:
as the log logs the timecode as well, it would be possible to filter out by timecode too.
So Osara or a ReaScript monitors, when the user hits play and when the user hits stop afterwards. So getting
all messages inbetween play and stop, filtered by the criteria a-c, would allow even more reducing.

Now, if such a message-queue is available, extensions like Osara could provide an interface, that allows filtering such things in
the most accessible way. It also means, Osara can decide, what to output and what not.


I'm still thinking about, how to extend this message-queue-attempt to ReaScripts as well(script-instances have
no guid and therefore it's hard for Osara to get, which script is talking to a message-queue and which is the
currently focused one, when using gfx-windows).

But for JSFX, this would allow tons, already. Especially, as currently, there's no way to output any accessibility-messages
at all, so all JSFX relying completely on UI(monitoring JSFX that only "draw" the states" for instance) are pretty
useless for accessibility-users.

And all it needs is a log, that can be filled by fx with message, priority, logtime, fx-guid and accessed via API.

(Sorry for the long novel-like post )

[Meo-Ada Mespotine]

In addition to my post above, the following code-snippet would output the last message, a focused-trackfx-JSFX would have sent to the message-log.
It would output, filtered only by jsfx focused, not by priority or keywords or time-code in this example.
But that would be easy to add.

It would output it to ReaScript's console-window:

function main()
  -- get focused fx
  retval, tracknumber, itemnumber, fxnumber = reaper.GetFocusedFX2()
  if retval>0 then
  -- get track
    if tracknumber==0 then 
      track=reaper.GetMasterTrack(0)
    elseif tracknumber>0 then
      track=reaper.GetTrack(0, tracknumber-1)
    end
    if track~=nil then
      -- get fx-guid
      fx_guid=reaper.TrackFX_GetFXGUID(track, fxnumber)
      
      -- get fx-message(assuming, 1 is the most recent message)
      message, timecode, priority = reaper.GetFXMessage(fx_guid, 1)
      
      -- show message
      reaper.ClearConsole()
      reaper.ShowConsoleMsg("JSFX sent the following message: "..message)
    end
  end
  reaper.defer(main)
end


main()

And all the JSFX must do, is set

#acc_description="new message"; // the message
acc_priority=10; // Priority(between 0 and 10); set to the highest priority
acc_message_changed=true; // true, if the message shall be seen as updated message and added to log
                          // false, to not add a message to the log at the moment

(Mercy, when I didn't perfectly write the EEL-code...but you probably get the idea.)

[RichCaloggero]

All this sounds great, but two issues I see with this approach:

1. I think there should be a way to create UIs irrespective of gfx - i.e. gfx is completely inaccessible, so if there could be a way to write string content somewhere in the UI (similar to the way sliders work) then there would be a way to build completely accessible JSfx plugins without the need for gfx. Of course, sighted folks would probably still want to use it to draw pictures, graphs, etc, but my point being there would be a way to create all the function UI without it. Sliders get you most of the way there, but without theability to handle string content efficiently (sure you can use options {on, off} etc), but arbitrary string UI (I/O) would be much more useful.

2. instead of needing to manipulate those variables in JSfx, why not just one function call to specify message text, priority, etc.

I like the filtering idea!

Again, I don't think having a solution only available to screen readers (osara) via API is the right approach.

THe latest mesage posted by Justin seems to be at least a start - writing a string to the FX panel where syntax errors and CPU percentage are currently written (at least I think this is what he is proposing). That would be helpful, as my screen reader can read this area (not very conveniently, but it can), and I presume it is also available visually.

Thanx for all the responses here. Reaper, and the people behind it are amazing!!

-- Rich

Originally Posted by Meo-Ada Mespotine

I was about to request something like this as well and been thinking about this for some time.
But less for "just" some debug-messages, but for a more generalized approach, so JSFX-coders could send filterable accessibility-messages too(or output debug-messages, like RichCaloggero wants to do).

Basically, what you need is an additional way for messages to be send, that is completely independent of any gfx-stuff.

You also need a way to separate useful and important messages from less important ones, to reduce the information sent.
UI's can show much more at the same time, than a description can say, so a way to filter out messages by some criteria
would be important.

So here's what I have in my mind, that might be an attempt to do it(and it has similarities with Justin's approach).

1) On FX-side:
I think, there should be an internal messages-log, that could be filled by JSFX/Video Processor/ReaScript/Cocko's VST-extension.

In JSFX for instance, you fill it by setting the variables #acc_description and acc_priority(more on priority later) and acc_message_changed.

This adds a message to an internal logger.
This logger logs:
- the description sent(acc_description)
- the priority(acc_priority between 0(low) and 10(high))
- the time of the log(like reaper.time_precise())
- guid of the jsfx that logs.
And it logs it everytime, when acc_message_changed=true(so the log wouldn't get spammed with 15623 versions of the same message, unless wanted to).


2) On API-side:
The API gets a function to access this internal log:

Something like this:

string message, integer timecode, integer priority = reaper.GetFXMessage(string fx_guid, integer message_log_index)

This can be used by extensions(Osara) or ReaScripts to get the sent messages, show them, filter them by priority, specific jsfx, message-content, time-range, etc.
And when they know, which message is wanted, they put it onto screenreaders, etc.


3) Filtering messages, using the information in the log:
Now, as I said, reducing and filtering the messages is important to prevent possible information-overkill.
With a log like that, this filtering can be done by Osara itself, with an interface designed for reducing messages
to the most important ones.

For this, we have multiple things we can filter for:

First, the fx itself. Second the message itself. Third, the message-priority. Fourth, the time-codes.


a) The FX itself:
This is simple. If you want to get only the messages of the currently opened JSFX and ignore the rest, you get
the guid of the fx and just get the messages of it.
That way, all JSFX can send messages, but I just see the one visible or even in focus.

Or I want to monitor messages of a certain JSFX I'm currently testing, how it works in combination with other FX,
no matter if the JSFX is visible or not.
The same way: get the JSFX's guid and get its messages only.

Means: reducing messages by specific JSFX.


b) Priority:
But what if I want to monitor multiple JSFX at the same time? They might constantly send messages, some important,
others not.

For instance: you have a JSFX, that monitors, if at certain frequency-ranges you have clippings.
It might want to send a message of "everything is fine and under xx dBa" constantly and once in a while
a "OMG, CLIPPING ALERT!"-message.

Now, getting the constant "everything is fine"-message written on your screenreader is tedious. You might want
to check this once in a while to get the current dBa-value but not constantly, so such a message is of low priority.

However, a "OMG, CLIPPING"-message is more important and you might want to know this instantly, so such a message gets
a higher priority.
So JSFX-coders, who want to send messages, can use the variable acc_priority to give the sent message a priority.
That way, the accessibility-user can set a threshold of messages, that they want to know instantly and those, they just get when needed.

The threshold being the priority and therefore importance.

Reduces the amount of messages and seperates between "give me instantly" and "I get them, when I want to".


c) The message itself:
This can be used to sort by keywords etc. So if all sorts of messages are sent via JSFX, but you want to have only a
handful of them in your message-list to check for, you can enter filter-keywords, that allow you ignoring messages by
content.

That also reduces the information in the most custom way.


d) The timecodes:
as the log logs the timecode as well, it would be possible to filter out by timecode too.
So Osara or a ReaScript monitors, when the user hits play and when the user hits stop afterwards. So getting
all messages inbetween play and stop, filtered by the criteria a-c, would allow even more reducing.

Now, if such a message-queue is available, extensions like Osara could provide an interface, that allows filtering such things in
the most accessible way. It also means, Osara can decide, what to output and what not.


I'm still thinking about, how to extend this message-queue-attempt to ReaScripts as well(script-instances have
no guid and therefore it's hard for Osara to get, which script is talking to a message-queue and which is the
currently focused one, when using gfx-windows).

But for JSFX, this would allow tons, already. Especially, as currently, there's no way to output any accessibility-messages
at all, so all JSFX relying completely on UI(monitoring JSFX that only "draw" the states" for instance) are pretty
useless for accessibility-users.

And all it needs is a log, that can be filled by fx with message, priority, logtime, fx-guid and accessed via API.

(Sorry for the long novel-like post )