ReaPack: Package Manager for REAPER (developer thread)

[mschnell]

I'll try to do a "description only" JSFX for a Reaper stock plugin and upload it to the community repository, just to see what happens...
-Michael

[mschnell]

Did so. Seems not to have worked. But no error message I could make sense of. "No description", even though there definitively is a description

[MonkeyBars]

Hi there!

I have a user claiming that my repo is looking for stuff in the wrong folders.

Here is my package path code:

package.path = package.path .. ";" .. string.match(({reaper.get_action_context()})[2], "(.-)([^\\/]-%.?([^%.\\/]*))$") .. "?.lua"

The main repo file lives here.

Do you have any idea why this could be happening?

[odedd]

I tried to find that info, but couldn't find anything about it so I'm asking here.

Here's a suggestion to help people lost in the sea of scripts find useful ones and sort thorough the abundance.

If there would be some kind of tracking of how many times a script had been installed, that could be useful information for both the developer, and the user, as in "huh, that seems like a popular script. I wonder what it does". Especially if that info would be shown in a column, and would be sortable (= most popular scripts at the top).

I realize that a lot of people just install everything, but still - not everybody does that, so the popular script will still probably stand out.

It could also be a more "algorithmic" popularity score - something that takes into account perhaps downloads over the last week/month? This could help finding scripts which are currently gaining momentum.

Same goes for update frequency - This can help sort outdated or abandoned scripts.

Again - I understand there are some inherent problems in some of these suggestions, but please see it more as a conversation starter than anything else.

What do you think?

[MonkeyBars]

Originally Posted by odedd

If there would be some kind of tracking of how many times a script had been installed, that could be useful information for both the developer, and the user, as in "huh, that seems like a popular script. I wonder what it does".

+1

[nofish]

Originally Posted by odedd

What do you think?

If it's implemented as additional columns, anyone who wouldn't like this feature could just ignore it (hide the columns, visibilty of columns is persisted across ReaPack launches), in fact I'm not sure if I'd use it myself, maybe to not not get biased or whatever, so having this would be fine what concerns me.

[Meo-Ada Mespotine]

Originally Posted by odedd

I tried to find that info, but couldn't find anything about it so I'm asking here.

Here's a suggestion to help people lost in the sea of scripts find useful ones and sort thorough the abundance.

If there would be some kind of tracking of how many times a script had been installed, that could be useful information for both the developer, and the user, as in "huh, that seems like a popular script. I wonder what it does". Especially if that info would be shown in a column, and would be sortable (= most popular scripts at the top).

I realize that a lot of people just install everything, but still - not everybody does that, so the popular script will still probably stand out.

It could also be a more "algorithmic" popularity score - something that takes into account perhaps downloads over the last week/month? This could help finding scripts which are currently gaining momentum.

Same goes for update frequency - This can help sort outdated or abandoned scripts.

Again - I understand there are some inherent problems in some of these suggestions, but please see it more as a conversation starter than anything else.

What do you think?

Hmm, I personally understand the idea but I think it's not as helpful as you might think.
The biggest issue is rather that everyone has a different workflow with different needs.
Means, it might help looking at the most popular scripts to get the script you are looking for. Or you are looking for something really odd and rarely installed, that does the job but would never appear on the top of a list and therefore: you still miss it.
It would also discourage new scripters adding new scripts, as people would have the tendency to look at the top but not the bottom of the list, so new scripts, who are still unpopular because of low install number, might never appear on top and therefore stay unnoticed.
It's the same thing, why encyclopedias had more words described in the first letters of the alphabet rather than the last ones: they were looked at first and got the most attention.

I would rather prefer better search functions, maybe a tag-system, that helps filter out, what you don't want vs what you might look for.
This could also be controlled by the scripters themselves, in the scripts and just added to the index-file for ReaPack to show it.
A download counter would be needed to be maintained by CFillion to have control over the counter(not mentioning legal requirements in many countries for tracking stuff).

The idea of an update-counter is actually a good one. This could give the scripter a way to see, whether a script might need maintenance after some years. Though it should be enough as @modified_date in the script or something.

[cfillion]

ReaPack doesn't send back any tracking data (I'm generally against such practices in software). Its package repositories are decentralized so technically anyone hosting one could setup a server-side download counter. However falsifying the data would be easy.

As Mespotine said, some rarely installed package can be the perfect fit for someone's specific needs. They shouldn't be buried just because they don't appeal to a wide user-base.

I prefer to have ReaPack be neutral with each package being equal and instead improve searching (eg. last update adds support for REAPER-like synonyms).

Originally Posted by Meo-Ada Mespotine

The idea of an update-counter is actually a good one. This could give the scripter a way to see, whether a script might need maintenance after some years. Though it should be enough as @modified_date in the script or something.

The package browser has an optional 'Last update' column (sortable). Not quite a counter or update score, but each package's history tab can give an idea of how often they're being updated (assuming the repository keeps old versions).

[odedd]

I totally understand, and I figured this (=non neutrality) would be a concern. It is definitely valid.

Some of the time I indeed search for a script to fulfill a certain need, but more often than not I go over the scripts in search of something that solves a problem I didn't realize I had. This is where I find the scripts really shine.

The way I currently do it is in this forum - I go to the scripting section and sort by views.
It does push new scripts to the bottom of the list, but sorting by date (the default way) solves that.
I also indeed use the recently updated column in the ReaPack browser to see if there's anything new I'm missing.

Again - while I definitely understand the reasoning behind NOT doing it, I just think it's a useful tool for discovery nevertheless, and as @nofish said, one could always hide those columns (they could be hidden by default). I'm also very much against tracking in general, but it could be done anonymously - no usage statistics. While it's true that those statistics could be falsified - I just don't think there are so many "evil" interests in the mix to actually worry about it - am I naive? Maybe...

Either way - as a repository grows it becomes increasingly harder to really find anything, and useful search tools become a necessity.

[mpl]

this:

Originally Posted by MonkeyBars

I haven't received updates via ReaPack for any mpl scripts for a while.

I tried reimporting the XML here and got the error below. I can't sync to your Github repo.

The received file is invalid: Premature end of data in tag html line 9
https://github.com/MichaelPilyavskiy/ReaScripts/blob/master/index.xml

I did a

reapack-index --rebuild

, but mostly it added a /fmodern tag inside indexer. I did inspect indexer, but not exactly understand synthax.

[cfillion]

Replied in that thread (blob instead of raw in the URL).

[mpl]

Originally Posted by cfillion

Replied in that thread (blob instead of raw in the URL).

Ah much thanks! Tried to inspect xml, but didn't point url isn't correct.

[pcp]

Hi everyone, I'm trying to set up ReaPack to share some stuff (using cfillion's GitHub template), and I can't find a way to include FX Chains. Is this possible or is there some workaround?

The Rfxchains in question are used by an associated script which inserts and removes two JSFX on Input FX with pre-baked modulation between them, so track templates won't work for this.

Edit: repo at https://github.com/s0u7h/pcp_scripts

[odedd]

Originally Posted by pcp

Hi everyone, I'm trying to set up ReaPack to share some stuff (using cfillion's GitHub template), and I can't find a way to include FX Chains. Is this possible or is there some workaround?

The Rfxchains in question are used by an associated script which inserts and removes two JSFX on Input FX with pre-baked modulation between them, so track templates won't work for this.

Edit: repo at https://github.com/s0u7h/pcp_scripts

I'm pretty sure you can use @provides for that within your jsfx. Just include a relative folder from the scripts folder to the fx chain folder.

[pcp]

Thanks @odedd, I'll try that, I thought from my desultory scanning of the ReaPack docs that you couldn't access arbitrary folders (unless they're provided by ReaPack, like [data] or [effects] - and there's no [fxchains] that I could see) but I'll see if it works trying a relative path. None of the scripts I've installed through ReaPack seem to have dropped anything in FX Chains though, which makes me cautious.

My other thought was to try hardcoding the relative path in the lua script (going up from 'FX Chains' to 'REAPER' then down to 'Scripts/pcp-scripts/fxchains), and putting my Rfxchain in a folder with the script itself. No idea if that'll work but will update here if so. In my case it'd work fine if it does as I don't need the user to see the fx chains.

[pcp]

Did not get to try with out-of-repo @provides, as hardcoding the path in the lua script worked, so I guess this will allow me to bundle everything together for ReaPack and use 'stealth' FX Chains.

reaper.TrackFX_AddByName(trk, "../Scripts/pcp_scripts/Let-It-Key-IFX.RfxChain", true, -1000)

[adu89]

Hello, what makes an extension a valid candidate for being part of the ReaTeam/Extensions ReaPack respository? I noticed that there aren't many listed there. I'm debating whether I should submit there or create my own repository. What are your thoughts?

[juan_r]

On a similar note, I've tried to add my repository (just a few JSFX as of now) to the master list, but it seems like my pull request for the edited master list didn't go through. Is there anything I'm missing?

[paat]

I'm interested in making my scripts available via ReaPack. I see there's a reapack-index tool for for making the index file. I have got it successfully building against a repo.

I'm curious specifically about the workflow when developing scripts that will end up on ReaPack. Here's what I do so far (no ReaPack involved yet).

1. I write a Lua script in the reaper resources/Scripts folder.
2. I open the actions list, and choose "New action->Load ReaScript"

Is that the approach for developing new scripts? I thought perhaps I could point ReaPack at my local repo so it would automatically pick up any scripts that I add... but it seems to assume that the repo is hosted on GitHub.

Anyway, just want to check how more experienced folks are doing this. Thanks.

[Jean Cremers]

Hello,
Before starting reapack i have to turn off my firewall (malware bytes firewall control) else i get errors about not able to connect.
I'd like to whitelist reapack but which file do i whitelist?

(i have reaper.exe whitelisted so it can check for updates and the like)

[juan_r]

More questions about Reapack. Since I made my own repository, I added a few more JSFXs and scripts. Not much yet, but they could be useful to someone as they have been to myself. So here are the questions

1. How do I get my repo added to the master list?

2. Every time I add or update a piece of code, github sends me a notice saying Run failed: check - master (55c8567)
What do I have to do to fix this? I can provide more info if needed, of course.

3. How do I add a multi-line description to my script so the "About this package" has some info in it? It might be good for users searching for specific keywords. I managed to do that with my JSFX: the info: tag does just that. For scripts, however (lua only for the time being), I am still not sure about the format for the headers. Any pointer to some docs would be welcome.

[MonkeyBars]

Originally Posted by juan_r

1. How do I get my repo added to the master list?

What master list?

2. Every time I add or update a piece of code, github sends me a notice saying Run failed: check - master (55c8567)
What do I have to do to fix this? I can provide more info if needed, of course.

3. How do I add a multi-line description to my script so the "About this package" has some info in it? It might be good for users searching for specific keywords. I managed to do that with my JSFX: the info: tag does just that. For scripts, however (lua only for the time being), I am still not sure about the format for the headers. Any pointer to some docs would be welcome.

Maybe my ReaPack dev basic steps post can help you

[nofish]

Originally Posted by juan_r

3. How do I add a multi-line description to my script so the "About this package" has some info in it? It might be good for users searching for specific keywords. I managed to do that with my JSFX: the info: tag does just that. For scripts, however (lua only for the time being), I am still not sure about the format for the headers. Any pointer to some docs would be welcome.

You can use the @about tag.
(Example of my ReaPack scripts:
https://github.com/nofishonfriday/Re...s%20pooled.lua)

Since it uses markdown format, line breaks can be done putting two spaces at the end of a line.

[juan_r]

Originally Posted by MonkeyBars

What master list?

https://reapack.com/repos

Originally Posted by MonkeyBars

Maybe my ReaPack dev basic steps post can help you

Hehe, that's how I got my own repo started some time ago, thanks for your writeup! I'm sure I'm not the only one who was helped, and I still recommend that resource to would-be ReaPackers.

Actually, my repo more or less seems to work, but almost every "workflow" (GitHub term) generates an error. Here's a typical one:

https://github.com/juanriccio/Reaper...uns/3069897370

The actual error bit seems to be the following:

Run git push
To https://github.com/juanriccio/Reaperism
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://github.com/juanriccio/Reaperism'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Error: Process completed with exit code 1.

Originally Posted by nofish

You can use the @about tag.
(Example of my ReaPack scripts:
https://github.com/nofishonfriday/Re...s%20pooled.lua)

Since it uses markdown format, line breaks can be done putting two spaces at the end of a line.

Thanks, @nofish! I'll do my reading.

[nofish]

Originally Posted by juan_r

The actual error bit seems to be the following:

Run git push
To https://github.com/juanriccio/Reaperism
 ! [rejected]        master -> master (fetch first)
error: failed to push some refs to 'https://github.com/juanriccio/Reaperism'
hint: Updates were rejected because the remote contains work that you do
hint: not have locally. This is usually caused by another repository pushing
hint: to the same ref. You may want to first integrate the remote changes
hint: (e.g., 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Error: Process completed with exit code 1.

It's actually (kinda) written in the error message what is wrong.
When you do a push to your remote (GitHub) repo the ReaPack indexer ('ReaTeam Bot') is auto-run and creates a new commit (e.g. here), which then you don't have in your local repo yet, so your local and your remote repo are diverged.
You first need to pull that commit from your remote repo to your local one, so it contains the Reapack indexer commit, then you can push to your remote with the new changes you've done locally.

[juan_r]

Uh, I see, thanks for the explanation. I don't have a local repository at all: just the source files, a bit scattered actually. Should I make a local repo, or is there a way to have it work withouht that?

[nofish]

Originally Posted by juan_r

I don't have a local repository at all

Oh, then my advice is maybe not suitable.
Sorry, I just assumed you'd have one (which is actually a bit stupid), as that's how I'm doing it.
I'm not familiar with ReaPack worklflow without having a local repo, so someone else would have to chime in here.

[mschnell]

@CFillion:
Any problems with the "mschnell_midi_transfer*.eel JSFXes ?

Thanks !
-Michael

[mschnell]

Originally Posted by mschnell

@CFillion:
Any problems with the "mschnell_midi_transfer*.eel JSFXes ?

These files were buggy, anyway

I did a new unified and improved plugin called mschnell_MIDI_Transfer.jsfx and will upload ASAP.

Lets see if I am able to delete the old ones...
Thanks again,
-Michael

[mschnell]

(How) Is it possible to work on and push my contributions to the ReaTeam repository with normal git means (command line tools or or with VSCode) instead of using the the Web ReaPack upload tool ?

E.g. : Do I need to clone the complete ReaTeam repository ?

Thanks,
-Michael

[cfillion]

Sure (it's a git repository like any other), but using the online package editor guarantees the metadata header is correct (+ standardized format = easier to review, the editor does many checks for common mistakes).

[mschnell]

The web editor does not seem to support modifying uploaded content which silly me did not transfer correctly
-Michael

[mschnell]

I understand that after pushing a file, your installation on GitHub automatically checks the metadata format.

Am, I just too silly to find how to use the web editor in "edit" mode loading a file, modifying it and then pushing it, or is this really not doable ?

Can I push to the ReaTeam when I want to edit a file with "normal (git...) means ? (BTW.: with VSCode you can preview the MarkDown doc.)

Thanks,

-Michael

[paat]

What’s the workflow for developing? Can you point your local REAPER at a local repo? Or do you have to load the Lua scripts individually?

If I load Lua scripts, I can’t reference other commands because they don’t have stable command IDs.

I’d love some info on the workflow for developing scripts that are intended to be published with ReaPack.

[nofish]

Originally Posted by paat

If I load Lua scripts, I can’t reference other commands because they don’t have stable command IDs.

Use NamedCommandLookup.
Background info:
https://forum.cockos.com/showthread.php?t=47729

(named IDs i.e. command ID strings are stable for scripts installed via ReaPack: https://forum.cockos.com/showpost.ph...50&postcount=4)

[paat]

Thanks for that. That's helpful, and addresses the issue with command IDs not being consistent.

I've had a chance to read every post in this thread now, to try to get a sense of how people develop and package ReaPacks.

Here's what it looks like so far:

1. Make a new script
2. Add it to REAPER with "New Action... -> Load ReaScript..."
3. Develop, get the nice auto reloading.
4. Commit, build index with reapack-index, ship.

Is that what everyone is doing?

---

It seems to me like it would be very useful to have a local developer mode. Instead of having to have two different mechanisms for loading scripts - manually loading them for dev, and adding the repo for packaging - we could add the local repo.

It seems pretty close with a file:/// based repo, but ReaPack effectively snapshots the repo by copying the files at an exact version.

Instead, it could either just read those files directly, or possibly symlink to the files on disk. That way changing files on disk would get live reloading.

Is this already possible? Or what do people think? I think it would make developing packages much smoother.

[paat]

I've started a proof of concept PR of this symlink approach, that I believe will make developing reapacks smoother.

[MonkeyBars]

Originally Posted by paat

I've started a proof of concept PR of this symlink approach, that I believe will make developing reapacks smoother.

Interesting! I don't really understand the problem this is meant to solve. Can someone explain simply?

[paat]

I described it in this post.

There's a disconnect between developing Reaper scripts, and packaging them. That's typical in software development, because there are so many operating systems and packaging systems, but Reaper only has ReaPack. For me, a strict separation between developing and packaging causes unnecessary friction.

I don't want to have my "development Reaper" where I manually add scripts ad hoc, and then my "production Reaper" where I install the package. I know they're going to be installed and organized by ReaPack, so I can use it from the get-go.

However, ReaPack copies the files. So any changes I make as I'm developing the script won't be reloaded in Reaper. I only have two options:

1. Add scripts ad hoc via "New Action->Load ReaScript", and then have a separate installation where I install the package.
2. Install the package, and re-install it every time I make a little script change.

This feature applies only to those developing a package. No end users will ever need (or even know about) it. It lets me iteratively develop a package, because I'm using it exactly the way an end user will when I ship a new version. Effectively, I am always using the bleeding edge of the pack locally.

---

I will turn the question around a bit to you: what is your current workflow for developing scripts, and packaging them? I asked that in the linked post a few days ago. I found a workflow that I assume is what people are using, but nobody has chimed in yet.

Another thing to consider: try it, and see if it benefits your workflow or not.

[cfillion]

For bigger scripts I'm using the first option: I like having separate WIP vs released versions of my scripts in the Action List. (Easy to compare & test for regressions.)
For simple scripts (that don't need between-release version control) I just directly edit the ReaPack-installed copy.
(Then in both cases drag/drop the updated files on https://reapack.com/upload to release.)

Originally Posted by paat

If I load Lua scripts, I can’t reference other commands because they don’t have stable command IDs.

local path = select(2, reaper.get_action_context()):match('^.+[\\//]')
dofile(path .. 'the_other_script.lua')