ReaPack: Package Manager for REAPER (developer thread)

[nofish]

Originally Posted by Infrabass

- Is this template still good to use

Yes. At least I'm still using it without issues.

and how exactly work the "automated reapack-index running from GitHub Actions"? Is it a way to avoid having to manually run the reapack-index software to update the xml file?

Exactly. If you push to your repo created from this template reapack-index is automatically run, tells you if everything worked ok and then automacically makes a new commit updating the index.xml file.

[Infrabass]

Thanks for the answers nofish!

Do I still need to follow this page if I'm using the reapack-index template?
https://github.com/cfillion/reapack-index/wiki

In other words, do I need to install Ruby, cmake, etc?

[cfillion]

Those instructions are for running reapack-index locally. Not needed if you use the GitHub Actions template since it will run remotely on GitHub's servers instead.

Originally Posted by Infrabass

- Do I need to create a repo for reascripts and another for jsfx?

Repositories can contain any kind of packages. Mixing scripts and JSFX ones is perfectly fine.

Originally Posted by Infrabass

- In this article from X-Raym, PHP file is mentioned as a way to add security and control over what is shared online. Could you please elaborate a bit on this?

That's a server-side script to dynamically generate a index.xml for each user giving access to purchased packages.

[Infrabass]

Thanks for the help cfillion!

It is very likely that I have forgotten something but I have this error each time I commit something.
Any idea?

[cfillion]

Click on the "reapack-index" job to open the log where the error messages are printed.

[Infrabass]

It looks like a permission issue.

[cfillion]

Set "Workflow permissions" to "Read and write" under Settings > Actions.

[Infrabass]

Thanks Cfillion, fixed but I have a basic issue because I'm not familiar with Github using the webpage.
I don't know how to pull from the web…

I used GitHub Desktop at the same time, is there a problem with that?
Can I use GitHub Desktop or the GitHub Actions are only triggered via the webpage?

[Infrabass]

hmm it's all good now.
Not sure what I was doing wrong.
Repository added via ReaPack yeaaaah

[cfillion]

That error is normal and safe to ignore. When pushing commits quicker than reapack-index can finish, any running job before the last are cancelled with that message. Only the indexing job for the latest commit need to succeed.

[Infrabass]

Ah ok, thanks for the explanation!

[mattchill]

Hello,

Is there any up to date documentation on creating a ReaPackArchvie file? I have a private repo I would like to use with Reapack to distribute scripts amongst a private group, but so far am met with an error that Reapack can't read the table of contents. I made a manifest.json for this and zipped with my script/changed file type.

I've tried importing my private repo through Reapack via it's index.html and it says it can't connect to the repo. I've also tried running a local git server to point to the index.html but Reapack says it cannot connect to server.

How would you suggest is the best way to move forward distributing scripts through reapack with a private repo?

[cfillion]

A ReaPackArchive by itself still needs an underlying repository with a fetchable XML index.

You'd have to setup a private HTTP server. It needs to serve index.xml as well as the package files for each current and past versions.

To configure the file download URLs generated by reapack-index, add this to a .reapack-index.conf at the root of the git repository:

--url-template "http://your-private-server.local/$commit/$version/$package/$path"

(this URL template can be anything, above is only an example showing all supported $variables)

Easiest to setup HTTP server would be GitWeb:

https://git-scm.com/book/en/v2/Git-on-the-Server-GitWeb

git instaweb --httpd=webrick

--url-template "http://localhost:1234/?p=.git;a=blob_plain;f=$path;hb=$commit

[Infrabass]

Greetings everyone,

Is there a way to avoid loading in the user action list a lua script provided via a metapackage?

[cfillion]

It's not possible to install *some* of the files from a package but not others. It's either the whole package or nothing. (Partial installs would result in some broken scripts where needed files are missing.)

Consider asking the author of the package if you think parts of it would be useful installed separately from the rest. (Or if it's your package, make multiple packages. )

If splitting is not practical due to the nature of the script and want to reduce the amount of actions in your Action List, you can always temporarily remove them. They will come back after updating or reinstalling the package.

[Infrabass]

Yes it's for my own package: advanced items repositioning:
https://forum.cockos.com/showthread....04#post2681604

I added a feature where clicking a button automatically generates a script without GUI from the current values.
This script is setting extstate values and run the script that I would like to hide from the user because it doesn't make sense to run the no_GUI script by itself.

As they are definitively part of the same reascript, I don't think having multiple packages is convenient for the user.

Example:

-- This script was generated by Script: VF - ITEM - Advanced items repositioning.lua
reaper.SetExtState("vf_reposition_items", "interval_sec_noGUI", 3.0, true)
reaper.SetExtState("vf_reposition_items", "interval_frame_noGUI", 9, true)
reaper.SetExtState("vf_reposition_items", "interval_beats_noGUI", 1, true)
reaper.SetExtState("vf_reposition_items", "interval_mode_noGUI", 0, true)
reaper.SetExtState("vf_reposition_items", "offset_state_noGUI", tostring(false), true)
reaper.SetExtState("vf_reposition_items", "offset_val_noGUI", 1.0, true)
reaper.SetExtState("vf_reposition_items", "toggle_val_noGUI", 1, true)
reaper.SetExtState("vf_reposition_items", "mode_val_noGUI", 0, true)
reaper.SetExtState("vf_reposition_items", "overlap_noGUI", tostring(true), true)
reaper.SetExtState("vf_reposition_items", "adjacent_noGUI", tostring(false), true)
reaper.SetExtState("vf_reposition_items", "disable_autoxfade_noGUI", tostring(true), true)
local script_path = debug.getinfo(1,'S').source:match[[^@?(.*[\/])[^\/]-$]]
dofile(script_path .. "VF - ITEM - Advanced items repositioning - last values without GUI.lua")

[cfillion]

Oh! In that case don't mark the no-GUI file with [main] to have it installed but not added to the action list. It's the default behavior:

@provides
  [main] this_will_be_added_to_the_action_list.lua
  this_one_will_not.lua
  [nomain] this_one_also_will_not.lua

[Infrabass]

Originally Posted by cfillion

Oh! In that case don't mark the no-GUI file with [main] to have it installed but not added to the action list. It's the default behavior:

@provides
  [main] this_will_be_added_to_the_action_list.lua
  this_one_will_not.lua
  [nomain] this_one_also_will_not.lua

This is exactly what I was looking for, awesome!

Last question about metapackage.
Do we need to update the version number and changelog in every script inside the package?

[cfillion]

The included files should not contain any metadata beyond "@noindex".

[Infrabass]

Ah ok, thank you!
I was confused by your Monitoring FX package that I used as an example where all the scripts have the full metadata in their header but I guess it's because you just copy/paste your code.

Last but not least, what would happen if I move all the files (including the main script) into a subfolder, now that this is a package? Is it really a bad idea knowing that people have already downloaded it?

[cfillion]

Moving the main script means deleting+creating a new package. The old one will become "obsolete" for users that already installed it.

My Monitoring FX package is made of a single source file in the repository that ReaPack installs multiple time under different names: https://github.com/ReaTeam/ReaScript...s%20bundle.lua.

[Infrabass]

Ok so I won't do that then, lesson learned :-)

My Monitoring FX package is made of a single source file in the repository that ReaPack installs multiple time under different names: https://github.com/ReaTeam/ReaScript...s%20bundle.lua.

Wow, don't even know this was possible!

[cfillion]

"Obsolete" means that the old copy will remain installed by default and users will be asked whether to uninstall it when they synchronize. (This is so existing workflows/shortcuts/toolbars/etc don't break without warning.) It's not a bad thing per se.

[Infrabass]

Ok, it's cool ReaPack ask for deletion, I was afraid users will end having a duplicate of the script.
Still, I don't want to break anything so I'll keep things like that.
Thanks Cfillion!

[odedd]

Let's say I have two scripts:

/<Category>/My Script/My Script.lua
/<Category>/My Other Script/My Other Script.lua

And some folders where I keep shared code:

/Resources/Common
/Resources/Common/Helpers
/Resources/Fonts
/Resources/Icons

In order to use the resources my scripts use @provides like that:

--@provides
--   [nomain] ../../Resources/Common/* > Resources/Common/
--   [nomain] ../../Resources/Common/Helpers/* > Resources/Common/Helpers/
--   [nomain] ../../Resources/Fonts/* > Resources/Fonts/
--   [nomain] ../../Resources/Icons/* > Resources/Icons/

I know I could use a ** wildcard on the "left side" to get all subfolders, but it does not recreate the folders recursively on the "right side" - I have to write them in one by one.

Is there a way to maintain the folder structure instead of manually doing each folder?

Something like

--   [nomain] ../../Resources/** > Resources/**

Thanks!

[Hypex]

I've thought of a question. This would apply to installing items by hand as well as using ReaPack. And I'm not sure what would work either way.


Is there a way to easily install a tool bar? I've looked in a menu set file and inside it states a tool bar to use. This makes it hard to install on another system because a specific named tool bar could clobber an existing tool bar.


So is there a way to dynamically install a tool bar in the next free slot or modify menu set file so the tool bar it uses is selected by Reaper? Be useful if it was a easy as changing toolbar specified in file to a custom name.

[jbradbury]

Originally Posted by cfillion

The first step is to decide how to split (or not to split) the scripts into separate user-installable packages. (Is ReaCoMa best described as a set of tools/actions updated individually, or "one thing" consisting of multiple actions? Would it make sense to let users choose to install only a subset or have mismatched versions?)
(Generally, ReaPack repositories are meant to hold many packages, not just one big one. Have you considered releasing on ReaTeam?)

Then those user-installable packages need to be moved in a subfolder, which serves as the category name in ReaPack.

To include additional files beyond the package's file itself:

@provides
  [main]   user_action.lua
  [nomain] library_file.lua

If the library files are shared between multiple packages, each one using them must provide a different copy:

-- @description Script A
-- @version 1.0
-- @provides shared_library_file.lua > Script A/library_file.lua

dofile(self_path .. '/Script A/library_file.lua')

Installing 'Script A' will pull the correct version of 'shared_library_file.lua' from the time of its 1.0 release. Independently from the version of the shared library that a 'Script B' v1.4.8 might need.

In addition to the documentation, and even if building your own repository, I'd recommend referring to the GUI ReaTeam uploader (https://reapack.com/upload) as an interactive example of valid metadata.

(Also this is the user thread, the developer thread for questions like this is here: https://forum.cockos.com/showthread.php?t=169127)

This is a follow on conversation with cfillion which being quite silly I posted in the package rather than developer forum.


ReaCoMa is best described as a suite of tools that should live together. There are 11 scripts that each perform a different kind of audio de-mixing / source separation or alternatively perform segmentation / slicing. I'd expect them to be installed / removed together because and because of my own architecture (and that of FluCoMa) it makes sense to version and change them as a group, rather than individually.

These are the only scripts a user would ever run or execute from the action menu. These scripts share a body of lua code inside a lib/ folder. The structure of these files looks like this:

.
├── fluid-ampgate.lua
├── fluid-ampslice.lua
├── fluid-audiotransport.lua
├── fluid-hpss.lua
├── fluid-nmf.lua
├── fluid-nmfcross.lua
├── fluid-noveltyslice.lua
├── fluid-onsetslice.lua
├── fluid-sines.lua
├── fluid-transients.lua
├── fluid-transientslice.lua
└── lib
    ├── algorithms
    │   ├── ampgate.lua
    │   ├── ampslice.lua
    │   ├── audiotransport.lua
    │   ├── hpss.lua
    │   ├── nmf.lua
    │   ├── nmfcross.lua
    │   ├── noveltyslice.lua
    │   ├── onsetslice.lua
    │   ├── sines.lua
    │   ├── transients.lua
    │   └── transientslice.lua
    ├── colors.lua
    ├── configuration.lua
    ├── container.lua
    ├── imgui
    │   ├── helpers.lua
    │   ├── widgets.lua
    │   └── wrapper.lua
    ├── layers.lua
    ├── ordered_tables.lua
    ├── params.lua
    ├── paths.lua
    ├── reacoma.lua
    ├── slicing.lua
    └── utils.lua

Anything inside lib should _not_ be exposed to users, but needs to exist once installed for the top-level scripts to work. My ideal outcome would actually be a single item in ReaPack can be installed which gives users all the top-level scripts plus the lib folder.

w/r/t/ what I've tried so far here is the repo and branch I'm trying to work through – it has the index.xml and the header information in the files. Any pointers would be great.

[cfillion]

Originally Posted by jbradbury

I'd expect them to be installed / removed together because and because of my own architecture (and that of FluCoMa) it makes sense to version and change them as a group, rather than individually.

My ideal outcome would actually be a single item in ReaPack can be installed which gives users all the top-level scripts plus the lib folder.

In that case, there is only one file that needs to be added to the repository: Category Name/Package Name.lua containing the packaging metadata describing what to install.

For example, "Fluid Corpus/ReaCoMa.lua" with contents like this:

@description ReaCoMa 2
@version 2.9.0
@author jbradbury
@changelog
  use new table unpacking for args
  update layers functions to accept table of item tables
  utility function for deselecting all items
@metapackage
@provides
  [main] ../fluid-*.lua
  ../lib/**/*.lua
@about
  ReaCoMa 2 is a project developed by James Bradbury. It enables you to use the FluCoMa tools directly within REAPER.

  Lorem ipsum dolor sit amet...

The @metapackage tag removes the current file from the install list (since it only contains metadata). In the @provides tag, [main] registers the files into the Action List while the default [nomain] does not. **/*.lua is recursive globbing (globstar).

All files listed in the @provides tag are taken from the time at which the @version bump is committed.

reapack-index will complain about the .lua files under 'lib' not containing packaging metadata. It's safe to ignore since you don't want to make those files be packages. The warnings can be silenced via the command line or adding the following in a .reapack-index.conf file at the root of the repository:

--ignore lib

[jbradbury]

This enormously helpful! I'm going to try this now and see where I get stuck.

Thanks

[jbradbury]

Originally Posted by cfillion

@description ReaCoMa 2
@version 2.9.0
@author jbradbury
@changelog
  use new table unpacking for args
  update layers functions to accept table of item tables
  utility function for deselecting all items
@metapackage
@provides
  [main] ../fluid-*.lua
  ../lib/**/*.lua
@about
  ReaCoMa 2 is a project developed by James Bradbury. It enables you to use the FluCoMa tools directly within REAPER.

  Lorem ipsum dolor sit amet...

I did something similar to what you have, but I put the metapackage script into the scripts folder. Maybe this is wrong. You can see how I've structured things here:

https://github.com/ReaCoMa/ReaCoMa-2...eapack-support

When I add the repo to reapack I get the option to install each of the fluid*.lua scripts separately (can this just be one item to install?) but also none of the dependencies seem to get pulled through with it:

[wastee]

Hi cfillion,

Have you considered adding retry settings in ReaPack? There are certain areas where the internet connection is not stable, resulting in ReaPack displaying the following failed message: "Timeout was reached (28): Operation too slow. Less than 1 byte/sec transferred in the last 15 seconds."

When users attempt to re-download packages, the download process starts from the beginning, even if some files were downloaded before the last failure. In areas with an unstable internet connection, the chances of encountering the failed message again are high. This makes it difficult for users to download packages.

It would be great if ReaPack had a retry settings option that retains the files already downloaded. This would greatly improve the user experience!

[cfillion]

Originally Posted by jbradbury

When I add the repo to reapack I get the option to install each of the fluid*.lua scripts separately (can this just be one item to install?) but also none of the dependencies seem to get pulled through with it:

reapack-index does not remove previously indexed packages if they lose their metadata in the future.

They also prevent the new metapackage from getting indexed:

warning: scripts/ReaCoMa.lua:
  'scripts/fluid-ampgate.lua' conflicts with 'scripts/fluid-ampgate.lua'

Aka "While adding the new ReaCoMa package, I found that there's already a package in the index that provides the same scripts/fluid-ampgate.lua file for an overlapping set of OSes. Some users won't be able to install that, so skipping indexing this commit of ReaCoMa.lua.".

There are few possible solutions:

• Remove the bad commits from the git history + reapack-index --rebuild
• Add @noindex tag to those files instead of no metadata at all
• Replace index.xml to be just the following then rerun reapack-index:
  

  <?xml version="1.0" encoding="utf-8"?>
  <index version="1" commit="9ced6780c2e0866924045133dd8f35a4dcf93c28" name="ReaCoMa">
  </index>

• Teaching reapack-index to remove previously indexed files once they lose their metadata (tricky because it can't distinguish between temporarily vs permanently + invalid vs missing metadata)

Originally Posted by wastee

It would be great if ReaPack had a retry settings option that retains the files already downloaded. This would greatly improve the user experience!

Can be a good idea, needs some safeguard to not forever litter the user's drive with partially downloaded files though.

[raquelgc]

Hi there! I made my first Reascript pull requests two weeks ago (as ilovemedia), and I'm curious about how long they usually take to be reviewed. Thanks!

[Dimson]

Can you please tell me, is there any way to restore dependencies of scripts installed in Reaper with ReaPack repository? The scripts were installed earlier through Reapack, but now ReaPack does not show them in the list of installed scripts.

[lalo]

Seems that
https://github.com/luckyxxl/reaper-s...ster/index.xml
is no longer accesible..should we remove it from the list?

[MonkeyBars]

Originally Posted by lalo

Seems that
https://github.com/luckyxxl/reaper-s...ster/index.xml
is no longer accesible..should we remove it from the list?

+1

[cfillion]

Done. (See also the "Edit this list" button near the top of the page.)

[inthevoid]

Hi all. New to Github and having trouble getting reapack-index installed. Getting this error in the terminal:

ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /Library/Ruby/Gems/2.6.0 directory

Running

ruby -v

gives me the result

ruby 2.6.10p210 (2022-04-12 revision 67958) [universal.arm64e-darwin22]

Has anyone else come across this? Thanks!

Was thinking of using sudo before the command but read a lot of comments saying not to do that...

[cfillion]

Originally Posted by inthevoid

ERROR:  While executing gem ... (Gem::FilePermissionError)
    You don't have write permissions for the /Library/Ruby/Gems/2.6.0 directory

That's the macOS-provided version of Ruby rather than Homebrew's.

The easiest way to use reapack-index is automatically, either via GitHub Actions or another similar CI service. See https://github.com/cfillion/reapack-repository-template.

[inthevoid]

Originally Posted by cfillion

That's the macOS-provided version of Ruby rather than Homebrew's.

The easiest way to use reapack-index is automatically, either via GitHub Actions or another similar CI service. See https://github.com/cfillion/reapack-repository-template.

Ah! Perfect. I completely missed this. Thank you!