ReaPack: Package Manager for REAPER (developer thread)

[mpl]

( [master +3 ~0 -0 !]> )
(The git source https://github.com/cfillion/metaheader.git is not checked out. Please run bundle install before trying to start your application)

[cfillion]

That's weird, it looks like bundler cannot find the git executable for some reason. Does the `git --version` command works if you run it in the same window?

Also, can you post the output of ($env:Path).Replace(';',"`n") ? I'll compare with mine. This is Git Shell from the GitHub app right?
EDIT: The output of `which git` might be useful too.

[mpl]

Yes, it is Git Shell

D:\github\mpl\ReaScripts [master +3 ~0 -0 !]> git --version
git version 2.5.3.windows.1
D:\github\mpl\ReaScripts [master +3 ~0 -0 !]> ($env:Path).Replace(';',"`n")
C:\Users\Михаил\AppData\Local\GitHub\PortableGit_c 7e0cbde92ba565cb218a521411d0e854079a28c\cmd
C:\Users\Михаил\AppData\Local\GitHub\PortableGit_c 7e0cbde92ba565cb218a521411d0e854079a28c\usr\bin
C:\Users\Михаил\AppData\Local\GitHub\PortableGit_c 7e0cbde92ba565cb218a521411d0e854079a28c\usr\share\ git-tfs
C:\Users\Михаил\AppData\Local\Apps\2.0\COO27C3M.TH C\03H2T0OT.H74\gith..tion_317444273a93ac29_0003.00 00_f3a9f44c4e4f6514
C:\Users\Михаил\AppData\Local\GitHub\lfs-amd64_1.0.2
C:\Windows\Microsoft.NET\Framework\v4.0.30319
C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\compiler
C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\ia32\compiler
C:\ProgramData\Oracle\Java\javapath
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Windows\System32\WindowsPowerShell\v1.0\
D:\Program Files (x86)\Skype\Phone\
C:\Ruby22\bin

[cfillion]

I did more testing in my Windows virtual machine and I think it might be an issue with the username encoding.

Make a copy of C:\Users\Михаил\AppData\Local\GitHub\PortableGit_c 7e0cbde92ba565cb218a521411d0e854079a28c in C:\, and rename it to "git_copy" like in this screenshot:



Then in Git Shell run the following command:

$ENV:PATH = 'C:\git_copy\cmd;C:\git_copy\usr\bin;' + $ENV:PATH

Hopefully it will make bundle install work as expected.

[mpl]

(I ran it from virtual machine)
Seems I did something both for my and common repo.
Now if I wanna say to extension it should update my repo, I should run

bundle exec rake index

right?

[cfillion]

@mpv Yes, that's it.

------

Status Update on the extension:



TODO:

- Display the changelogs
- Polish small things
- Error reporting for failed downloads
- Support non self-contained scripts (eg. those with images resources)? Does anyone know any such script I can use for testing?
- Make the extension update itself?

[Sexan]

One question, this tool downloads all scripts from git?

[cfillion]

They are downloaded from GitHub using HTTPS. Technically the extension can fetch them from any web server specified in the index file by the repository maintainer, on a version-by-version basis.

[Sexan]

sorry I meant, will user choose which scripts to download or it will download all scripts?

[cfillion]

Oh sorry! Yes it installs every scripts from the configured repositories.

[mpl]

As you see, me and X-raym have almost same names for git repos. So I guess I should name my repo like "MPL ReaScripts". Can we do something to make it possible to decide with what name repo will be downloaded without renaming git repo actually?
Maybe via .ReaPackRemote ?

[cfillion]

The name of the destination directory is specified by the first line in the .ReaPackRemote file. It is totally independent from the git repo's name.

So for example, a .ReaPackRemote file for your own repository would be like this:

MPL ReaScripts
https://github.com/MichaelPilyavskiy/ReaScripts/raw/master/index.xml

[Sexan]

oh,in the future will there be some filter like in that mockup?its very cool tool, but what when number of scripts grow?or is this script for something else right now?

[mpl]

Awesome, thanks!
Another question, it will not talk with kb.ini, right?

[mpl]

Originally Posted by Sexan

oh,in the future will there be some filter like in that mockup?its very cool tool, but what when number of scripts grow?or is this script for something else right now?

Do you really think about quantity when installing SWS?

[X-Raym]

@cfilion
Oh man, that is taking shape !

Good job !

[cfillion]

@mpl No the user have to manually load the scripts he whish to actually use in the Action List.
Writing to reaper-kb.ini directly would require REAPER to be restarted for the changes to take effect...

[mpl]

OK, got it.
I guess I'll build script which add/update/clean all scripts from /Scripts to/from action list since for me it is better to find script via native action list search than by manual browsing.
Did we decided to use "_class" to say extension it is imported function and not master script?

[X-Raym]

@mpl
Or simply _functions, as class refers especially to Object Oriented approach
It may also be more obvious for curious people that will Browse the directories to understand that it is just functions, and does nothing especially.
These scripts may look like this

• spk77_Get audio stats from take_functions
• spk77_Buttons_functions

etc...

[X-Raym]

@cfillion
I just saw ony our last screenshots that common scripts are importerd under a master subfolder (and not mixed with other repos).

Is it name of the Repo or name of the GitHub user ?

I'm a bit confused since actually all my scripts are mixed together in my script folder (no matter what is their souece repo, I only kept folder name)

But I have to admit that having them separate from different repo seems more clever.

Unfortunately for me, Scripts commandID are based on path so I will have to remake all my scripts keyboard shortcut and toolbar...

@mpl
... unless you think it is be possible to have a script to batch re-import scripts (and actualize toolbars and custom actions accordingly, replacing the old script -with obsolete path and commandID- by the new one) ?

[mpl]

X-Raym, I can do that by parsing/checking kb.ini with existed "updated" structure, so all previously binded toolbars, custom actions, mouse modifiers will work after moving/renaming repo->applying my script->rebooting reaper (of course if you didn`t change scripts name actually = only path was changed)
OR you can try to do it manually by Search and replace inside kb.ini (which is dangerous though)

I firstly though all scripts will be downloaded into the same folder beetween repos, but now I think current behaviour is ok. Same folders name still should be the same to make user non-frustrated by "why same actions have different folder names?".

[X-Raym]

@mpl
Totally agree.

Maybe put in your script the "cleaning actions list" functin that you already made (the action that delete actions that are not linked to any link anymore.

About this script remapper... It will be a huuuuuge workflow improvements for all scripts lovers that have download and put scripts everywhere on their PC :P

[spk77]

Originally Posted by cfillion

Yes, that's it.


TODO:

- Display the changelogs
- Polish small things
- Error reporting for failed downloads
- Support non self-contained scripts (eg. those with images resources)? Does anyone know any such script I can use for testing?
- Make the extension update itself?

Awesome

Originally Posted by cfillion

- Support non self-contained scripts (eg. those with images resources)? Does anyone know any such script I can use for testing?

Here's Track IO mixer 14nov15: https://github.com/spk1977/ReaScript...ixer%2014nov15

• "Track IO Mixer 14nov15.lua" is the main script. Other files are "classes" and png files.

[timothys_monster]

Originally Posted by cfillion

Yes, that's it.

Wow!!!

[X-Raym]

@cfillion
Could the main Folder (when downloaded) be named like this ?
GitHubUserName - RepoName ?

so that my repo for exemple beame

X-Raym - REAPER-ReaScripts
and not just REAPER-ReaScripts.

Thanks for considering this if you do

[cfillion]

@X-Raym
You can set the target directory in the .ReaPackRemote file to anything you like, as long as it doesn't conflict with another .ReaPackRemote the user imported before yours.
I made an example for mpl's repository a few posts earlier: http://forum.cockos.com/showpost.php...&postcount=172.

Originally Posted by spk77

Here's Track IO mixer 14nov15: https://github.com/spk1977/ReaScript...ixer%2014nov15

• "Track IO Mixer 14nov15.lua" is the main script. Other files are "classes" and png files.

Thanks!

[mpl]

X-Raym, do you know some way to list files in directory including subfolders? I mean recursive running find-in-path or something like this.

For now I did this but this lists only 3 levels from parent folder.

[X-Raym]

@mpl
I didn have to do it yet but htis sounds possible...
I can investigate if you want.

[mpl]

Yes, you can, thanks.
I think it is all about logic and possible using native functions.

[mpl]

X-Raym, some progress

[X-Raym]

@mpl
You are closer than me :P
This recursive functions are a bit disturbing when you are not used to it.

Note: don't forget that the character separator is OS based,
so / may be \\
Here is a code snipper

function init()
    -- OS BASED SEPARATOR
    if reaper.GetOS() == "Win32" or reaper.GetOS() == "Win64" then
        separator = "\\"
    else
        separator = "/"
    end
end

end then file path should be define as

folder .. seprator .. name

[mpl]

X-Raym, I turned everything to "/" since it works everywhere (also kb.ini) both on Win and OSX.

Check this
Is that return right results? (Returning to kb.ini wrapped to the comment)

[X-Raym]

@mpl
This look promising, I will test it on both Mac and PC on Fresh REAPER install, I'll tell you how it goes in few hours. Good job !

[X-Raym]

@mpl
On windows:

• Scripts detections works
• Scripts moved (and new location works)
• But re assignation (new command ID) don't.

found line: SCR 4 0 RS55c12bd9a0a93607d4310481a3f358fb44cff00b "Custom: X-Raym_Display color of selected tracks items and takes in the console.eel" "X-Raym_Display color of selected tracks items and takes in the console.eel"
   Action list: "Custom: X-Raym_Display color of selected tracks items and takes in the console.eel"
   Address Line: X-Raym_Display color of selected tracks items and takes in the console.eel
   Address Full: E:/Bureau/Scripts/X-Raym_Display color of selected tracks items and takes in the console.eel
   Exists: No
   New Location: "E:/Bureau/Scripts/lol/X-Raym_Display color of selected tracks items and takes in the console.eel"
   Replaced
new line: SCR 4 0 RS55c12bd9a0a93607d4310481a3f358fb44cff00b "Custom: X-Raym_Display color of selected tracks items and takes in the console.eel" "E:/Bureau/Scripts/lol/X-Raym_Display color of selected tracks items and takes in the console.eel"

As you can see, it can detect that it has moved in the "lol" sub directory but it output the same commandID than before, so the script can't be launch from the action menu (because he has moved).

[cfillion]

I made a few changes to the indexer today to make it stop spamming the terminal with debug output. "Invalid metadata" warning messages can also be silenced now (useful when running the script for the first time on a repository ) using

bundle exec rake index -- --no-warnings

(that's X-Raym's repository being indexed for the first time – the full commit history is being read)

This update requires the new Rakefile from https://github.com/cfillion/reapack-repository-tools to work. I already pushed it on the common repository.

Also metadata keys with trailing spaces like "Version<Space>: 1.0" are now correctly parsed (oops).

(@mpl, the command to update the indexer and parser is `bundle update`. It does not fetch the new Rakefile though.)

----------

Originally Posted by X-Raym

@cfillion
Ok, If you propose a way to add Version in header more easily in bulk editing, I can do it.

Here you go: https://gist.github.com/cfillion/6d99012e7eb971fdf937
It splits, sorts and moves the changelogs to the first paragraph and it creates Version tags as appropriate from the existing data.

cd REAPER-ReaScripts
ruby path/to/converter.rb

It overwrites the files in place so just in case there's a bug or something I didn't notice, you can use git reset --hard to revert the changes.

Before: http://sprunge.us/gQJG
After: http://sprunge.us/SbLf


Oh and it marks any script with unknown version as "0.1". If you'd prefer it to use 1.0 or anything else instead just edit the first line.

[mpl]

X-Raym, I thought it can be same. Btw, it works for me:

(uncommented this and restarted REAPER):

PHP Code:

    -- return table to kb ini
    file = io.open(filename, "w+")
    content = file:write(table.concat(new_kb_table, '\n'))
    file:close() 


To-do: option to auto-add all scripts in /Scripts directory (which are not in action list already).
I can also add them with defined command id (and update IDs for existed scripts), for example by parsing headers:

* Command ID: _XRaym_something

[X-Raym]

@mpl
Scripts command ID has been discussed relatively recently with Jeff0s,
Better Scripts Management: Importation, Folders, ID, Deletion - Cockos Confederated Forums

This leads to the recent behavior : scripts command ID are generated from path.
But if the script is inside the user ressouce Scripts folder, then the command ID is the same across platforms.

Originally Posted by Jeff0s

I guess a better command ID could be like a crossplatform version of the script path itself: assuming the script stands in the REAPER/Scripts resource directory, this would make everything portable between users, out of the box (toolbars, etc) -- a hash could still be used otherwise.

Anyway, we hear you... Please, just don't hack command IDs for the time being!

So, in order to have compatibility with

• custom actions that include scripts
• scripts that include scripts
• scripts put on toolbar buttons
• people who manually import/imported scripts

... we should respect this script command ID principle.
In a optimum way, your script you be able to import script to REAPER (and so the command ID would be generate by Cockos) OR to determine how REAPER generate command ID.
Maybe Cockos can help you on this - I guess it is near impossible to determine how the hash is generated :P ?
(also it should be able to replace old command IDs in menu on custom action :P )



Your script is very promising and can defenitely really help managing scripts, but it ha to be done the Cockos way to assure a better compatibility with all the way scripts can be imported/integrated within REAPER.

@cfilion
Oh thanks, it looks indead very effective


Where does it place the "Version" in the header ? Under each fields ?



So if I undertsnad, the process is to

• install ruby
• run your version script

Is that all ? :P

[daxliniere]

Originally Posted by semiquaver

Stash is not versioned - there is no process to work collaboratively - its impossible to find anything.

Agree 100%.

Mikhail, though I'm not a coder, count me in on this.

[mpl]

I already put my hands down after no getting proper results for base64 decoding/encoding while working on modular view for fx chain. So I guess no one can help here also (except schwa and Justin)

[cfillion]

Originally Posted by X-Raym

Oh thanks, it looks indead very effective

Where does it place the "Version" in the header ? Under each fields ?

So if I undertsnad, the process is to

• install ruby
• run your version script

Is that all ? :P

Yes once ruby is installed just run the script from your repository's directory (`cd` to it first). That's so it won't try to search for lua/eel files everywhere.
You can refer to the video I made a few days ago for mpl, it shows the ruby installation in addition to how to run ReaPack's repository tools on windows: http://forum.cockos.com/showpost.php...&postcount=157.

As for where it places the Version header, this is the changes it makes (as seen by git diff):




The indexer only sees the last version/changelog tag in the header, that's why they're sorted from old to new.