Old 11-12-2009, 05:22 AM   #1
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: City
Posts: 9,561
Default Empty Item cues from a CSV spreadsheet

This is a project I'm starting right now, so I thought I'd document it here and maybe throw some ideas around with you ladies and gents.

In foley, ADR and dubbing of foreign language films a take list is generated beforehand. The is spotted for each of these tasks. The result is a spreadsheet of timecodes and descriptions, like so :
Code:
Character  Actor        Track  IN           OUT          Line
John D.    Johnny Depp  01     04:02:16:20  04:02:20:07  Comon Sam.
John D.    Johnny Depp  02     04:02:20:08  04:02:26:00  You and me is going places.
In Perl it's pretty easy to deal with tab-delimited CSV files.

The goal now is to create empty items for all the takes, with the item notes holding the character name and the lines. In the case of foley, it'll be character specifics and loads of other stuff. You might have more things to do there and the script might need to take that in to account.

How to do it

There are actions to set the edit cursor position, set the time selection start and set the time selection end. There are actions to select specific tracks. In particular the SWS action "Select track xx only". Then I use "Insert empty item" and that's that.

Now I need to enter notes.

From what I see, this can be done with the method described in the Wiki http://www.cockos.com/wiki/index.php...etSetItemState

All I need to find for the notes is this:
Code:
      <NOTES
        |This is the actual note content.
        |This is a second line and so on.
      >
Each text line in the notes is preceded by the "|" character, so that shouldn't be hard.

I'm still figuring out whether or not to just create an empty session and set specific session parameters like offset, fps and timecode timeline up myself, or rely on preprepared sessions.
__________________
Dialogue/FX Editor & Re-Recording Mixer
Using Latch Preview
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 11-12-2009, 06:57 AM   #2
MikeLacey
Human being with feelings
 
Join Date: Dec 2006
Location: UK
Posts: 762
Default

Hi,

Sounds greate, tell us more....
__________________
Mike Lacey, Leicestershire, UK
MikeLacey is offline   Reply With Quote
Old 11-13-2009, 08:52 AM   #3
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: City
Posts: 9,561
Default

Got the file size checking, timecode integrity checks and all data ready and loaded.

Getting a filename from the user will be simple enough on the commandline, though I don't know if that's even possible in Reaper. Perhaps a file requester is the best here. For testing purposes it's static.

Now the Reaper stuff comes. Getting timecode and positioning the cursor, calling the actions correctly. Then there's more checks I need to implement so the session and data match in frame rate.

Not sure how to handle the timecode yet. Need to see what the cursor-positioning functions expect.
__________________
Dialogue/FX Editor & Re-Recording Mixer
Using Latch Preview
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 11-16-2009, 05:17 PM   #4
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: City
Posts: 9,561
Default

Here is the first version without putting notes in to the items as of now, but everything else works nicely.

A downloadable ZIP (~4kB) of both the script and the example spreadsheet.
http://stash.reaper.fm/oldsb/241778/...mpty_Items.zip

This code beneath is had been edited to keep within the 10kB posting limit. Just comment characters mostly.
Code:
#! /usr/bin/perl -w

use constant CURRENT_PROJECT => 0;

# Action command ids

@selected_track = 40939 .. 41037; # all the "Select Track xx" commands

use constant REMOVE_TIME_SELECTION_AND_LOOP_POINT_SELECTION => 40020;
use constant TIME_SELECTION_SET_START_POINT => 40625;
use constant TIME_SELECTION_SET_END_POINT   => 40626;
use constant INSERT_EMPTY_ITEM              => 40142;
use constant UNSELECT_ALL_ITEMS             => 40289;
use constant UNSELECT_ALL_TRACKS            => 40297;


## Section 1 and 2 - Loading the CSV ##

#
## The csv file is now hardcoded.
#  In the next revision we'll ask the user for a filename to enter
#  Perhaps we can pop up a file request at some point as well

$csv_file	=	'g:\Dev\ReaScript_CSV_to_Empty_Items\csv_data.csv';
$csv_delimiter	=	"\t";

$csv_fps = 25; # this will be configurable by checking for the FPS in the current project

$timecode_number_delimiter = ':';
$timecode_format_check =                             '\d\d' .
                         $timecode_number_delimiter . '\d\d' .
                         $timecode_number_delimiter . '\d\d' .
                         $timecode_number_delimiter . '\d\d'; # hh:mm:ss:ff
@csv_data					=	();
$csv_filesize     = ();
$csv_filesize_max	= 2**20 * 40 ; # 40 Megabytes. Hopefully this is an adequate upper limit
$csv_filesize_max_readable	= ();

## Section 3 - Checkups ##
##########################
@csv_file_line= ();
@csv_timecode_check_split = ();
$csv_column_description = 2;
$csv_column_track       = 3;
$csv_column_tcin        = 4;
$csv_column_tcout       = 5;

@timecodeslots 					= ($csv_column_tcin,$csv_column_tcout); # the slots our data lives in on each csv line
$timecode_ok_count = 0;


## Section 4 ##
###############
$csv_selected_track = ();
$reaper_time = ();



# Create an string from the maximum filesize that looks good for reports
if 				($csv_filesize_max < 2**10) {$csv_filesize_max_readable = $csv_filesize_max . " bytes\n";
	} elsif	($csv_filesize_max < 2**20) {$csv_filesize_max_readable = $csv_filesize_max / (2**10) . " kB\n";
	} elsif	($csv_filesize_max < 2**30) {$csv_filesize_max_readable = $csv_filesize_max / (2**20) . " MB\n";
	} elsif	($csv_filesize_max < 2**40) {$csv_filesize_max_readable = $csv_filesize_max / (2**30) . " GB\n";
	} else {
		reaprint ("This max filesize is too damn large at $csv_filesize_max_readable !");exit;
	}
#
##
###### end of variable initialization #####################
	




#### 1. Check CSV filesize and compain if it exceeds a given maximum

# Size check so we don't burn up too much memory with bogus data files
$csv_filesize = (stat($csv_file))[7];
if ($csv_filesize > $csv_filesize_max) {die "\nData file size exceeding maximum of $csv_filesize_max_readable\n\n"};



#### 2. Get the CSV Datafile


reaprint ("\n\nLoading file \"$csv_file\" ...\n");

open (DATAFILE,"$csv_file") || die "\nError: $!\n\n"; # open it safely
@csv_data = <DATAFILE>; # store it
chomp @csv_data; # remove linefeeds
close (DATAFILE) || die "\nError: $!\n\n"; # safely close it


#  [to be done]
#  Checks to make sure the project and data file have the same frame rate
#  Checks to make sure all other data is present, i.e. the data file has all the data cells we need

#### 3. Timecode integrity check in CSV data

##
reaprint ("Checking data integrity . . .\n");

## Checking

$i=1; # we start on the second line of @csv_data

until ($i eq scalar(@csv_data)) {
	# check timecode format
	@csv_file_line = split(/$csv_delimiter/, $csv_data[$i]);	# grab and split a line by the csv delimiter
	foreach $k (@timecodeslots) {
		# here we check for dd:dd:dd:dd for d=digits
		unless ($csv_file_line[$k] =~ /$timecode_format_check/)
				{die "Line " . $i+1 . " of the CSV file contained illegal timecode";} # format of timecode does not match expectations

		# here we check whether any of the numbers in the timecode have illegal values
		# get each of the numbers in the timecode
		@csv_timecode_check_split = split(/$timecode_number_delimiter/, $csv_file_line[$k]);

		# Hours cannot exceed 23, minutes 59, seconds 59 and frames the current frame rate - 1
		unless ($csv_timecode_check_split[0] < 24 ) { die "$k in line " . $i+1 . " contains an illegal HOURS number\n";}
		unless ($csv_timecode_check_split[1] < 60 ) { die "$k in line " . $i+1 . " contains an illegal MINUTES number\n";}
		unless ($csv_timecode_check_split[2] < 60 ) { die "$k in line " . $i+1 . " contains an illegal SECONDS number\n";}
		unless ($csv_timecode_check_split[3] < $csv_fps ) { die "$k in line " . $i+1 . " contains an illegal FRAMES number\n";}
		$timecode_ok_count++; # report back this resulting number
	}
	$i++; # next line
}
##
####### END of Timecode integrity check in CSV data ########

reaprint ("\nTimecodes that checked out ok : $timecode_ok_count\n");



#### 4. Create Empty Items

##
# RPR_Main_OnCommand(actionnumber, 0)
#
# Reaper commands we'll be using :
# Command name                         Cmd #
# ________________________________________________
# Track: Unselect all tracks           40297
# Track: Select track 01-99            40939-41037
# Time Selection: Remove time selection and loop point selection  40020
# Time Selection: Set start point      40625
# Time Selection: Set end  point       40626
# Insert empty item                    40142
# Item: Unselect all items             40289
#
#############################################
# Sequence of events for creating Empty items
#############################################
#
## 1. Clear time selection, loop selection, item selection and track selection(actions)
##
## 2. Select track specified in CSV file(action)
##
## 3. Select time selection start and end point 
##
## 4. Insert Empty item(action)
##
## 5. The item notes are changed (API call to get item info and another to set it)
##
## 6. Go to 1. until CSV list is processed
##
####
#### REMOVE_TIME_SELECTION_AND_LOOP_POINT_SELECTION => 40020;
#### TIME_SELECTION_SET_START_POINT => 40625;
#### TIME_SELECTION_SET_END_POINT   => 40626;
#### INSERT_EMPTY_ITEM              => 40142
#### UNSELECT_ALL_ITEMS             => 40289;
#### UNSELECT_ALL_TRACKS            => 40297;


## Preparation - Getting the data
#
#### 4.0 Setting up the loop and gathering the data

$i=1; # we start on the second line of @csv_data
for ($i=1; $i < scalar(@csv_data); $i++) {
	# Get the line and split it
	@csv_file_line = split(/$csv_delimiter/, $csv_data[$i]);	# grab and split a line by the csv delimiter
	
	
	## 4.1 Clear time selection, loop selection, item selection and track selection(actions)
	#
	RPR_Main_OnCommand(REMOVE_TIME_SELECTION_AND_LOOP_POINT_SELECTION, 0); # Remove time selection and loop point selection
	RPR_Main_OnCommand(UNSELECT_ALL_ITEMS, 0);	# Unselect all items
	RPR_Main_OnCommand(UNSELECT_ALL_TRACKS, 0); # Unselect all tracks

	## 4.2 Select track specified in CSV
	#
	$csv_selected_track = $selected_track[($csv_file_line[$csv_column_track])-1]; # tracks go from 1-99, the list goes from 0-98
	reaprint("Track $csv_file_line[$csv_column_track] selected.\n");
	RPR_Main_OnCommand($csv_selected_track, 0); # select the track chose in the CSV line


	## 4.3 Select time selection start and end point 
	#
	#RPR_SetEditCurPos2(CURRENT_PROJECT, double time, bool moveview, bool seekplay)
	$reaper_time = RPR_parse_timestr_pos($csv_file_line[$csv_column_tcin], 5); # get frame time code and convert to seconds, 5->this is frame-timecode
	RPR_SetEditCurPos2(CURRENT_PROJECT, $reaper_time, 0, 0);                   # set the cursor to the Timcode IN position
	RPR_Main_OnCommand(TIME_SELECTION_SET_START_POINT,0);                      # set the time selection start

	$reaper_time = RPR_parse_timestr_pos($csv_file_line[$csv_column_tcout], 5);# get frame time code and convert to seconds, 5->this is frame-timecode
	RPR_SetEditCurPos2(CURRENT_PROJECT, $reaper_time, 0, 0);                   # set the cursor to the Timcode OUT position
	RPR_Main_OnCommand(TIME_SELECTION_SET_END_POINT,0);                        # set the time selection end
	

	## 4.4 Insert Empty item(action)
	#
	RPR_Main_OnCommand(INSERT_EMPTY_ITEM,0); # insert the empty item

	## 4.5. The item notes are changed (API call to get item info and another to set it)
  #  Nothing there yet.
  #  Put the description in to the item notes
  get_and_set_item_state ($csv_file_line[$csv_column_description]);
  

  ############################1.23456789012345
  # Reaper timecode format :  0.00000000000000

}

# Cleanup
	RPR_Main_OnCommand(REMOVE_TIME_SELECTION_AND_LOOP_POINT_SELECTION, 0); # Remove time selection and loop point selection
	RPR_Main_OnCommand(UNSELECT_ALL_ITEMS, 0);	# Unselect all items
	RPR_Main_OnCommand(UNSELECT_ALL_TRACKS, 0); # Unselect all tracks

exit; # END OF MAIN PROGRAM

##############################################################################

sub reaprint {
	
	#print $_[0];
	RPR_ShowConsoleMsg($_[0]);
};



sub get_and_set_item_state {

  return; # this is not working yet. Skipping for now
}
__________________
Dialogue/FX Editor & Re-Recording Mixer
Using Latch Preview
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 11-17-2009, 03:17 AM   #5
MikeLacey
Human being with feelings
 
Join Date: Dec 2006
Location: UK
Posts: 762
Default

Pretty cool airon, a good example of what ReaScript makes possible.

You mind if I snag it (and the CSV data file) as examples for the Wiki?
__________________
Mike Lacey, Leicestershire, UK
MikeLacey is offline   Reply With Quote
Old 11-17-2009, 04:33 AM   #6
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: City
Posts: 9,561
Default

Go right ahead. Edit it for clarity if you must.

I'm still trying to figure out how to put notes in to the items. There appears to be no API call, and inserting the text in to the items is not something I've been able to pull off yet. It's in the last block of the script in the zip.

Btw, a screencast of what the script does is here : http://screencast.com/t/NzI2YzdlN
__________________
Dialogue/FX Editor & Re-Recording Mixer
Using Latch Preview
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 11-18-2009, 08:05 AM   #7
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: City
Posts: 9,561
Default

Updated the script. Thanks to some help from Mike Lacey, it now puts Notes in to these newly created Empty Items.

http://stash.reaper.fm/oldsb/242619/...mpty_Items.zip

A demonstration with the data included in the ZIP.
http://screencast.com/t/MGVlNTAzZT

Since precise whitespace placement doesn't seem to matter, it shouldn't be too hard to adapt the routine to erase the current notes and replace them. You actually only need to erase all the lines that describe the notes (<NOTES\nstuff or not>) and insert new ones with the existing routine.

One thing I still need to implement is to check the given text for < and > characters, so they don't muck up the item.

The new code, implemented as a sub routine that takes the new text to be used as an argument :
Code:
	my $text = $_[0];    # the only argument - the text to be inserted as a note
	my @text_split = (); # where multiline text lives until we build the text block to be inserted
	my $notes = ();      # where we'll place the finished data to be inserted in to the item
	my $delimiter = "\n";
	my ($bool, $it, $chunk, $maxlen);
	my $z;
	
	my $length = -1;
	my $result = "fail";
	
	# Build the notes block
	
	@text_split = split(/$delimiter/, $text);	# split the text in to lines
	chomp @text_split;
	
	$notes = "<NOTES\n";
	for ($z=0;$z<scalar(@text_split);$z++){
	$notes = $notes . "|" . $text_split[$z] . "\n";
	}
	$notes = $notes . ">\n";
	
	# Get the first selected item in the current project
	$it = RPR_GetSelectedMediaItem(CURR_PROJ, 0);
	
	# set-up for call to GetSetItemState
	$chunk="";     # Get, not Set
	$maxlen=2048;  # max num of chars to return
	
	# Get the ItemState
	($bool, $it, $chunk, $maxlen) = RPR_GetSetItemState($it, $chunk, $maxlen);
	$result = "pass" if $bool;
	#RPR_ShowConsoleMsg("GetSetItemState reports $result\n$chunk\n");
	
	
	# insert them after the IGUID line
	$chunk =~s/(IGUID.*}\n)/$1$notes/;
	
	# Set the ItemState
	$result = "fail";
	($bool, $it, $chunk, $maxlen) = RPR_GetSetItemState($it, $chunk, $maxlen);
	$result = "pass" if $bool;
As you can see there's still a bit of cleanup to do. But it's ready for demoing at the presentation on friday.
__________________
Dialogue/FX Editor & Re-Recording Mixer
Using Latch Preview
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 11-18-2009, 01:41 PM   #8
MikeLacey
Human being with feelings
 
Join Date: Dec 2006
Location: UK
Posts: 762
Default

Neat... And thanks for sharing your Notes code, having this kind of starting point for people really helps/

And all the best for your demo on friday, this is good stuff.
__________________
Mike Lacey, Leicestershire, UK
MikeLacey is offline   Reply With Quote
Old 11-18-2009, 07:09 PM   #9
Shan
Human being with feelings
 
Shan's Avatar
 
Join Date: Mar 2007
Location: Vancouver
Posts: 2,277
Default

Quote:
Originally Posted by airon View Post
Updated the script. Thanks to some help from Mike Lacey, it now puts Notes in to these newly created Empty Items.
Congrats on this great script! This will be a good one for the POST users.

Shane
__________________
"Music should be performed by the musician not by the engineer."

Michael Wagener 25th July 2005, 02:59 PM
Shan is offline   Reply With Quote
Old 05-05-2017, 02:45 AM   #10
Sebadier
Human being with feelings
 
Join Date: Apr 2016
Location: Berlin
Posts: 4
Default Is this still working

This script seems to be great and extremely useful to me as I'm working in ADR.

But there is no way to get perl scripts running in Reaper64, is there?

Sebastian
Sebadier is offline   Reply With Quote
Old 05-05-2017, 03:16 AM   #11
airon
Human being with feelings
 
airon's Avatar
 
Join Date: Aug 2006
Location: City
Posts: 9,561
Default

Reaper stopped supporting Pearl a while ago if I remember correctly.

This script would need to be converted to Lua.

Maybe there's a script somewhere that deals with CSV data though. Worth a search.

On the flipside, I wrote a script that does the opposite, export selected items as a CSV file. Iirc it even includes the item notes.

I posted it here: http://forum.cockos.com/showpost.php...08&postcount=3

When I get around to it, I'll take a crack at writing a Lua version of the CSV to Empty Items script. It doesn't look very hard.
__________________
Dialogue/FX Editor & Re-Recording Mixer
Using Latch Preview
"My ego comes pre-shrunk" - Randy Thom
airon is offline   Reply With Quote
Old 05-06-2017, 06:44 AM   #12
Sebadier
Human being with feelings
 
Join Date: Apr 2016
Location: Berlin
Posts: 4
Default

Ok, thanks for the fast reply...
Unfortunally I don't have any clue about LUA, otherwise I would try to translate it myself. But if you want to give it a try some time, great. Many people would appreciate it I assume.
Sebadier is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 05:04 AM.


Powered by vBulletin® Version 3.8.9
Copyright ©2000 - 2017, vBulletin Solutions, Inc.