Website Maintenance Process

How I keep this website up-to-date - first of all, new tune page creation/updating;

Now here is an alternate, more up-to-date (Visio-generated) view; FF_TT Source Create Process Drawing_Dec2008.htm

As can be seen from the above flowchart, it's mainly done using a variety of perl scripts. These are generally found in C:\User\Wayne\FumblinFingers\TT_webproject, and include;

1. MakeTuneFolderTT.pl (Create new tune folders en masse), and
2. MakeABCps6.pl (Generate .pdf from .abc file(s).)
3. wdm_abc2lynew1.pl (Generate lilypond file(s) from .abc file(s).)

The second one, for instance, is used to automagically create .pdf versions of new or updated .abc files, using the console command;

  >perl -w MakeABCps6.pl -v 

The command has a variety of switches, the one shown is used to track progress (verbose) of the script. By default, the script only processes files that do not have a .pdf file older than the .abc file, in other words, .abc files that have been changed since the last run of the script. You can force it to work on all files with the switch -a. These are the most useful ones, for the others, see the script itself.

All files which are processed by this command result in a .pdf file, as well as a .log file, showing the output of the jaa2ps program.

I also use a very convenient perl script set included with the standard perl installation, called "ttree". It uses a (set of) configuration files (in our case called 'TTreeCfgHome.cfg', located in the TT_webproect folder), and template files; individual ones stored in a 'source' folder, as well as common ones stored in a 'lib' folder. It is invoked with the console command;

  >ttree -f TTreeCfg.cfg       #For use at work
  >ttree -f TTreeCfgRoot.cfg   #To compile the Music index page
  >ttree -f TTreeCfgHome.cfg   #For use at home

Note that the easiest way to ensure the html file gets recompiled when you've made changes to anything other than the template file (such as the wikipedia notes file, or the abcs or their pdfs), is to simply delete the entire contents of the output folder in question. Alternatively, you can run ttree with the -a switch, and wait a while longer for it to complete.

I still a few more tools to complete the kit; in particular, I need one to automatically update the main Music page, to keep track of which files are which, and include new ones as they are added.

This last function is done somewhat differently than the previous tools - they still use perl sripts and the Template Toolkit, but with a preprocessing function (at least for now). The first thing I do is extract embedded (Template Toolkit) variables from the individual tune template, that indicate the type of tune, and the status of the whole folder. This is done using a perl script "ScanSongs1.pl", which scans all the tune files and extracts the variables, placing all of them in a text file ("SongsList.txt"). This is then used when I run TTree on the music index template, to sort the tunes and report on their status. At the present time, I also have to move the resulting Music Index html page back into the web page folder...

  >perl -w ScanSongs1.pl 
  >ttree -f TTreeCfgRoot.cfg   #To compile the Music index page

The final steps in processing a given tune is to generate a lilypond source file for the FF version, using the wdm_abc2lynew1 perl script, edit the result to work properly (there are still some gotchas in the process), and create a snippet picture of the first 4 bars or so. The latter is done by copying the final .ly file, and editing it to only those first 4 bars (see "Atholl Highlanders" for an example). The resulting .pdf file can then be exported (using adobe acrobat) to a .png file and cropped to fit on the webpage.

Note that the lilypond source file created by the wdm_abc2lynew1 perl script is not perfect, and is subject to some editing before it is useful. In particular, at this time (and possibly forever), the guitar chords generation process creates a guitar chord for each note, because it steals the duration from the notes themselves. I thought this would be fine, since lilypond only displays chords when they change. Unfortunately, it also seems to 'reserve space' for all of them, which pushes out the notes. So, as part of the inevitable editing of the lilypond source file, we need to compress the guitar chords, i.e. take the multiple (identical) chords and create a single chord, of the same duration as the set. The other editing we have to do includes adding or deleting the \partials (depending on whether I create them or not in the perl script), and adding in \breaks where they are needed.

16 September 2010 - I noticed the other day that the postscript files generated by lilypond are all in A4 format - the reason I didn't notice before now is that my printer driver is configured by default to shrink to fit! Unfortunately, my search for the configuration file that sets this by default to A4 turned up nothing, so I need to insert the line "#(set-default-paper-size "letter")" at the top of every lilypond file. Since I've added a default "\include "../../FF_LilypondInclude.ily"" to all of my .ly files, I simply needed to add it to that file to have it included in all, but of course, now (almost) all of my postscript files will "roll over" to the next page. Since this won't actually happen unless I recompile the .ly file, I'll deal with it on a case-by-case basis. One of those cases will be the tune books, but again, I'll have to sort that out when I do it anyway.

19 Sept 2022 - After doing a re-org of my music, I will have to update all my PERL scripts. While I am at it, I would like to do my best to simplify and streamline the processes. Hopefully I will get some use out of this before I get the notion to recreate it all using a database and Flask or Django...

So, the first thing I would like to do is do away with separate .cfg files based on which platform (Linux, Win10 laptop, Win10 desktop). I'm pretty sure I can do this using the Config::INI module, placing the .ini file at the root (so that it doesn't get rolled up whenever I do a WinMerge synch) - I'll call it wdmFFPerlConfig.ini.

Testing: make a simple perl script to read the config.ini file - the .ini file (wdmConfig.ini) for the laptop will look something like:

[wdm_this_site]
os = Win10
home = D:\Users\wmerc\Documents\FileSync\
ff = FumblinFingers\

For the desktop, it will look something like:

[wdm_this_site]
os = Win10
home = E:\Users\Wayne\FumblinFingers\
ff = FumblinFingers\

And for linux, it will look something like:

[wdm_this_site]
os = Linux
home = ~/
ff = FFmusic/

To install the Config::INI module, from a CLI type: "cpanm config::INI"

To test the config file, create a test script:

my $hash = Config::INI::Reader->read_file('family.ini');