Machine readable results

FreeLCS 3.0 lets you define how it outputs its results. Most users will want the old behavior from FreeLCS version up to 2.4 that is: create a loudness corrected audio file, a history graphics file and delete multistream files immediately after all audiostreams have been extracted to individual files (multistream extraction requires installation of FFmpeg or libav-tools). This "old" behaviour is achieved by setting "Use FreeLCS file output defaults" to "Yes".

If you would like to integrate FreeLCS as a loudness measurement step in your automated file processing workflow, then you might want to change the defaults.

Separator characters in the machine readable results file

There can be many audio streams in one input file, FreeLCS writes these streams to individual audio files before loudness processing (multistream support requires FFmpeg or libav-tools). As a single input file can result in many output files, the machine readable results file may consist of loudness results of many audio files.

Loudness measurement results of a single file needs to be separated from each other by a known character or group of characters for a machine be able to interpret the results. If the input file resulted to many output files, then we need to separate each record of results with another known character or group of characters. The separator characters must be chosen from a group of characters that can never appear in a file name.

This problem has been solved when the ASCII character set was designed. The designers assigned two characters for this job: ASCII character 31 is meant for separating different values from each other and ASCII character 30 is meant for separating different records of data from each other. Character 31 is called the Unit Separator and character 30 is called the Record Separator. The operating system doesn't allow these characters in file names, so they are safe to use as data separators.

The machine readable results file is a simple text file with loudness measurement results written to it. Sometimes there might be reason for a person to open a machine readable results file in a text editor to check the values inside it. This is not possible if the ASCII character 30 is used as the record separator. In this case the user sees the results as one huge continuous line of text. This is why I chose to use the windows line feed string (ASCII character 13 followed with character 10) as the default record separator. When this record separator is used, then a machine readable results file becomes much more readable when opened in a text editor, because the editor splits results for individual audio files as separate lines of text.

However you can select any non printable ASCII character as the unit and record separator. You can use 1 or 2 character strings for each.

Machine readable results file format

The machine readable results file name is constructed like this:  InputFileName-machine_readable_results.txt

Example: Input file name is:  Euroviisut-2007-Final.mp4

Machine readable results file name is:  Euroviisut-2007-Final.mp4-machine_readable_results.txt

When you know the name of the input file, then you can get the machine readable results file name just by appending "-machine_readable_results.txt" to the name.

A machine readable results file has information about all audio streams of a single input file. Values for a single output stream are separated with the unit separator string. The record separator string is placed where a record of values for one audio stream ends and a record of values for the next audio stream begins. Unit and record separator strings are both defined during FreeLCS installation.

The items in the machine readable results file are:

Note: item count starts from zero.

00 = The number of this audio stream.
01 = Total number of supported streams in the input file.
02 = Total number of streams in the input file.
03 = Create Loudness Corrected Files (True / False).
04 = Number of files in this audio stream.

05 = Integrated Loudness (I) (If loudness is below measurement threshold, then this value is 'inf').
06 = Loudness Range (LRA).
07 = Highest Peak (dBTP or dBFS depending which measurement method the user selected during installation).
08 = Number Of Channels in this audio stream.
09 = Sample Rate.
10 = Bit Depth.
11 = Duration rounded to seconds.

12 = Loudness calculation error code (Codes explained below).
13 = Reason for error (text string).

14 = List of output filenames.

Detailed explanations of the items:

Items 00 - 04:

• Item 0: The number of this audio stream. If there are many audio streams in the input file, then this number tells you which one of these streams this mix is.
• Item 1: Total number of supported streams in the input file. There can be both unsupported and supported streams in a input file. This value tells you how many streams FreeLCS is able to process from the input file.
• Item 2: Total number of streams in the input file. This number contains both the unsupported and supported streams in a input file.
• Item 3: Create Loudness Corrected Files (True / False). This value tells you if the user has instructed FreeLCS to create loudness corrected files or not (this is set during FreeLCS installation). If this is "False" then FreeLCS will not produce loudness corrected audio files.
• Item 4: Number of files in this audio stream. This tells you how many audio files this audio stream resulted to. This number is always 1 except in two cases. It is 0 when an error happened and FreeLCS was not able to process the stream. In this case this number tells you that there are zero resulting output files.  The other case when this value differs from 1 is when the output file size is so big that FreeLCS needs to split the file to one file per audio channel. In this case this number tells you the number of audio files the audio stream was split into. Items from number 14-> will tell you names of the output files.

• Item 5: Integrated Loudness of the input file. If loudness is below measurement threshold, then this value is 'inf'.
• Item 6: Loudness Range of the input file.
• Item 7: Highest Peak of the uncorrected input file  (dBTP or dBFS depending which measurement method the user selected during installation).
• Item 8: Number Of Channels in this audio stream. This tells you how many channels there are in this audio stream.
• Item 9: Sample Rate.
• Item 10: Bit Depth.
• Item 11: Duration rounded to seconds.

• Item 12: This value holds possible error codes. If this value is 0 then FreeLCS successfully processed the stream, otherwise processing failed and there will be no loudness calculation results nor loudness corrected files for this stream. Error code 9 means that in addition of supported files, there are some audio files inside the MXF wrapper that FreeLCS is not able to process. When MXF audio remixing feature is on, then this situation may produce unexpected results meaning that audio channels inside the MXF file might be combined in ways that the user did not anticipate. MXF audio remixing works correctly only when FreeLCS is able to process all files inside the MXF wrapper.
• Item 13: This item is the error message text. If there is no error then this item is empty.

• Item 14: The items from number 14-> will tell you the names of output audio files. If the output file has been split to one file per audio channel, then here are all the output file names separated by the unit separator..

Error codes for item number 12:

Example 1:

Example 2:

Example 3:

MXF audio remixing

FreeLCS 3.0 lets you define how you want audio inside an MXF file to be recombined before loudness processing. This feature is called MXF audio remixing.

If you want to use MXF audio remixing, then you must install FFmpeg or libav-tools and allow MXF wrapper format processing during FreeLCS installation.

Important note: This function is not guaranteed to work !!!!!!! MXF is not one format, but a family of formats (more info on Wikipedia). Every hardware and software vendor seem to have their own version of it. MXF files created with one device might not be compatible with a device created by another vendor. Because of this there is no guarantee that FreeLCS can process every MXF - file you feed it. It might be a good idea to do a thorough test with a large set of different MXF files before you make this feature part of your workflow.

The MXF Problem

Consider an example where video and audio of a program is wrapper inside an MXF wrapper. The program has two versions of the audio: a stereo version (2 channels) and an 5.1 version (6 channels). Audio might be stored inside the MXF wrapper as one 8 channel audio file (Example 1), or as 8 separate mono files (Example 2).

Example 1:

Input #0, mxf, from 'Testfile-1.mxf':
    Duration: 00:45:37.20, start: 0.000000, bitrate: 59232 kb/s
        Stream #0.0: Video: mpeg2video (4:2:2), yuv422p, 1920x1080 [PAR 1:1 DAR 16:9], 50000 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc
       Stream #0.1: Audio: pcm_s24le, 48000 Hz, 8 channels, s32, 9216 kb/s

Example 2:

Input #0, mxf, from 'Testfile-2':
    Duration: 00:45:37.20, start: 0.000000, bitrate: 60157 kb/s
        Stream #0.0: Video: mpeg2video (4:2:2), yuv422p, 1920x1080 [PAR 1:1 DAR 16:9], 50000 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc
       Stream #0.1: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.2: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.3: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.4: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.5: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.6: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.7: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
       Stream #0.8: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s

Audio from these MXF files can't be processed with FreeLCS because it has no knowledge about which files belong to the same audio version (mix). This information can be provided to FreeLCS in the form of a MXF remix map.

MXF remix map

A MXF remix map tells FreeLCS how audio channels inside a MXF file must be recombined before loudness processing. When a remix map is active then FreeLCS first extracts audio from the MXF wrapper and then recombines audio channels according to the remix map and writes each resulting mix to a separate audio file. Then FreeLCS moves these files to the HotFolder so that they go through the normal loudness processing.

Default MXF remix map

You can define a default remix map during FreeLCS installation. This default map is used when there is no other information available about a specific MXF file.

In the picture below a green number above a box corresponds to the number of the mix. The number inside a box defines how many audio channels this mix has. A total of 48 remix parameters can be defined meaning that channels inside an MXF file can be recombined to max 48 new audio files. The default remix map is all number 2's. This means that if there is no file specific MXF remix map found then recombine all audio files found inside a MXF file to stereo files.

1. Stereo
2. 5.1
3. Stereo
4. Stereo

Then you could define a default remix map of: 2, 6, 2, 2  meaning that you put number 2's in boxes 1, 3 and 4 and number 6 in box 2. You can have more mix definitions than you have physical audio channels inside your MXF files, FreeLCS only recreates those mixes that can be completely created with audio channels available in the MXF file. If there are more audio channels inside an MXF file than is needed to recreate all mixes in the remix map, then the extra channels are not used for anything.

File specific MXF remix map

In addition to the default remix map the user can use file specific MXF remix maps. A file specific remix map overrides the default remix map defined during installation. For example if most of your files have the same channel configuration, then you would assign this configuration to the default MXF remix map during FreeLCS installation. Once in a while you might want to process a file that has a channel configuration differing from your regular MXF files. In this case you could override the default channel configuration only for this file with a file specific remix map.

A file specific remix map is a simple text file that contains channel counts separated with a comma. The name of the remix map file is always the same as the name of the MXF file it is used for, except the file extension of the original file ".mxf" is replaced with ".remix_map". You can have more mix definitions than you have physical audio channels inside your MXF file, FreeLCS only recreates those mixes that can be completely created with audio channels available in the MXF file. If there are more audio channels inside an MXF file than is needed to recreate all mixes in the remix map, then the extra channels are not used for anything.

Example:

As you can see, it is very easy to use file specific remix maps. You just create a text file and put some numbers in it separated by commas. The numbers are the channel counts for each mix you want FreeLCS to recombine before loudness processing.

Important: a file specific remix map must be in the FreeLCS HotFolder BEFORE the MXF file arrives there. FreeLCS checks once for each MXF file if it can find the corresponding file specific remix map, and if it is not found then FreeLCS processes the file according to the information it has available then. FreeLCS won't check for the existence of a remix map again after processing has started.

Detailed explanation of the channel remixing feature

I have seen some very peculiar MXF files. One file had several mono audio files in it except in between the mono files there was a single stereo file. How is FreeLCS supposed to know from which file to take the audio channels user has defined in a remix map ? Because of this peculiarity I designed FreeLCS to be "blind" to the physical audio files inside a MXF file, FreeLCS only "sees" a bunch of audio channels available to it. FreeLCS does not care which physical file the channel comes from, it just uses channels starting from file 1 channel 1 and going forward to the next file until it gets all channels needed to fulfill a channel count defined in a remix map.

Example:

Most MXF files will have several mono files or 8 channel files inside but consider this theoretical example. Suppose the channel counts in audio files inside a MXF file was:

What happens if the user defines a channel map where required channel counts for each mix to recreate is the opposite of the available channels inside the MXF:

Remix map: 4, 3, 2, 1

This is how FreeLCS gets channels for each channel count defined in the remix map:

Channels left in physical audio files before processing starts: 1, 2, 3, 4

Remix map item 1 needs 4 channels: FreeLCS  takes all channels from physical files: 1 and 2 totalling 3 channels, then it needs one more channel and takes the first channel of file 3. After the first mix is created channels left in physical audio files: 0, 0, 2, 4.

Remix map item 2 needs 3 channels: FreeLCS takes the 2 channels that are left in physical file 3 and then the first channel from physical file 4. After the second mix is created channels left in physical audio files: 0, 0, 0, 3.

Remix map item 3 needs 2 channels: FreeLCS takes channels 2 and 3 that are still unused in physical file 4. After the third mix is created channels left in physical audio files: 0, 0, 0, 1.

Remix map item 4 needs 1 channels: Then we only have channel 4 left of physical file 4 and that fulfills the last remix map definition for 1 channels. After the fourth mix is created channels left in physical audio files: 0, 0, 0, 0.

FFmpeg / Libav-tools allowed wrapper and codec formats

In FreeLCS 3.0 it is possible to restrict FFmpeg / libav-tools to only process a small set of formats or allow it to process all formats it supports. This feature is mainly designed to allow the user to restrict processing only to patent free formats even when FFmpeg or libav-tools is installed. Format patentability varies from country to country, see more about it in FAQ.

If you enabled MXF remixing, then you must install FFmpeg or libav-tools and allow MXF wrapper format processing. In the picture below libav-tools is allowed to process all formats it supports (All includes MXF :)

The option "Mp4" enables wrapper formats: mp4, m4v, m4a and formats based on mp4: mov, 3gp, 3g2, mj2.