NIH or multi-lingual interactive help

(Interactive help system allowing an harmonious coexistence between English applications and localized applications) Current release is 0.3
Author: Benoît GILON

Overview

Target audience for this document

You are an end user speaking a language other than English (German, Spanish, Italian...);
You are a developer wishing to market applications outside English speaking countries;
In case you speaks French, then you'd better navigate to the French speaking sister page

Background

Localization of RISC OS applications is likely to be the most straightforward and flexible on our planet.

People who would like to develop and market applications in non English speaking countries have to use standard development tools, with one file per resource type and supported language (or a directory per supported language).
Applications written in BASIC can benefit from the code fragment named ResFind (written by Olaf Krumnow and Herbert zur Hedden from the German Archimedes Group).
Applications written in C can benefit from the SetRes program (I wrote) for all Toolbox based Wimp applications (or else refer to the Local article for applications still using RISCOSlib).
Interactive help allows client applications, as they submit text messages to display in !Help application's window, to specify tokens which will be then translated by the !Help application into readable sentences (after having read an internal dictionary).
This shortens the data (and then help to put more text in a single Wimp message).
Every token is prefixed in the transmitted data by the backslash character (\) which acts as an escape character. Following this prefix is a single character (code is greater than space) specifying the pattern for the !Help application to use when reading from the dictionary. You can find samples of such messages by viewing message files in the Resources:$.Resources hierarchy. Navigate to the subdirectories associated with classic Acorn desktop applications (such as !Alarm).
Unfortunately, only translating the Messages file within the Help resource directory can only solve the problem provided that every installed application or system component on the user computer speak the same language. At most times, this should be English because:

Some English applications (on your desktop) still have to be translated;
You hope to install new English speaking only applications (and using the interactive help facility) in the future.

If all you do is to translate the Message file in your Help resource directory, then such terrible message as "Cliquez SELECT pour merge the paths." would pop up in your Interactive Help window (I assume here that the user speaks French, "Cliquez SELECT pour" means "Click SELECT to"). The underline red italics part comes from the !Help application internal dictionary and the trailing part comes from the English speaking application (the text included in the Wimp message being "\Smerge the paths").
The well known user friendliness of the RISC OS desktop should also apply in these domains (text clarity). So (and up to now), interactive-help definitions from within resource files supporting other than English language of a localized application must held no token. This implies:

Even more keystrokes during the development phase to input clear text for non English languages;
Bigger memory footprint for the application;
Downgraded performance (?).

Provided tools abilities

The following paragraph details a method to implement inside the two most well known IH applications (namely Acorn !Help and Miles Sabin's !BubbleHlp) an alternate dictionary for user's language token translation along with the unmodified English version.
To guide the IH application to differentiate the messages, the new tokens will be prefixed by two backslashes instead of only one. The incoming text data "\\Srassembler les chemins." will then be translated to the string "Cliquez SELECT pour rassembler les chemins." to be displayed in the IH application's main window. This implies:

The !Help application (as well as the Bubble help application) should be patched to handle both token sets, and (for the former only) to conform to differencies between OS releases in term of token definitions (for non UK Messages files);
It should also be updated so that the Help$Path environment variable setting reflects the user country setting;
Client applications should be updated in order to benefit from the new interactive help system and should be able to detect the interactive help application which is currently installed and is likely to be started.
The application developer (or the end user himself) can use the tool provided (named unsuprisingly !NIHTool)to assist in converting the message files to the new standard.

!Help application patch (!RunImage file)

Token expansion

The BASIC procedure in charge of translating data originated from the client applications is named "e": below is a code extract from it:

DEFPROCe:LOCALx%,T%,M%,A%,p%,m%,N%,I%,y%,oa%: \
SYS39,a%+20,n%,512+(1<<31)TOp%,,A%:A%+=n%:?A%=13:p%?-1=13:t$=$(a%+20):IFt$=c$ENDPROC
c$=t$:y%=J%:FORx%=n%TOA%:?y%=?x%:IF?x%<>ASC("\")THEN
y%+=1
ELSE:x%+=1:IF?x%=ASC("\")THEN
y%+=1
ELSE:$(y%)=FNa("T"+CHR$(?x%)):IF?y%=ASC("T")AND?(y%+1)=?x%ANDLEN($(y%))=2THEN?y%=ASC("\")
y%+=LEN($(y%))
ENDIF
ENDIF

And now, the patched version which uses an additional local string variable z$ (holding the first letter from the token to be used as a key when reading the token definition from the dictionary):

DEF PROCe:LOCAL x%,T%,M%,A%,p%,m%,N%,I%,y%,oa%,z$: \
SYS39,a%+20,n%,512+(1<<31)TOp%,,A%:A%+=n%:?A%=13:p%?-1=13:t$=$(a%+20):IF t$=c$ENDPROC
c$=t$:y%=J%:FOR x%=n%TOA%:?y%=?x%:IF?x%<>ASC("\")THEN
y%+=1
ELSE:x%+=1:IF?x%=ASC("\")THENx%+=1:z$="U"ELSEz$="T"
$(y%)=FNa(z$+CHR$(?x%)):IF?y%=ASC(z$)AND?(y%+1)=?x%ANDLEN($(y%))=2THEN?y%=ASC("\")
y%+=LEN($(y%))
ENDIF

Dealing with different token dictionaries (for non UK Messages hosts)

The procedure in charge of opening the dictionary message file is procedure "t". Here is the new procedure which handles an OS specific version at first.

DEF PROCt:LOCAL F%,M$:M$="Help:<Boot$OSVersion>"
DIM e%256:SYS30,6,,,17+LEN(M$)TO,,t%:$(t%+16)=M$:SYS398592,,t%+16TO;F%
IF F%AND1THEN$(t%+16)="Help:Messages"
SYS267521,t%,t%+16,0:ENDPROC

BubbleHelp application patch

Two topics are covered here:

Originally, Bubble help resource file selection was based only on the Territory status of the host computer. Now it is based on the country setting too (the reason for this is that many non UK computers (mine included) don't have a territory module following their user country/language setting).
As in !Help application, a patch sould be applied to make BubbleHelp deal with two token sets instead of only one.

The first update involves changes to Config.s.ConfigMain; Help.s.ConfigMain and Kernel.s.AString source code files. The second update involves changes to Help.s.HelpExpand source code files.

`Help$Path` environment variable setting

This is usually done within the !Run file of the !Help application. As !Help is a BASIC application, the use of ResFind comes to mind. Below is the original !Run file content (stored in the Resources:$.Apps.!Help directory):

| > !Run
| Check for the required amount of memory first
WimpSlot -min 32K -max 32K
| Having succeeded there, do not start up if
| Help is already running.
If "<Help$Dir>"<>"" Then Error Help is already running
| Then start the application
Set Help$Dir <Obey$Dir>
Set Help$Path <Obey$Dir>.,Resources:$.Resources.Help.
Run Help:!RunLink

And now the new !Run file (a modified version can be found in the archive, to be used if the application should be started from within the Apps icon on the iconbar, the RISC OS command AddApp was used to place this icon during the boot sequence).

| > !Run
| Check for the required amount of memory first
WimpSlot -min 32K -max 32K
| Having succeeded there, do not start up if Help is already running.
If "<Help$Dir>"<>"" Then Error Help is already running
| Then start the application
Run <Obey$Dir>.Resources.ResFind
Unset HelpRes$Path
Unset Help$Dir
If "<HelpRes$Dir>"="<Obey$Dir>" Then /Resources:$.Apps.!Help
Set Help$Path <HelpRes$Dir>.,Resources:$.Resources.Help.
Unset HelpRes$Dir
Set Help$Dir Resources:$.Apps.!Help
Run <Obey$Dir>.!RunImage

Client applications changes

Toolbox based applications

The current paragraph will describe a method for updating Acorn toolbox based applications so that they can easily benefit from the new help system. I presume that they already used the SetRes program I often use in applications I've written. As a reminder, SetRes sets an environment variable (<AppNameR$Dir>) according to the current setting of another environment variable ("<AppName>") and to the country currently set (which value can be read/written with the Country RISC OS command).
For every supported language (other than English), we'll have two versions for the file res (presuming that all text data going to the IH window are defined within this file).

the file <AppNameR$Dir>.res for messages which make no use of tokens;
the file <AppNameR$Dir>.nhelp.res for messages making use of tokens.

An issue with the toolbox_initialize API is that it requires that the pathname given for it to search the res and Messages files must not be a pseudo FS specifier (like AppNameR: which use the setting of the AppNameR$Path environment variable). To work around this glitch, we have to duplicate the Messages file in both directories. The choice of one file from the list above is based on the criteria below:

The first detection to perform is to determine the "default" interactive help application (either Acorn !Help or !BubbleHlp) according to the value of the Help$Start system variable.

It exists and its value equals "<BubbleHelp$Dir>" (macro expansion no withstanding), then !BubbleHlp is the default application; the program then tries to find an answer to each query below:
- Does the BubbleHelpTokens$Path system variable exist?
- Does the BubbleHelpTokens:CountryName directory exist?
The resource file <AppNameR$Dir>.nhelp.res will be selected only if all answers to the above queries are yes.
It does'nt exist or its value is empty, then Acorn !Help is the default application; then the program tries to find an answer to each query below:
- Does the Help$Path system variable exist?
- Is the file system supporting the file Help:!RunImage ResourcesFS?
- Does the Help$Dir exist? If yes, this implies that the !Help application is active.
The resource file <AppNameR$Dir>.nhelp.res will be selected only if all predicates below are true:
- The Help$Path environment variable exists;
- The file Help:!RunImage is not supported by the ResourceFS file system (cf the short BASIC program Resources:$.Resources.Help.RunLink);
- The country which eventually appears in the Help$path environment variable setting is the same as the one selected for the application;
- The directory <AppNameR$Dir>.nhelp exists and holds at least the two resource files res and Messages;
It does exist but its value is something else, then another interactive help application is installed and we choose not to select the res file from the .nhelp directory.

The program which performs all those checking is a small piece of BASIC code (which is inspired from code inside ResFind) and is included in the archive. This program takes the current value of the <AppName>R$Dir variable, and output a modified value for it according to the predicate evaluation described above.

BASIC and RiscOSLib based applications

BASIC applications update mechanism is based upon the use of two small BASIC modules.

ResFind written by Olaf Krumnow and Herbert zur Hedden from the German Archimedes Group you already know;
MResFind which is part of the enclosed archive. Takes the result value of the AppNameRes$Path system variable being set as a result of the call to the module above, and preprends it (or leave it unchanged) with the adhoc directory to take into account a new "Messages" file holding tokenized text for returning to the IH application. The same rules as above apply for selecting the right resources res file.

From the point the change occurs, trying to access the file under the pathname AppNameRes:Messages will access the file <AppNameRes$Dir>.nhelp.Messages instead of <AppNameRes$Dir>.Messages.

Sample applications which have been updated this way to make them compliant with NIH are the latest release of !Player from ESP and my own localisation initiative of !StrongED (4.62) you can find on this Web site too.

An helper application: !NIHTool

This tool can help you doing these tasks:

Convert your old localised Messages files (with no or not many tokens in them) to the format compatible with NIH (tokens prefixed by '\\');
Starting from a new format Messages file, build the original version from it: that means that you can have only on reference document;
Work with the original Messages files from not yet localised applications to find out if you could shrink their size a bit by finalising the "tokenisation" process.

Typically the process can be described shortly: It takes input from three sources; two are files (the last one being some command line option setting)

A "dictionary" which holds one or more sets of definitions (token toward clear text and vice versa);
A source file which holds the data to be converted in some way.

Those two mandatory parameters and others can be entered with the help of the dialog box below:
NIHTool setup window

The source file pathname should be entered in the 'Source' writable field (dragndrop operations of desktop file icons can also occur);
The dictionary file pathname can be changed from the default value in the same manner (writable field 'IH message';
The throwback check button specifies that the user whishes to fix potential errors to come via an external editor;
The expand check button specifies which conversion should apply to the source data file:
- to convert tokens to 'clear text' segments, leave this button checked;
- to convert 'clear text' back to tokens: this button should be unckecked

The more esoteric options can be activated via the use of the popup menu below:

The dictionary character specifies which character set is to being used for reference in the conversion process to come. By default, it's 'U' which means that the localized token set will be used (eg. extract from the !Help:Messages in my RISC PC with NIH installed: US:Cliquez SELECT pour ). However, if you want to deal with original message files, you can specify the original token set by setting this parameter to the value 'T' (eg. another extract from the !Help:Messages in my RISC PC: TS:Click SELECT to ).
Usually, the conversion process ignores sections of text from the source file which are commented. However, some applications (!StrongED among others) have a dedicated format where the '#' sign serves other purpose than marking a commented section begin. This is the 'raison d'être' of the 'Process commented segments' option which lets the tool consider the '#' character as a normal character.
By default, the escape sequence is defined as '\\' which is adequate for dealing with NIH Messages files, however, you might need to deal with the original escape sequence ('\') just to ensure that you fully benefit from the compression facility provided by the IH dictionary system (in terms of disc/memory footprint).
In addition, an option ('-p') exists to specify lines which are due to be processed and lines which sould be kept unchanged. If the option includes a string parameter (in GSTrans format), then only those lines which begin with that string will be converted.
NIHTool -p Hicon ... will process only lines from the source file which begin with the Hicon string pattern.

Conclusion

After having performed the above changes, you will have:

(Not yet localized) applications speaking English at all times;
Localized and properly configured applications speaking user selected language at all times;
Independence between the interactive help system and the client applications (hence the change description in the client applications which makes them compliant with both the original and new interactive help systems);
Big localized applications consume less space in memory if the user chooses to install the new help system.

I hope that this article will help the reader who is involved in localizing is desktop. Do not hesitate to contact me if in need of further explanations.
Remarks are welcome... click here to drop me a note.