Internationalization, Localization, and Branding of NetBeans

Overview

# Contents

  Overview and How to Test
  I18N Features
    Bundle messages
      Bundle debugging
      How to write good `*.properties` files
    Other localized text (not in source code)
    Templates
      Template descriptions
    Localized file names
    Localizing layers
    Images
    Help documentation
    Module names in manifest files
  Physical Placement of Localized Resources
  Lists of files to be localized
  Branded Localization
  Discussion

# Overview and How to Test

This document gives recommendations on how to organize I18N (localization) of modules in the IDE or platform.

You will also want to see the JDK I18N documentation (including the list of supported locales) for background information.

Japanese localization is a likely candidate for a non-US-English locale, so if you are working heavily on I18N you probably want an OS with Japanese support installed: fonts and so on. To test localization, start the IDE with the `-locale` switch, e.g.:

    -locale ja

Or the JDK may predefine this information for you depending on the host computer's native locale. Note that for this switch, language and country and variant may be separated with a colon (not underscore), e.g. `ja:JP`.

There should be no problem starting the IDE with a different locale than you did the last time; every label you did not enter yourself should switch to the new locale. If you find a problem with localized strings "sticking" in the user directory, file a bug with details to reproduce.

The bundle debugging mode (see below) is a great way for localization teams, Quality Assurance, and developers to see exactly how the IDE is dealing with localization. Many otherwise subtle problems can be made apparent and easy to fix this way.

A complete I18N test plan should ideally check the result of starting the IDE in Turkish locale - since in Turkish capital `I` and lowercase `i` are not case variants (they also differ in an accent mark: `İ` and `ı` are valid), code which confuses locale-sensitive and -insensitive operations is likely to display errors in Turkish locale.

# I18N Features

Most of this document describes specific I18N features in NetBeans and how they are intended to be used.

## Bundle messages

All message strings should be contained in a `Bundle.properties` file (containing default message strings - i.e. in US English). This resource bundle is conventionally in the same directory (package) as the source files which use it; other source packages should have their own bundles. Localized (non-US-English) message strings should be inserted into a `Bundle_LOCALE.properties` file - this file must be placed in the same directory as the default bundle. For example, `Bundle.properties` might have:

    # A simple bundle key:
    LBL_some_text=Text in English for a button.
    # A nicely formatted one:
    # {0} - number of missing files
    EXC_missing_files=There {0,choice,1#is one file|1#are {0,number,integer} files} missing.

In `Bundle_cs.properties`, however, there may be:

    LBL_some_text=Text v \u010De\u0161tin\u011B na tla\u010D\u00EDtko.
    EXC_missing_files=Chyb\u00ED {0,choice,1#jeden soubor|2#dva soubory|3#t\u0159i \
            soubory|4#\u010Dty\u0159i soubory|4#{0,number,integer} soubor\u016F}.

Remember that all non-ASCII characters present in bundle files should be escaped in Unicode format (`\uXXXX`). The `native2ascii` utility present in the JDK can help with this.

Message strings are identified by keys and these are entered in source files. These keys are non-localized strings. The recommended way to use these in a source file is like this:

    import org.openide.util.NbBundle;
    // Looks in Bundle.properties for LBL_some_text key:
    String localized = NbBundle.getMessage(ThisClass.class, "LBL_some_text");
    // Same, but formats the text as well:
    throw new IOException(NbBundle.getMessage(ThisClass.class, "EXC_missing_files",
                                              new Integer(missingFiles)));

There is a script `nbbuild/misc/i18ncheck.pl` which can help check for I18N violations - unlocalized strings. You can run this script on source files just like a compiler: in fact, from the IDE if run as an external compiler its error messages should be hyperlinked. Standard constructions such as `NbBundle.getMessage` are recognized as code which retrieves localized text, so the bundle key itself does not trigger a warning. But if the bundle key is on a different line, or if in general you have a string literal in code which you know should never be localized, then end that line of source code with a comment like //NOI18N. This will indicate to the script that it should be silently passed over.

Also the I18N module includes support for finding I18N violations like this, and correcting them. There is a setting to use `NbBundle.getMessage` as the localization device, which is suited to module code.

### Bundle debugging

It is possible to start the IDE with the special command-line option `-J-Dorg.openide.util.NbBundle.DEBUG=true` which enables a special bundle debugging mode. In this mode, any string loaded from a bundle using the normal `NbBundle` automatically include a special numeric annotation and line number in parentheses. You can cross-reference the annotation to a indexed list, printed on the console, of bundle locations that strings are loaded from. (It only works when `NbBundle` is used, not plain `ResourceBundle`.) This has several purposes:

1. Any text visible in the IDE not referring to something codelike (such as a method name or file name) which lacks this annotation can be noticed as being unlocalized and therefore corrected.
2. Displayed text which is incorrect can be easily traced to the offending file and line and corrected.
3. IDE features which stop working with this mode turned on might be found to be erroneously over-localized, i.e. use of localized messages in places where a hardcoded string is in fact required.

Strings which are looked up in a localized hashtable are also annotated with the key used to find them. Normally this would apply to localizable strings taken from module manifests, such as the module name.

Currently localized files (such as HTML pages which may be selected by locale, or JavaHelp helpsets) are not annotated in any way.

### How to write good `*.properties` files

There are some rules for how to write good `*.properties` files for easy localization, starting with the most important.

1. Localizable messages should be stored in files named `Bundle.properties` by convention.
2. If it is necessary to mix localizable and nonlocalizable messages in one `Bundle.properties` (to create groups of messages, where some of these messages are localizable and part not, and it is unreasonable to split off the unlocalizable ones): notate each nonlocalizable message with the preceding line `#NOI18N`. For example:

       #NOI18N
       var.POSSIBLE_FILE_STATUSES.value="Up-to-date", "Locally Modified", "Locally Added"
       var.POSSIBLE_FILE_STATUSES_LOCALIZED.value="Up-to-date", "Locally Modified", \
               "Locally Added"

   In some cases it would be preferable to hardcode the unlocalizable messages in source to begin with.
3. If there is `*.properties` file which must have a name different from the default and there are some localizable strings in it, the strings should be individually notated with `#I18N`. For example:

       var.POSSIBLE_FILE_STATUSES.value="Up-to-date", "Locally Modified", "Locally Added"
       #I18N
       var.POSSIBLE_FILE_STATUSES_LOCALIZED.value="Up-to-date", "Locally Modified", \
               "Locally Added"

4. If (by the previous two cases) one message must be partially localized, the message should be annotated as `#PARTI18N` or `#PARTNOI18N` (above the message) indicating the exceptional part of the message. For example, in a properties file not named `Bundle.properties`:

       #PARTI18N The module name
       EXEC_CHECKOUT_MODULE=${RUN} ${QUOTE}${CVS_EXE}${QUOTE} ${USER_GLOBAL_PARAM} checkout \
             ${USER_PARAM(-N -R)} ${PROMPT_FOR[SELECTOR_MODULES](The module name)} ${NUR}

   Where there is more than one exceptional part, indicate them separated by commas, for example:

       #PARTI18N Directory to export into, Date, The module name
       EXEC_EXPORT_MODULE_DATE=${RUN} ${CD} ${PROMPT_FOR_DIR(Directory to export into)}&& \
             ${CVS_EXE} ${USER_GLOBAL_PARAM} export -D ${PROMPT_FOR_DATE_CVS(Date)} \
             ${USER_PARAM} ${PROMPT_FOR[SELECTOR_MODULES](The module name)} ${NUR}

   Analogously, for files named `Bundle.properties` with some nonlocalizable bits in some messages, use `#PARTNOI18N`.

The bundle debugging mode (above) recognizes these conventions and deals with them accordingly. For example, a message marked `#NOI18N` in a file `Bundle.properties` will not be annotated, though other messages in the same file will. Partial localization of messages is not supported; the message will not be annotated, and a warning will be printed on the IDE's console to remind you.

When changing the format of a bundle value - for example, adding a `{0}`-style parameter or removing one, or changing a pair of label and mnemonic to just a label to be set with `org.openide.awt.Mnemonics` - always create a fresh key. Otherwise it could happen that an older localization could supply a value which is not of the correct form, causing runtime errors. If you use a fresh key, the worst that can happen is that the base locale value will be used; the localizers will notice the change of key name when refreshing the localization and be able to confirm that they are using the correct new format.

## Other localized text (not in source code)

There are files named `README.html` and `LICENSE.txt` and localized versions should exist in the same directories: by convention, `README_LOCALE.html` and `LICENSE_LOCALE.txt`. Similarly for other miscellaneous text.

Note that while bundle files should always use Unicode escapes and not rely on the platform's encoding, HTML and XML files may specify the desired encoding for their bodies using standard metainformational tags, for example:

XML
  `<?xml version="1.0" encoding="UTF-8" ?>`
HTML
  `<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">`

Of course you may always use numeric character entity escapes instead.

For textual files not used at runtime (such as licenses), please use UTF-8.

## Templates

Templates can be localized too, if desired. Localized templates would have modified file contents (e.g. code comments in a different language). The simplest way to localize template content is to use the `nbresloc` URL protocol in the location of the content when specifying the template in the module's XML layer.

### Template descriptions

Most templates, especially frequently-used ones, should have template descriptions. These are small blocks of HTML to be displayed to a user in the New wizard giving an overview of what the template is for. Descriptions should be HTML resources within the main module JAR; the API Support module provides a convenient way of associating the description to the template before packing the template JAR.

All templates should have localized description files, and the template file attributes in the localized JAR should point to the localized descriptions; by convention, such descriptions should be named e.g. `SomeTemplateDescription_LOCALE.html`. (It is not necessary to localize the template's contents just to localize its description.) The URL used may be for example:

    nbresloc:/org/netbeans/modules/html/TemplateHelp.html

where this URL loads the appropriate locale variant at runtime. When doing this, the naming scheme for localized template descriptions becomes a requirement rather than a convention.

## Localized file names

The system filesystem is composed principally of files installed by XML layers in enabled modules, sometimes modified by overrides in the `system/` subdirectory of the user directory. Sometimes these files are presented to the user: templates under `Templates/`, for example, are displayed in the New dialog.

In some cases the contents of the file suffice to provide a pleasant localized name and icon to the user. For example, `*.instance` files used to supply `SystemAction`s need no special treatment: the action itself provides a display name and icon.

In most cases, however, you will give the file itself a simple, unlocalized name, and separately localize its display name. (Technically: you are affecting the display name but not the name of the data object's node delegate.) This is done with the file attribute `SystemFileSystem.localizingBundle`, which gives the abstract name of a resource bundle; in that bundle there should be a key named according to the resource path of the virtual (layer) file to be localized. Otherwise the raw file name is presented. Additionally, `SystemFileSystem.icon` can give a 16x16 icon to use instead of the default icon for that file type. (`SystemFileSystem.icon32` is rarely used but can supply a 32x32 icon.)

Here then is an example of a module adding a menu with a bookmark, and a template:

Localizing Bundle and Icon Example
File Path in JAR | Contents
`com/power/module/ resources/layer.xml` |
    <filesystem>
      <folder name="Menu">
        <folder name="PowerBuilder">
          <attr name="SystemFileSystem.localizingBundle"
                stringvalue="com.power.module.Bundle"/>
          <file name="com-power-home-page.url" url="home-page.url">
            <attr name="SystemFileSystem.localizingBundle"
                  stringvalue="com.power.module.Bundle"/>
            <attr name="SystemFileSystem.icon"
                  urlvalue="nbresloc:/com/power/module/resources/power.gif"/>
          </file>
        </folder>
      </folder>
      <folder name="Templates">
        <folder name="Power">
          <attr name="SystemFileSystem.localizingBundle"
                stringvalue="com.power.module.Bundle"/>
          <file name="thing.pwr" url="thing.pwr.template">
            <attr name="SystemFileSystem.localizingBundle"
                  stringvalue="com.power.module.Bundle"/>
            <attr name="SystemFileSystem.icon"
                  urlvalue="nbresloc:/com/power/module/resources/power.gif"/>
            <attr name="template" boolvalue="true"/>
            <attr name="templateWizardURL"
                  urlvalue="nbresloc:/com/power/module/resources/power.html"/>
          </file>
        </folder>
      </folder>
    </filesystem>
`com/power/module/ resources/home-page.url` |
    http://www.power.com/powerbuilder/
`com/power/module/ Bundle.properties` |
    # Mnemonic: Alt-W
    Menu/PowerBuilder=Po&wer Builder
    # Mnemonic: Alt-H
    Menu/PowerBuilder/com-power-home-page.url=Power Builder &Home Page
    Templates/Power=Power
    Templates/Power/thing.pwr=Blank Power Builder Widget
`com/power/module/ resources/power.gif` | a 16x16 color GIF icon
`com/power/module/ resources/power.html` |
    <html><body>
    Makes a blank Power Builder widget.
    Not really very exciting.
    </body></html>
`com/power/module/ resources/thing.pwr.template` | contents of the template

## Localizing layers

XML layers themselves can be localized in the expected way: `layer_LOCALE.xml` can contain overrides of `layer.xml`. (You can add files or folders, or mask files or folders using special `*_hidden` files, or just replace layer file contents with a variant.) Localizing layers in this way is however quite rare and not nice to potential translators. Usually there is a better way, such as using a single fixed layer entry and looking up localizable text in a separate `Bundle.properties`.

Branding layers (see below) is however reasonably common, for example to hide unwanted menu items in an application based on the NetBeans Platform.

## Images

Images which are localized should be saved as `imageName_LOCALE.gif` or `imageName_LOCALE.png`. As usual, the localized image should be stored alongside that in the default locale. The filenames of images should not be stored in `Bundle.properties` files, because such files should be used for translatable text only. To access localized images, you may use `Utilities.loadImage(path, true)`; or use URLs with the `nbresloc` protocol, which accesses a resource by path (searching all enabled modules) considering also localized variants. Note that automatic localization will not change the extension of the image file, so it is preferable to use PNG format consistently in place of GIF, and name the base resource with the `.png` extension.

## Help documentation

Help documentation should be localized according to the JavaHelp specification. The `HelpSet.hs` file should be localized as `HelpSet_LOCALE.hs` and stored in the same package as the default one. Additionally, each individual file referred to by the help set (especially HTML files) may be individually localized or not; URL links among the files can point to the "base" version while in fact a localized (or branded, see below) variant may be selected. Furthermore, cross-links between files contained in different physical locations (directories or JARs) are possible; the base URL for every document is considered to be a resource path rather than a physical path. (Offline browsing of such links will naturally not work as is.)

For example, using only a single helpset, ID map, and navigation view files, all pointing to simple relative URLs, you can create a file `moreinfo.html` as well as `moreinfo_f4j.html`; the map (`*.jhm`) file as well as other HTML files should simply point to e.g. `../moreinfo.html`. When browsing offline, of course only the base file will be visible. But when viewed in the IDE with branding set to e.g. `f4j_ce`, then only `moreinfo_f4j.html` will be displayed. This may be convenient when most of a helpset is constant and only a couple of files need to be branded or otherwise localized.

Note that XML files used by JavaHelp can specify a file encoding if localized text is required in these files (helpset title, etc.).

## Module names

Modules have several attributes which contain localizable text. They must be placed in a bundle file in the module, and this bundle must be referred to from the manifest, e.g.

    OpenIDE-Module-Localizing-Bundle: org/foo/mymodule/Bundle.properties

Here the file is automatically localizable, so e.g. `Bundle_ja.properties` would be used in Japanese locale.

The bundle may contain the keys `OpenIDE-Module-Name`, `OpenIDE-Module-Display-Category`, `OpenIDE-Module-Short-Description`, and `OpenIDE-Module-Long-Description`.

Also, the bundle debug mode (above) will display special annotations for localizable manifest-derived text.

# Physical Placement of Localized Resources

In the simplest case, localized resources may simply be placed directly inside a module's JAR, distinguished only by their locale suffixes in the resource path. For example, a single JAR might have both default and Japanese bundles:

`jar:file:/netbeans/ide5/modules/mymodule.jar!/com/me/mymod/Some.class`
  (code referring to `com.me.mymod.Bundle`)
`jar:file:/netbeans/ide5/modules/mymodule.jar!/com/me/mymod/Bundle.properties`
  `Some_key=Some value in US English`
`jar:file:/netbeans/ide5/modules/mymodule.jar!/com/me/mymod/Bundle_ja.properties`
  `Some_key=\u540c\u671f\u5316`

Though simple, this system may be inconvenient for both users and localizers: there is no way to quickly tell if you have resources for a given locale available; no way to quickly remove unwanted locales; updates to a single localization require patching the entire module; and so on. For these reasons, NetBeans supports splitting locale variants off from the main contents of a module. For example, the module above could be stored as:

`jar:file:/netbeans/ide5/modules/mymodule.jar!/com/me/mymod/Some.class`
  (code referring to `com.me.mymod.Bundle`)
`jar:file:/netbeans/ide5/modules/mymodule.jar!/com/me/mymod/Bundle.properties`
  `Some_key=Some value in US English`
`jar:file:/netbeans/ide5/modules/locale/mymodule_ja.jar!/com/me/mymod/Bundle_ja.properties`
  `Some_key=\u540c\u671f\u5316`

The rule here is that resources specific to a non-default locale can go into a separate JAR file, named according to the master JAR but with an appropriate locale suffix, and placed in a subdirectory `locale` below the position of the master JAR. (`/netbeans/ide5/lib/locale/` works too to localize libraries present in `/netbeans/ide5/lib/`, and similarly `/netbeans/ide5/modules/ext/locale/`, etc.) If you are using country variants or branding (see below), you can analogously split off multiple variants, such as `/netbeans/ide5/modules/locale/mymodule_f4j_ce_ja.jar`.

Remember that resources inside locale variants must still be individually marked with locale suffixes! The presence of appropriate locale variant JARs just tells the IDE to search these JARs in addition to the master JAR, but the lookup is still ultimately controlled by the resource names.

The NetBeans translation project normally creates these locale variants for modules included in the NetBeans distribution.

There is no need to include a manifest in a locale variant JAR file. Any manifest present will just be ignored. The name of the JAR alone suffices to identify it.

Another feature of locale variants is that you can have a default locale variant JAR. This just contains all localizable resources for the module, in the default locale. Analogously, it is stored in the `locale` subdirectory with no suffix. For example:

`jar:file:/netbeans/ide5/modules/mymodule.jar!/com/me/mymod/Some.class`
  (code referring to `com.me.mymod.Bundle`)
`jar:file:/netbeans/ide5/modules/locale/mymodule.jar!/com/me/mymod/Bundle.properties`
  `Some_key=Some value in US English`
`jar:file:/netbeans/ide5/modules/locale/mymodule_ja.jar!/com/me/mymod/Bundle_ja.properties`
  `Some_key=\u540c\u671f\u5316`

This makes no difference to the IDE, it will continue to load and run the JAR as before. But seeing all localizable resources split off this way may be helpful to localizers, as it presents a compact summary of what is and is not localizable in the module.

# Lists of files to be localized

In each module root directory there should be an `l10n.list` file. This file should be kept up to date by the module owner and consists of a list of file names which should be localized or translated (HTML, `*.properties`, GIF images and so on). There must be one line per file or pattern, a path in the repository including the name of the module. Each line must start with module name. A pattern can use:

- `**` matches zero or more path segments of the name (e.g. `**/*.class` matches all `.class` files/dirs in a directory tree)
- `*` matches zero or more characters of the name
- `?` matches one character of the name

Users of the Ant build tool will find the syntax familiar.

Example for `form` module:

    form/src/org/netbeans/modules/form/Bundle.properties
    form/src/org/netbeans/modules/form/actions/Bundle.properties
    form/src/org/netbeans/modules/form/resources/AWTForms.html
    form/src/org/netbeans/modules/form/resources/Application.html

An example using patterns:

    form/**/Bundle.properties
    form/src/org/netbeans/modules/form/resources/*.html

To more easily see which actual files which will be matched by a given pattern, just run:

    ant -f nbbuild/build.xml display-l10n-list-matches

and enter a module name when prompted. If run inside NetBeans, you can also click on each match to open the file.

# Branded Localization

There is a special feature of `NbBundle` that permits people making distributions of the IDE to conveniently "brand" it, i.e. selectively replace a few prominent pieces of messaging, images, and so on with specialized versions suited to their product or distribution. Since in practice branding-related changes tend to be structurally very similar to localization, the IDE permits developers to brand items just as they would localize them.

Within the IDE, there are public methods in `NbBundle` to examine and set the current "branding" (much like the default locale) - but there is no need to use these directly. Instead, decide on a branding token which will identify your brand. For example, Sun may use a token such as `f4jce` when creating its Forte for Java Community Edition (now Sun ONE Studio - a branded IDE based on NetBeans). Now you may create branded resources exactly as you would localized resources. The difference is that the search list for bundles and resources will include your branding suffix. (This suffix comes before any localization suffix, since it takes precedence over the localization suffix - though it is unlikely this would ever matter.) For example, in Japanese locale the search list for a bundle might look like this:

1. `/some/path/to/your/Bundle_f4jce_ja_JP.properties`
2. `/some/path/to/your/Bundle_f4jce_ja.properties`
3. `/some/path/to/your/Bundle_f4jce.properties`
4. `/some/path/to/your/Bundle_ja_JP.properties`
5. `/some/path/to/your/Bundle_ja.properties`
6. `/some/path/to/your/Bundle.properties`

As with all localizations, searches automatically fall back to simpler resources if the specific locale plus branding was not found; when using bundle files (with `NbBundle.getBundle` and similar calls), there may be multiple matching bundles, in which case any given key will be searched in all available bundles in order (so that you can override only the keys you need). This makes it convenient to brand the IDE by only adding files and not replacing them - just as with localization. For example, a given directory which supported both the `cs_CZ` locale and the `yoyodyne` branding might have files such as these:

`Bundle.properties`

      MSG_Welcome=Welcome to NetBeans!
      LBL_boring_feature=Choose a color for your text.

`Bundle_cs.properties`

      MSG_Welcome=Vítáme Vás v NetBeansu!
      LBL_boring_feature=Vyberte si barvu pro svůj text.

`Bundle_yoyodyne.properties`

      MSG_Welcome=Welcome to Yoyodyne HappyBuilder!

`Bundle_yoyodyne_cs.properties`

      MSG_Welcome=Vítáme Vás v Yoyodynské Stavebničce!

You may also provide a branding token separated into segments with underscores. In this case, progressively shorter substrings will be searched, just as for locales. For example, in plain locale with branding `f4j_ce`, you would get the search sequence:

1. `/some/path/to/your/Bundle_f4j_ce.properties`
2. `/some/path/to/your/Bundle_f4j.properties`
3. `/some/path/to/your/Bundle.properties`

This permits for example a single "brand" to encompass multiple "products", where some resources are product-specific and some apply across products within a brand, while minimizing the number of duplicated resources.

To run the IDE in a particular branding mode, use the `--branding` startup switch, e.g.:

    --branding yoyodyne

Since branding is generally used to provide product-specific overrides which ideally would live in a different location than the base NetBeans product, you are permitted to place branding JARs in parallel locations in other clusters. For example, if the NetBeans IDE lives in `/opt/netbeans` with clusters like `platform5` and `ide5`, and then `/opt/netbeans/ide5/modules/foo.jar` may be branded by `/opt/product1/modules/locale/foo_brand.jar` if you pass `--branding brand` and `/opt/product1` is a cluster included in the NetBeans-based product.

# Discussion

If you have comments or questions on this document, or wish to discuss localization in the IDE in general, please use the mailing list `nbdev@netbeans.org`. Discussions around UI strategies pertaining to localization may go to `nbui@netbeans.org`. For discussions on use of the Open APIs to support localization of code, please use `dev@openide.netbeans.org`.

@FOOTER@