Social Translation of a NetBeans Platform Application
We found that several more companies provide web applications to do social translation, here is a list of them:
Google Translate Toolkit https://developers.google.com/translator-toolkit
Far East languages support (Simplified Chinese and Japanese).
Java Properties support (we did not care about UTF-8 support).
An API callable from java (REST is fine) or a way to extract the translations.
Multiple users support.
Possibility to approve or rank translations.
Most of the companies offered a trial period from 10 to 30 days, some allowed to test it on a dummy project without registration, others are open-source and allow a local server installation.
The ones we like the most were transifex, crowin and lately pootle, but we considered also getlocalization because it provides a Java API and wordbee that has a lot of features but they are more individual translator oriented.
Transifex was my personal choice because it was easy to use, offered a REST API, allowed revisers to check the translation and supported Java Properties. We have also asked for support many times and were always promptly answered. transifex is written in Python and it was open-source since almost a year ago. It offers free hosting for open-source projects and several prices accordingly to the size of the project you want to translate. There is another NetBeans Platform based application called Gephi that uses Transifex for its translations; you can see how they work together here:
http://wiki.gephi.org/index.php/Localization . They also wrote some scripts in Python to automatize the work here: http://github.com/gephi/gephi/tree/master/translations .
My project Leader really liked crowdin, it has the same features as transifex, it looks a little better and seems more focused on quality. It is not open-source.
So at first, I created a 30 day trial account on Transifex and wrote some tools to upload our Java Property files and download the translated version. It was easy as I just have to run a Linux program called curl. Curl does an HTTP request and outputs the reply to a file or to the standard output; the reply is in JSON format and I used Google's Gson (code.google.com/p/google-gson/) to parse it. The API is pretty simple as you have just to create a class whose fields match the keys in the JSON data and the object is automatically populated. Everything went well until I tried to install their software locally on my Linux Box. I discovered that some APIs in some Python library had changed and so it wasn't working. I little knowledge of Python, so I did not know how to fix the problem and the opensource version is not supported or updated anymore.
So, we thought we had to pay for the support, and then we found out Pootle. Pootle is an open-source web application, it is also written in Python and based on Django. It is used by Mozilla and OpenOffice.org and it is still developed and supported. Some API to interact with it are in going to be released soon, but we can install it on our server and manage it with with the command line tool.
Another advantages of Pootle are the following: we can also change the strings in the English language, useful for our internal reviewers. Then, after you change a text in English the others translator tools did reset the translation in the other languages, in Pootle the translations are preserved.
installed Pootle as described here
. I had a bunch of trivial problems since our server had an old Linux
distribution and because the configuration file is different if you
are using django 1.3, but finally it worked.
After that, Pootle was up and running on our server, we set up a project with English as a main language and Italian, Portuguese and Chinese as target languages. We added Italian because the development team is based in Italy.
Pootle can rely on a database to handle the translations, but it can also read data from the file-system and synchronize with that. So I created a simple Java application that searched for all Bundle.properties in our NetBeans RCP application and copied into the en directory (en stands for English and it is the default language or template language) of our newly created project. Then this application launches
pootle –config=/etc/pootle.conf update_stores –keep --project=myproject
This command forces Pootle to read the content of the files into its database. Then my application launches
pootle –config=/etc/pootle.conf update_against_templates –keep --project=myproject
This command synchronizes the other languages to the English template. The documentation can be found here:
Our Bundle.properties files (the files that Java use for internationalization) are renamed as packagename.properties like org.scantamburlo.i18ntest.properties .
So now the user can start translating after having log in to our server.
After some text has been translated, we need to get the translated Properties files, so our tool launches this command:
pootle –config=/etc/pootle.conf sync_stores --overwrite
This copies the data held in the Pootle database into files. Every language has its own directory, so if we have a file called org.scantamburlo.i18ntest.properties inside the en directory, there would be a file with the same name inside the it, zh_CN and pt_BR directories .
I18N simplified with NetBeans
We have always followed the good programming practice to put all the strings that will be visualized to the user in a Properties files. The default language is English and the Properties file contains couples like MyPanel.MyLabel.text=Text on the label . If we want to provide an Italian translation we just have to create another file that contains the same keys with the Italian text like MyPanel.MyLabel.text=Testo sulla Label . By default in NetBeans this files are called Bundle.properties and there is one per package, the file that contains the translations is called Bundle_it_IT.properties for the Italian that is spoken in Italy (for your information, Italian is also spoken in Switzerland).
When we want to retrieve the value of the String we use the org.openide.util.NbBundle class, it has some useful static methods to retrieve the translated strings like:
NetBeans will check first the language that comes out from Locale.getDefault(), and if it is not found it will make some effort to find the closest one; for instance if a translation in Bundle_it_IT.properties is not found, it will search in Bundle_it.properties and finally in Bundle.properties.
Creating modules for translations
There were several possibility to distribute the translations to our users, but there is no automatic wizard or tool to automatize the process, so we had to create the scripts that does it.
The simplest way to provide an internationalization would be to put the translated Property files into the package of the original file. For instance we copy the translated files from the Pootle it directory (it stands for Italian) to the corresponding java package and rename the file Bundle_it_IT.properties . The same thing for other languages, if it was for Brazilian Portuguese the file would have been Bundle_pt_BR.properties. This method has a major drawback, every time we update our translations we had to publish the whole module.
Luckily we are on the NetBeans platform and it provides a way to distribute the translations on a different module as stated here:
For each module we could have created another module for translation but it would have been very bothersome, we have almost one hundred modules, and supporting two languages would have tripled that number.
One hint came from our project leader and from this post http://blogs.kiyut.com/tonny/2008/08/06/netbeans-platform-localization-as-plugins/#.Uf6td21GaHM : we can create a module for each language, Italian, Chinese and Portuguese and distribute it separately.
Assuming you have one ore more modules you want to translate and you want to build a module that contains the translation for a language:
First of all we created a module for language, for instance you can give it the codenamebase org.scantamburlo.translation.it or whatever you like.
Create this directories into it <module_directory>release\modules\locale .
For each module you want to translate create a jar file in this directory, the jar file must be named as the codenamebase of the module with dots replaced by the “-” character, an underscore, the code for the language and “.jar”. For instance to translate a module with codenamebase org.scantamburlo.i18ntest in Italian you must create a file called org-scantamburlo-i18ntest_it_IT.jar . This jar file must contain the same packages as the original module and wherever a Bundle.properties file is present in the original module a Bundle_it_IT.properties, of course it must contain the translate strings.
The modules must require restart, so right click on it, select “Properties” → “Packaging” and check “Needs Restart on Install”
If your module belongs to a suite of modules add this line to the project.properties nbm.target.cluster=<name_of_the_cluster>
If you want to test the translation and the default locale is different than Italian/Italy add this to the default_options property –locale it:IT , notice the colon instead of the underscore.
At this point after you install the module and restart the application, you will see the translation applied.
In our RCP as a first step we decided to translate just the text, but the Netbeans platform offers the possibility to add internationalized entries in the layer and also have internationalized images.
All the process of
sending the Property files to Pootle and the creations of the modules
containing the translations was automatized via Jenkins
will come into production the next month.
Thanks to my coworkers
Matteo Di Giovinazzo and Paolo Repele for helping with this project!
I have attached a working example.