I just released grunt-jspot yesterday, a simple grunt task for jspot, an extraordinary gettext extractor tool for JavaScript.
jspot is like the missing piece I was looking for so long to do internationalization in JavaScript. I mean, real internationalization, using standard and stuff.
But talking about it make me realize internationalization is not well understood and seems like a very difficult beast to harness for JavaScript application.
I hope this tutorial could help a lot and make developers use internationalization everywhere without fear.
Big picture
What is internationalization?
I will just quote Wikipedia, because it sum it up very well:
Internationalization is the process of designing a software application so that it can potentially be adapted to various languages and regions without engineering changes. Localization is the process of adapting internationalized software for a specific region or language by adding locale-specific components and translating text.
So you will design your application in a way it can display
Hello world!
or
Salut tout le monde !
Internationalization is often see as i18n. Yeah, lazy developers that can not write a long word :p
What is the difference with localization?
It's more about adapting your format to regional specifications. Date format, currency, numbers are good example.
$9,999.99
or
99 999,99 $
Localization is often see as l10n. Lazy is good :)
What is gettext?
Again, thank you Wikipedia
gettext is an internationalization and localization (i18n) system commonly used for writing multilingual programs on Unix-like computer operating systems. The most commonly used implementation of gettext is GNU gettext, released by the GNU Project in 1995.
It is very common and standard so it's interesting to use it. Translators know how to deal with files it generates and it has nice features like pluralization.
Wow, why do I have to care about pluralization?
Because plurals do not follow the same rules for each language! For example, in English 0 (zero) is plural, whereas it is singular in French. Irish could have 4 plurals form. That's why good pluralization feature is very important.
Is gettext gonna translate for me?
Hell no! Gettext will expose a standard so all your application words and sentences could fit together in a dictionary. Then translators will create a catalog per language you wanna expose into your application and your application will read those catalogs with the gettext API.
Show me how it works in a software!
Ok, first, the big picture, please dont run away, it's easier than it's look like ;)
Concrete example in JavaScript
Using a gettext JavaScript implementation
Looking at the presentation above, you see you need a JavaScript implementation of gettext. I will use Jed. But you are free to use whatever you like. That's the beauty of this system, each tool at each step can be changed by another tool if you want.
Remember, the gettext implementation will need to load a catalog and will expose a gettext API.
// I will show you later how to really load a catalog
// + some more specific cases
var myFrenchCatalog = require('fr');
var i18n = new Jed(myFrenchCatalog);
var translation = i18n.gettext('Hello world!');
translation === 'Salut tout le monde !' // true
You are now using gettext!
var myFrenchCatalog = require('fr');
// no translation, for example english selected
// and your keys are already good english
// or you just wanna start playing with Jed without catalogs
var i18n = new Jed({});
var translation = i18n.ngettext('apple', 'apples', 0);
translation === 'apples' // true & plural
// fr translation, french selected
var i18n = new Jed(myFrenchCatalog);
var translation = i18n.ngettext('apple', 'apples', 0);
translation === 'pomme' // true & singular
You are now using pluralization!
Alright, you just have seen the right part of the schema above. You know how to start a gettext implementation and how to use the gettext API in your JavaScript application. (not all of it, but the essential :p)
Using a gettext extractor to create your dictionary
I will suppose you know how to use Grunt inside your project. We will use the jspot gettext extractor with grunt-jspot.
If you are not using Grunt, jspot can also be used in JavaScript or command line.
After the classic npm install grunt-jspot, in your project's Gruntfile, add a section named jspot to the data object passed into grunt.initConfig().
grunt.initConfig({
jspot: {
myCoolProject: {
options: {
keyword: 'i18n'
},
files: {
'locales': ['src/**/*.js']
}
}
}
});
grunt.loadNpmTasks('grunt-jspot');
Side note: I'm using the keyword option because the jspot default keyword is gettext, which is pretty standard. jspot is looking for keyword(), keyword.gettext(), keyword.ngettext(), ... calls. But I prefer to not named my Jed instance gettext because it can be confusing.
If your application contains a call to gettext like this
i18n.gettext('Hello World!');
Running grunt jspot generate you a file locales/messages.pot and it certainly looks like
msgid ""
msgstr ""
"POT-Creation-Date: 2014-08-09 05:32:+0000\n"
"Project-Id-Version: PACKAGE VERSION\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"Language: \n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
#: app.js:1
msgid "Hello World!"
msgstr ""
That's it. Your dictionary is ready. And yeah it's what we called a pot file. Please do not edit it manually. Like never! Its format is very important.
Generate catalogs and keeping them updated
You had seen how Jed exposes the gettext API and uses catalogs. You had seen how jspot creates the dictionary pot file. One more step, how to generates catalogs:
Two way, the easy way, or the... Nah. Both are easy, depending on what you are use to.
If you remember, and read Wikipedia, gettext has a well known implementation as GNU Gettext which comes with Linux. In fact, in most Linux distributions we have commands likes msginit, msgmerge, ... These are gettext commands. msginit is for creating a new catalog and msgmerge is for updating a catalog with all new entries from an updated dictionary.
But I will told you about the graphical way, which allow me to show you how to translate too :)
The tool I'm using to create and manage my catalogs is POEdit. It's free, open source, and runs on different OS.
Launch it, and go in file > New from pot file (or a thing like that). Select your dictionary previously created and fill in some information. Do not forget to fill in the correct plural form for you catalog.
There is also a catalog manager, which help you to: see your catalogs per project, update your catalogs when the dictionary was updated, see your catalogs status (green: translation = 100%).
Select a catalog to edit it and update translations. For a translation with singular & plural keys, you just have to fill the different possible forms the language can have without bothering about when to use it. Gettext will handle it for you.
Now you know about creating and updating catalogs but also translate your application!
Hey, one last thing, catalogs are .po or .mo files. Jed only read JSON!
True! I told you I will show you how to load catalogs in Jed
Last step! Convert your catalogs into JSON
This one is so easy. grunt-po2json.
And again, if you are not using Grunt, you can just use po2json directly or jspot which include po2json too.
After the classic npm install grunt-po2json, in your project's Gruntfile, add a section named po2json to the data object passed into grunt.initConfig().
grunt.initConfig({
po2json: {
myCoolProject: {
src: ["locales/*.po"],
dest: "locales/",
options: {
pretty: true,
format: 'jed'
}
}
}
});
grunt.loadNpmTasks('grunt-po2json');
The format: 'jed' option matters because it will wrap the generated object into jed options. With this you will just have to write new Jed(myCatalog) instead of new Jed({domain: 'myDomain', locale_data: MyCatalog})
About how to load the catalog, I will leave this to what you prefer. You can var frCatalog = require('./locales/fr'); if you are using CommonJS syntax, or define['json!./locales/fr.json', ...], frCatalog, ... if you are using AMD syntax. Or just use ajax if you are not really in modules.
That's it! I think if you are looking at the schema above, the circle is complete by now.
Bonus
I'm using Handlebars, can I extract gettext strings from it with jspot?
Sure you can!
Supposing i18n is your Jed instance. Add a helper like this:
Handlebars.registerHelper('i18n.gettext', function(context) {
// arguments = in template params + a handlebars hash
// this = in template context
return i18n.gettext.apply(i18n, Array.prototype.slice.call(arguments, 0, arguments.length - 1)));
});
Handlebars.registerHelper('i18n.ngettext', function() {
// arguments = in template params + a handlebars hash
// this = in template context
return i18n.ngettext.apply(i18n, Array.prototype.slice.call(arguments, 0, arguments.length - 1)));
});
Use it:
<div>{{i18n.gettext "I'm singular" }}</div>
<div>{{i18n.ngettext "cat" "cats" much }}</div>
And configure your grunt task to add Handlebars files. Those files must end with .hbs. (This will be configurable very soon).
grunt.initConfig({
jspot: {
myCoolProject: {
options: {
keyword: 'i18n'
},
files: {
'locales': ['src/**/*.js', 'src/**/*.hbs']
}
}
}
});
grunt.loadNpmTasks('grunt-jspot');
I'm using Underscorejs template, can I extract gettext string from it with jspot?
Not yet! But jspot can extract string from JavaScript very well. It also can have external extractors attached to its main extractor. As Underscore compiles templates in pure JavaScript, with source attached, a jspot-underscore-extractor will land very soon :)
I'm using XXX template engine, can I extract gettext string from it with jspot?
Not yet! Like for Underscore template, if it generates pure JavaScript, an external extractor should be easy to write. Contribution into grunt-jspot and jspot are welcome.
How can I handle numbers in plural sentences?
With the sprintf tool. An implementation exists for JavaScript and you have it directly with Jed: Jed.sprintf.
var numToasters = 3;
var translation = Jed.sprintf(i18n.ngettext('I have one toaster.', 'I have %d toasters.', numToasters), numToasters);
// I have 3 toasters
Translators will see I have %d toasters. as the plural key for this entry and will have to let the %d but can put it anywhere in the sentence. This is useful because some language do not have the same words order than English.
You can also use positional syntax if you have more than one number in your sentence and care about order in other languages. http://www.diveintojavascript.com/projects/javascript-sprintf
Moreover, you can do the same into Handlebars, thanks to Handlebars sub-expressions. Supposing you had registered a sprintf helper
<div>{{sprintf (i18n.ngettext "%d cat" "%d cats" much) much }}</div>