Working with Translations in Dynamics 365 Business Central

Intro

Languages: what an almighty headache. Computerphile have a great video that describes just how big the problem is: https://www.youtube.com/watch?v=0j74jcxSunY

Perhaps my perception is skewed by my ignorant native-English-speaker point of view. I haven’t grown up in a country where learning multiple languages and being able to switch between them is essential. Sure, I wish* I could speak more languages but mostly I can get by assuming other people will speak English.

*wish as in “finding a magic lamp” not as in “actually having the patience to put in the effort required to learn and practise”

However, if you are publishing apps into AppSource or any other setting where you plan on supporting different countries you are going to need to deal with translations at some point.

Overview

Visual Studio Code will create a .xlf (xliff) file containing all the literal strings that you have used in your application. They will mostly come from Caption, Tooltip and Label properties. This file will, therefore, contain your English (US) captions – assuming that you are coding in US English.

We need additional .xlf files for each translation that we want to support. The files should be in the format [language]-[COUNTRY].xlf e.g. en-GB.xlf, fr-FR.xlf, de-NL.xlf

Although xliff is a standard for software translations, after scouring the internet for hours I couldn’t find a tool that did what I wanted. It seems I’m not alone, judging by the number of people that have started to write their own tooling:

• NAB AL Tools (Visual Studio Code extension)
• IN Translate (Business Central app on AppSource)

(I haven’t tried these solutions myself, I’m just aware of them). So what do I want? At least the following:

• Somewhere to maintain a list of languages and countries that my app needs to support
• Creation of new .xlf files for each language/country combination
• Keeping the translation units (the strings present in the app that need translating) in sync between the master .xlf file and each translation file
• Support for submitting strings to machine translation and feeding the results back into each translation file

See also https://community.dynamics.com/business/b/businesscentraldevitpro/posts/translate-your-extension-automatically-with-azure-translator-text which describes a similar approach to ours in a Visual Studio Code extension.

My preference is to build some support for translations into our PowerShell module. The main reason is so that we can use the functions in our build process.

Translations to Maintain

I’ve written before about us having an environment.json file which holds settings about the repository which we use in the build. This seemed like a sensible place to also keep our list of translations. It looks like this:

{
  "translationSource": ".\\Translations\\Hello World.g.xlf",
  "translations": [
    {"country": "FR", "language": "fr"},
    {"country": "BE", "language": "nl"},
    {"country": "DE", "language": "de"},
    {"country": "GB", "language": "en"}
  ]
}

The translationSource key holds the path to the main .xlf file that is updated by the compiler and translations is an array of country/language pairs that are required.

Translator Text in Azure Cognitive Services

You’ve got some choice when it comes to online translation services. We use the Translator Text service that is part of Azure Cognitive Services. We’re already using a bunch of Azure services so it makes sense to keep them in one place. It has a REST API that we can call with the strings to translate and the language to translate them into. Perfect. But, first we’ll need an API key to authenticate with the service.

• Log in to https://azure.portal.com to manage your Azure resources
• Use to search bar to find “Cognitive Services”
• Click to Add
• In the Marketplace search for “Translator Text” and click Create
• Give the service a name
• Select an Azure subscription to link it to – you can either grab a free trial or create a paid subscription. I’ve created a Pay-As-You-Go subscription. You need credit card details but we’re going to use the free pricing tier for now anyway
• Select a pricing tier (see https://azure.microsoft.com/en-us/pricing/details/cognitive-services/translator-text-api/) or just select F0 for the free tier
• Select or create a new resource group to hold your new service
• Open your new resource from the list of Cognitive Services and click on Keys (left hand navigation menu)
• You can now use either of the two keys that you’ve got to call the service

PowerShell

We’ve got two key functions in our PowerShell module:

• Translate-App – this is the entry point to call other functions which:
  • Find the source .xlf file and the environment.json file
  • Create any new .xlf files that are required (by copying the source file and changing the target language)
  • Synchronise the translation units between the source and the target files – add any new strings that require translation and remove any strings that are no longer present in the source file
  • Identify strings that require translation and call the Translator Text service to translate them into the target language
  • Populate the target .xlf file with the translated strings
• Test-TranslationIsComplete – we use this as part of our build process to verify that
  • All of the required translation files exist
  • Each of those files has all the translation units that are present in the source .xlf file
  • It will throw an error if either of those things is false otherwise it will return true

This is the code (hosted here if you can’t see it: https://gist.github.com/jimmymcp/41bd8d3ac3fd6aa742089029fcd990fb)

A few notes about it:

• I’ve just lifted it from the PowerShell module so it won’t work as is
  • You’ll need to remove the Export-ModuleMember lines
  • Line 173 in Translate-App.ps1 makes a call to a function I haven’t given you to read the API key for the Translator Text service. The module creates a json config file with keys for various settings and this is one of them
• The free tier of the Translator Text service is throttled. You’ll probably hit the limit if you’ve got more than a few hundred strings to translate into several languages – you just need to wait for a few minutes and run the function again (or choose a paid tier)

	function Test-TranslationIsComplete {
	param (
	# the directory with the source code of the app to be translated
	[Parameter(Mandatory=$false)]
	[string]
	$SourceDir = (Get-Location),
	# whether to surpress the error
	[Parameter(Mandatory=$false)]
	[switch]
	$SurpressError
	)
	$EnvJson = ConvertFrom-Json (Get-Content (Join-Path $SourceDir 'environment.json') -Raw)
	if ($TranslationSource -eq '' -or $null -eq $TranslationSource) {
	$TranslationSource = $EnvJson.translationSource
	}
	if (!(Test-Path $TranslationSource)) {
	if ($SurpressError.IsPresent) {
	return $false
	}
	else {
	throw 'Could not find translation source file.'
	}
	}
	foreach ($Translation in $EnvJson.translations) {
	$TranslationPath = Join-Path 'Translations' ('{0}-{1}.xlf' -f $Translation.language, $Translation.country)
	if (!(Test-Path $TranslationPath)) {
	if ($SurpressError.IsPresent) {
	return $false
	}
	else {
	throw ('Translation file {0}-{1} is missing' -f $Translation.language, $Translation.country)
	}
	}
	else {
	Sync-TranslationUnits -SourcePath $TranslationSource -OutputPath (Join-Path (Get-Location) $TranslationPath) | Out-Null
	if ($null -ne (Get-StringsToTranslate -SourcePath $TranslationPath)) {
	if ($SurpressError.IsPresent) {
	return $false
	}
	else {
	throw ('Translation file {0}-{1} has missing translations' -f $Translation.language, $Translation.country)
	}
	}
	}
	}
	return $true
	}
	Export-ModuleMember -Function Test-TranslationIsComplete

view raw Test-TranslationIsComplete.ps1 hosted with ❤ by GitHub

	function Translate-App {
	Param(
	[Parameter(Mandatory=$false)]
	[string]$TranslationSource,
	[Parameter(Mandatory=$false)]
	[string]$SourceDir = (Get-Location)
	)
	$EnvJson = ConvertFrom-Json (Get-Content (Join-Path $SourceDir 'environment.json') -Raw)
	if ($TranslationSource -eq '' -or $null -eq $TranslationSource) {
	$TranslationSource = $EnvJson.translationSource
	}
	if (!(Test-Path $TranslationSource)) {
	throw 'Could not find translation source file.'
	}
	Write-Host "Using translation source file $TranslationSource"
	foreach ($Translation in $EnvJson.translations) {
	Write-Host ('Translating to {0}-{1}' -f $Translation.language, $Translation.country)
	Translate-XlfFile -SourcePath $TranslationSource -TargetCountry $Translation.country -TargetLanguage $Translation.language
	}
	}
	function Translate-XlfFile {
	Param(
	[Parameter(Mandatory=$true)]
	[string]$SourcePath,
	[Parameter(Mandatory=$false)]
	[string]$OutputPath,
	[Parameter(Mandatory=$false)]
	[string]$TargetLanguage,
	[Parameter(Mandatory=$false)]
	[string]$TargetCountry
	)
	if ($OutputPath -eq '' -or $null -eq $OutputPath) {
	$OutputPath = (Join-Path (Split-Path $SourcePath -Parent) ($TargetLanguage.ToLower() + "-" + $TargetCountry.ToUpper())) + ".xlf"
	}
	#create xlf file if it doesn't already exist
	if (!(Test-Path $OutputPath)) {
	cpi $SourcePath $OutputPath
	[xml]$OutputXml = Get-Content $OutputPath
	$TargetLanguageAtt = $OutputXml.CreateAttribute('target-language')
	$TargetLanguageAtt.Value = '{0}-{1}' -f $TargetLanguage.ToLower(), $TargetCountry.ToUpper()
	$OutputXml.xliff.file.Attributes.SetNamedItem($TargetLanguageAtt)
	$OutputXml.Save($OutputPath)
	}
	#add any translation units that are present in the source but not in the output
	Sync-TranslationUnits $SourcePath $OutputPath
	$StringsToTranslate = Get-StringsToTranslate -SourcePath $OutputPath
	if ($null -eq $StringsToTranslate) {
	Write-Host 'Already up to date'
	}
	while ($null -ne $StringsToTranslate) {
	$Strings = @()
	$StringsToTranslate | ForEach-Object {$Strings += $_.Source}
	$TranslatedStrings = Translate-Strings -Strings $Strings -TargetLanguage $TargetLanguage
	Write-TranslatedStrings -OutputPath $OutputPath -StringsToTranslate $StringsToTranslate -TranslatedStrings $TranslatedStrings
	$StringsToTranslate = Get-StringsToTranslate -SourcePath $OutputPath
	}
	}
	function Sync-TranslationUnits {
	Param(
	[Parameter(Mandatory=$true)]
	$SourcePath,
	[Parameter(Mandatory=$true)]
	$OutputPath
	)
	[bool]$SaveFile = $false
	[xml]$SourceXml = Get-Content $SourcePath
	[xml]$OutputXml = Get-Content $OutputPath
	[System.Xml.XmlNamespaceManager]$NSMgr = [System.Xml.XmlNameSpaceManager]::new($OutputXml.NameTable)
	$NSMgr.AddNamespace('x',$SourceXml.DocumentElement.NamespaceURI)
	#add missing sources to the output file
	foreach ($SourceTUnit in $SourceXml.SelectNodes('/x:xliff/x:file/x:body/x:group/x:trans-unit',$NSMgr)) {
	$OutputTUnit = $OutputXml.SelectSingleNode(("/x:xliff/x:file/x:body/x:group/x:trans-unit[@id='{0}']" -f $SourceTUnit.Attributes.GetNamedItem('id')."#text"),$NSMgr)
	if ($null -eq $OutputTUnit) {
	$OutputXml.xliff.file.body.group.AppendChild($OutputXml.ImportNode($SourceTUnit,$true))
	$SaveFile = $true
	}
	elseif ($OutputTUnit.source -ne $SourceTUnit.source) {
	$OutputTUnit.source = $SourceTUnit.source
	$OutputTUnit.RemoveChild($OutputTUnit.SelectSingleNode('./x:target',$NSMgr))
	$SaveFile = $true
	}
	}
	#remove orphaned sources from the output
	foreach ($OutputTUnit in $OutputXml.SelectNodes('/x:xliff/x:file/x:body/x:group/x:trans-unit',$NSMgr)) {
	$SourceTUnit = $SourceXml.SelectSingleNode(("/x:xliff/x:file/x:body/x:group/x:trans-unit[@id='{0}']" -f $OutputTUnit.Attributes.GetNamedItem('id')."#text"),$NSMgr)
	if ($null -eq $SourceTUnit) {
	$OutputXml.xliff.file.body.group.RemoveChild($OutputTUnit)
	$SaveFile = $true
	}
	}
	if ($SaveFile) {
	$OutputXml.Save($OutputPath)
	}
	}
	function Get-StringsToTranslate {
	Param(
	[Parameter(Mandatory=$true)]
	[string]$SourcePath,
	[Parameter(Mandatory=$false)]
	[int]$Top = 100
	)
	$StringsToTranslate = @()
	$ElementNo = 1
	[xml]$SourceXml = gc $SourcePath
	foreach ($TUnit in $SourceXml.xliff.file.body.group.'trans-unit') {
	if ($TUnit.target -eq $null) {
	$StringToTranslate = New-Object System.Object
	$StringToTranslate | Add-Member -MemberType NoteProperty -Name ID -Value $TUnit.id
	$StringToTranslate | Add-Member -MemberType NoteProperty -Name Source -Value $TUnit.source
	$StringToTranslate | Add-Member -MemberType NoteProperty -Name Target -Value ''
	$StringsToTranslate += $StringToTranslate
	$ElementNo++
	if ($ElementNo -gt $Top) {
	break
	}
	}
	}
	$StringsToTranslate
	}
	function Translate-Strings {
	Param(
	[Parameter(Mandatory=$true)]
	[string[]]$Strings,
	[Parameter(Mandatory=$true)]
	$TargetLanguage
	)
	#don't send English strings for translation
	if ($TargetLanguage -eq 'en') {
	$Translations = $Strings
	return $Translations
	}
	#convert input object into json request
	$Request = '['
	foreach ($String in $Strings) {
	$String = $String.Replace('\','\\')
	$String = $String.Replace('"','\"')
	$Request += '{Text:"' + $String + '"},'
	}
	$Request = $Request.Substring(0,$Request.Length – 1)
	$Request += ']'
	$Key = Get-TFSConfigKeyValue 'translationkey'
	$Headers = @{'Ocp-Apim-Subscription-Key'=$Key}
	$Headers.Add('Content-Type','application/json')
	$Response = Invoke-WebRequest ('https://api.cognitive.microsofttranslator.com/translate?api-version=3.0&to={0}' -f $TargetLanguage) -Headers $Headers -Method Post -Body $Request
	$Translations = @()
	(ConvertFrom-Json $Response.Content).translations | % {
	$Text = $_.text
	$Text = $Text.Replace('% 10',' %10')
	$Text = $Text.Replace('% 1',' %1')
	$Text = $Text.Replace('% 2',' %2')
	$Text = $Text.Replace('% 3',' %3')
	$Text = $Text.Replace('% 4',' %4')
	$Text = $Text.Replace('% 5',' %5')
	$Text = $Text.Replace('% 6',' %6')
	$Text = $Text.Replace('% 7',' %7')
	$Text = $Text.Replace('% 8',' %8')
	$Text = $Text.Replace('% 9',' %9')
	$Translations += $Text
	}
	$Translations
	}
	function Write-TranslatedStrings {
	Param(
	[Parameter(Mandatory=$true)]
	[string]$OutputPath,
	[Parameter(Mandatory=$true)]
	$StringsToTranslate,
	[Parameter(Mandatory=$true)]
	[string[]]$TranslatedStrings
	)
	[xml]$OutputXml = Get-Content $OutputPath
	$ElementNo = 0
	foreach ($StringToTranslate in $StringsToTranslate) {
	$TUnit = $OutputXml.xliff.file.body.group.'trans-unit' | ? ID -eq $StringToTranslate.ID
	$TargetNode = $OutputXml.CreateElement('target',$OutputXml.DocumentElement.NamespaceURI)
	$TargetNode.InnerText = $TranslatedStrings.Get($ElementNo)
	$SourceNode = $TUnit.FirstChild.NextSibling
	$TUnit.InsertAfter($TargetNode,$SourceNode)
	$ElementNo++
	}
	$OutputXml.Save($OutputPath)
	}
	Export-ModuleMember -Function Translate-XlfFile
	Export-ModuleMember -Function Translate-App
	Export-ModuleMember -Function Get-StringsToTranslate
	Export-ModuleMember -Function Sync-TranslationUnits

view raw Translate-App.ps1 hosted with ❤ by GitHub

Of course, being an English-only speaker I don’t have any way of checking how good these translations are but at least it gives a starting point for a human to verify.