Replace all strings in sourcecode to constants for language translation?

[Caronte3D]

Hi
Anyone has already done a tool to generate different language versions of an aplication?

I mean something like:

Search throug all the source code for "" and replace it with auto-generated constants, so then we can translate every string (at the begining) without need to search throug all the soucecode manually.

I know would be better to do it from scratch, but my application is already done and I'm looking for an easy way.

Any idea?

[RASHAD]

You can use Multi Array for your strings
Use the Array contents instead of your strings
All you need in the start is the Array branch
I hope I understood you correctly

Code: Select all

dim title.s(30,3)
title(0,0) = "English"
title(0,1) = "German"
title(0,2 = "Arabic"
.
.
.

[normeus]

Use a regular expresion to find all text encoded in quotes:

Code: Select all

\"(.)?\"

I would use an editor that displays all regEx lines matched like notepad++, just to see what you are up against.

Norm.

[BarryG]

Don't forget that the size of gadgets will need to be increased when supporting multiple gadgets. For example, a ButtonGadget() that is just the right size for English will have cropped text for Asian languages, etc. I once read you need to increase them by at least 33% in width so all languages can fit (this was in my Visual Basic 5 manual, so I assume Microsoft knows best).

If your app is a small tool or utility that is easy to use, then a possible alternative is just to use icons instead of text for everything.

[Saboteur]

This is a little example I use with preferences command.
This way you can write all languages you need, I think it is easy to understand:
· write language files to a subfolder
· load texts in a map variable
· write the text assigned to a code

And be careful about BarryG warning, text with changes in other languages.
I'm not sure if this is what you are looking for.

Code: Select all

---------------------------
FILE: LANG.PB
---------------------------

Structure appsettings
  LANG.s                      ; App language
EndStructure

Global G_SETTINGS.appsettings       ; Structure with app settings
G_SETTINGS\LANG="es"                ; default language, change it
Global NewMap G_TEXTS.s()           ; Array with app texts, stored in "lang/lang-xx.txt"

Procedure OpenLang()
  If(OpenPreferences("lang/lang-"+G_SETTINGS\LANG+".txt"))
    ExaminePreferenceGroups()
    
    While NextPreferenceGroup()
      If(PreferenceGroupName()="AppTexts")
        ClearMap(G_TEXTS())
        ExaminePreferenceKeys()
        While NextPreferenceKey()
          G_TEXTS(PreferenceKeyName())=PreferenceKeyValue()
        Wend
      EndIf
    Wend
    ClosePreferences()
  EndIf
EndProcedure

Debug G_TEXTS("T001")
Debug G_TEXTS("T002")
Debug G_TEXTS("T003")
Debug G_TEXTS("T004")

---------------------------
FILE: lang/lang-en.txt
---------------------------

[AppTexts]
T001 = File
T002 = Open
T003 = Save
T004 = Exit

---------------------------
FILE: lang/lang-es.txt
---------------------------

[AppTexts]
T001 = Archivo
T002 = Abrir
T003 = Guardar
T004 = Salir

[RASHAD]

Hi Caronte3D
From your request I understood that you don't want to use any external file but you need everything included in your code so
#1 : Use Multi Array as my first post
#2 : Or include your strings in DataSection

Code: Select all

DataSection
english:
data.s "english",
englishend:

spanish:
data.s .......
spanishend:

EndDataSection

[Caronte3D]

Thanks to everyone for the ideas and code examples, but... I'm looking for a "tool" to do the dirty job:

1.- Scan the sourcecode for texts inside " "
2.- Create a constant for each string text
3.- Put all the constants at begining of the source code

Then I only need to replace the constants at sourcecode begining (before compile) for each language.

I know it's posible to create a tool to do the major part of work, my question is if someone have a tool like that already done

P.D:
Mi app is not small tool, is a big project with many, many controls and any other texts on screen.
I don't want to use variables for this, because is a threaded application and is already a pain of global variables and mutex.

[spikey]

Personally I'd recommend not going in this particular direction for implementing localisation. I know from past experience it's a huge pain in the butt for maintenance purposes as strings get added or modified over time. Either you end up with time consuming synchronisation problems between the base source and translation resources if a string gets inserted in the middle of the table; or you only ever add strings to the end of the table to avoid sync problems which means that you can end up with a muddled layout for the table which can be almost as bad. My resources were in separate files and could be managed separately to some extent, yours will be mixed up in the real source too which would make things worse I think! The bigger the project gets the worse the problem is...

The group/key = value implementation used by the PureBasic IDE itself is a much easier system to maintain in the long run even if it is more work to set up in the first place and uses slightly more memory in operation. See viewtopic.php?f=12&t=26729&p=191073 or 'Language.pb' if you've downloaded the IDE source code.

However if you really want to go that way, this is a section of code taken from a larger project that hunts for strings and tries to be intelligent about it, maybe you could modify it:-

Code: Select all

;- EXAMINESOURCE Structure
Structure EXAMINESOURCE
  LineNum.I
  Source.S
  StringText.S
  Action.I
  Remark.S
  ExistingGroup.S
  ExistingKey.S
  NewGroup.S
  NewKey.S
  GroupID.I
  KeyID.I
  StringID.I
EndStructure

;- Action types
Enumeration ActionTypes
  #ActionNoneDefined 
  #ActionIgnoreString
  #ActionIgnoreDuplicate
  #ActionExistingGroupExistingKey
  #ActionExistingGroupNewKey
  #ActionNewGroupNewKey
  #ActionReuseGroupNewKey
  #ActionReuseGroupReuseKey
EndEnumeration

Enumeration ErrorCode
  #ErrSuccess                  ; Success.
  #ErrCantOpenFile             ; A file could not be opened.   
  #ErrUnterminated             ; A file contains an unterminated string.
EndEnumeration

Procedure.I fSourceScanFile(aFile.S, List aList.EXAMINESOURCE())
  ; Scan a source file aFile for strings, building a list of candidates in aList.
  ; The list must use the EXAMINESOURCE structure.
  ; Returns an ErrorCode enum value.
  
  ; 1.00.00 spikey 21/10/2019 Created.
  
  Protected.B lbytUnterminated
  Protected.I lintFile, lintStart, lintEnd, lintLength, lintLineNum, lintResult
  Protected.S lstrLine, lstrString
  NewMap lmapStrings.I()
  
  ; Open the file.
  lintFile = ReadFile(#PB_Any, aFile)
  If lintFile = 0
    ProcedureReturn #ErrCantOpenFile
  EndIf
  
  ; Iterate the file.
  While Eof(lintFile) = 0
    
    ; Reinitialise.
    lintStart = 0
    lintEnd = 0
    lintLength = 0
    lstrString = #Empty$
    
    ; Get the next line from the file.
    lintLineNum + 1
    lstrLine = Trim(ReadString(lintFile))
    
    ; If there's no double quote, skip the line.
    If FindString(lstrLine, #DQUOTE$) = 0
      Continue
    EndIf
    
    ; If it's a comment, skip the line.
    If Mid(Trim(lstrLine), 1, 1) = ";"
      Continue
    EndIf
    
    ; Look for strings in the line.
    Repeat 
      
      ; Look for an opening quote.
      lintStart = FindString(lstrLine, #DQUOTE$, lintEnd + 1)
      
      If lintStart
        lintStart + 1
        lintEnd = FindString(lstrLine, #DQUOTE$, lintStart)
        
        ; Check for unterminated strings.
        If lintEnd = 0
          AddElement(aList())
          aList()\LineNum = lintLineNum
          aList()\StringText = "WARNING: Unterminated string."
          lbytUnterminated = #True
          Break
        EndIf
        
        ; Extract the string.
        lintLength = lintEnd - lintStart
        lstrString = Trim(Mid(lstrLine, lintStart, lintLength))
        
        ; Skip if empty.
        If lstrString = #Empty$
          Continue
        EndIf
        
        ; Check for redundancies.
        If Right(lstrString, 1) = "." Or Right(lstrString, 1) = ":" Or Right(lstrString, 1) = "?"
          lstrString = Left(lstrString, Len(lstrString) - 1)
        EndIf
        
        ; Add it to the list.
        AddElement(aList())
        aList()\LineNum = lintLineNum
        aList()\Source = lstrLine
        aList()\StringText = lstrString
        
        ; Check for accelerators.
        If Left(lstrString, 4) = "Ctrl" Or Left(lstrString, 3) = "Alt"
          aList()\Action = #ActionIgnoreString          
        EndIf
        
        ; Check for duplicates.
        If FindMapElement(lmapStrings(), lstrString)
          aList()\Action = #ActionIgnoreDuplicate
          aList()\Remark = "See line " + StrU(lmapStrings(lstrString)) + "."
          
        Else
          lmapStrings(lstrString) = lintLineNum
          
        EndIf
        
      EndIf
      
    Until lintStart = 0
    
  Wend
  
  ; Clear the map.
  ClearMap(lmapStrings())
  
  ; Close the file.
  CloseFile(lintFile)               
  
  ; Set the return value.
  If lbytUnterminated
    ProcedureReturn #ErrUnterminated
    
  Else
    ProcedureReturn #ErrSuccess
    
  EndIf
  
EndProcedure

;- Demo
NewList Results.EXAMINESOURCE()
fSourceScanFile("<a source name here>", Results())
ShowVariableViewer()
CallDebugger

[eck49]

Something else to watch out for, especially with single words...

Words which look the same but mean something different depending on the context.

English examples: Ball, which could be the round object used in football, one of a series of plays in cricket, or a social gathering for dancing.
Pound, which could be hitting a door to demand entry or the UK unit of currency.

[darius676]

Why not generate external files for each supported language?
The strings used are known at the start of the program? So why not check at program start, which language is used by system. Or let the user set the language at program start..
So you fill the strings with the data of the language file.

[Caronte3D]

Why not generate external files for each supported language?...

My application is not aimed to several languages from begining, so now I have near 60.000 lines of code and many strings spreaded over all the source.
I only want to find an easy solution to translate to another language.

[eck49]

However you do it, keep a good record of where every string is (or, better(?), how you derived the list).

If you find your translation needs amendment because it turns out to be unclear or inaccurate, you'll need to go easily to the right place to make the change.

[darius676]

My solution for this problem was:

https://mark-dowen.itch.io/the-quest-of ... arfes-king
https://www.youtube.com/watch?v=HnYGI_CoU5M

Code: Select all

Procedure.i E_GET_NPC_TEXT_RANDOM()
  ;german random default conversation textoutput, make the npcs a bit more  different
  
  Define _file.i=0
  Define _count.b=0
  Define _text_random_id.b=Random(e_NPC_maximum_text_alternative.b)
  Define _text_random_id_string.s=Str(_text_random_id.b)
  
; 
;   If ChangeCurrentElement(world_object(),*e_object_system_id2)=0
;     ProcedureReturn #False
;   EndIf
  
  
  
  e_npc_debugging_dummy_text_var.s=v_engine_base.s+world_object()\object_NPC_text_path+_text_random_id_string.s+e_npc_language_file_suffix.s
  e_npc_dummy_text_for_debugging.s=v_engine_base.s+world_object()\object_NPC_text_path
  _file.i=ReadFile(#PB_Any,e_npc_debugging_dummy_text_var.s)
  ProcedureReturn _file.i
  
  
  
  

EndProcedure






Procedure E_GET_NPC_TEXT(_mode.l)
  
  ;_mode.l=randomm language yes /no 
  
  Define _file.i=0
  Define _count.b=0
  
  If ChangeCurrentElement(world_object(),*e_object_system_id2)=0
  ProcedureReturn #False  
  EndIf
  
  
  Select e_npc_language.l
    Case #DE
      e_npc_language_file_suffix.s=".de"
    Case #EN
      e_npc_language_file_suffix.s=""
  EndSelect
  


  
    _count.b=E_NPC_SHOW_CORE_INFO_SYSTEM()
  
  
    If _mode.l=#NPC_USE_RANDOM_TEXT
      _file.i= E_GET_NPC_TEXT_RANDOM()
      If IsFile(_file.i)=0
     e_npc_dummy_text_for_debugging.s=v_engine_base.s+world_object()\object_NPC_text_path
    e_npc_debugging_dummy_text_var.s=v_engine_base.s+world_object()\object_NPC_text_path+e_npc_language_file_suffix.s
    _file.i=ReadFile(#PB_Any,  e_npc_debugging_dummy_text_var.s,#PB_File_SharedRead) 
      EndIf
      
    EndIf
    
    
    ;no random conversation : 

  
 
  If IsFile(_file.i)
    
    While Not Eof(_file.i)
      
      npc_text\text_text[_count.b]=ReadString(_file.i)
      If Len(npc_text\text_text[_count.b])>0  ;no empty lines (workaround for old text files, which used emty lines as buffer for lines displayed)
        npc_text\text_text[_count.b]= E_INTERPRETER_GET_KEYWORD(npc_text\text_text[_count.b]) 
        _count.b+1
      EndIf
      
    Wend
    
   
    
    CloseFile(_file.i)
    
     npc_text\text_text[_count.b]="[B]"
     npc_text\text_last_line=_count.b
    
  Else
    
    ;no german/alternative language pack found? we try To use english version
    _file.i=ReadFile(#PB_Any,v_engine_base.s+world_object()\object_NPC_text_path,#PB_File_SharedRead)
    
   
    If IsFile(_file.i)
      
      While Not Eof(_file.i)
        npc_text\text_text[_count.b]=ReadString(_file.i)
        
        If Len(npc_text\text_text[_count.b])>0  ;no empty lines (workaround for old text files, which used emty lines as buffer for lines displayed)
         npc_text\text_text[_count.b]= E_INTERPRETER_GET_KEYWORD(npc_text\text_text[_count.b])  ;here we inject some dynamic text, if a keyword is given
        _count.b+1
      EndIf
        
      Wend
      CloseFile(_file.i)
        npc_text\text_text[_count.b]="[B]"
        npc_text\text_last_line=_count.b
      
    EndIf
    
  EndIf



EndProcedure

https://mark-dowen.itch.io/the-quest-of ... arfes-king

https://www.youtube.com/watch?v=HnYGI_CoU5M

[AZJIO]

download
This code was written by me in the AutoIt3 programming language. I adapted it to translate programs to PureBasic.

1. Drag the pb file into the program window. You will get a list of strings.
2. Remove strings from the list that you do not want to translate. Click the "Reread List" button.
3. Make a translation on the same list. You can do it in another editor, and then paste it into the program window. The number of lines must match.
4. Click the "Run" button. The finished code will appear in the clipboard.

In the settings, you can specify the use of an array or replacement by location.
You can see an example of how it looks in my StopwatchTimer program

[Caronte3D]

download
This code was written by me in the AutoIt3 programming language...

Thanks!
I think your solution may be the one I'm waiting for, but...
I can't download that file, every web browser says it contains a virus.