An easy to use solution for multilanguage programs

[Jacobus]

Hello,
A quick update is needed.
I'm using Freak's code (first post) to create a multilingual program, and everything works perfectly with PB 6.21B9 x86, but the program crashes on x64. I'm not given a reason, just the message:

Code: Select all

The debugger executable closes unexpectedly.

The problem only occurs when an external language file needs to be loaded, but not when loading the main language, which is defined in the Datasection. Same problem with PB 6.12 or PB 6.20.
I've tried looking for where the problem might be in the LoadLanguage(Filename$ = "") procedure, but I can't see or understand why the debugger closes unexpectedly.

Code: Select all

If FileName$ <> ""
                                   ;  Debug FileName$  ; ok
    If OpenPreferences(FileName$) ;: Debug "fichier ouvert" ; ok
      For Group = 1 To NbLanguageGroups
        If LanguageGroups(Group)\GroupStart <= LanguageGroups(Group)\GroupEnd  ; sanity check.. check for empty groups / vérification de cohérence... vérification des groupes vides
          PreferenceGroup(LanguageGroups(Group)\Name$)
         
          For StringIndex = LanguageGroups(Group)\GroupStart To LanguageGroups(Group)\GroupEnd
            LanguageStrings(StringIndex) = ReadPreferenceString(LanguageNames(StringIndex), LanguageStrings(StringIndex))
          Next StringIndex
          
        EndIf
      Next Group
      ClosePreferences()   
      
      ProcedureReturn #True
    EndIf    

  EndIf

Has anyone encountered this problem before? Do you have any ideas or solutions to fix it?

[Axolotl]

Just a cautious attempt to help:
I had a brief look at the original code.
Maybe you use some characters > 256 in your language files....
Because with these lines of code this could be problematic:

Code: Select all

        If Asc(Left(LanguageNames(StringIndex), 1)) <> char
          char = Asc(Left(LanguageNames(StringIndex), 1))
          LanguageGroups(Group)\IndexTable[char] = StringIndex
        EndIf

[Jacobus]

Hello Axolotl,
Thanks for the feedback. Unfortunately, I've already tried reducing the size of the strings (I have three that are actually longer than 256 characters, but the indexes are much shorter) and it doesn't change anything, especially since it works fine with PB x86.
The problem, which I was able to isolate, occurs when loading the language file with PB x64. The strangest thing is that it worked, only once!!. Now the debugger always crashes unexpectedly. Hence my incomprehension...

[Edit]
I just ran several tests in a row, with the debugger disabled, then enabled, and in both cases, I noticed that the compilation completed normally, then didn't... I'm lost for words...
Configuration: Windows 11 Professional x64, 24H2.

[Axolotl]

Hi Jacobus,
it is not the length. As you can see at my example it is the first character which is limited to 256, with unicode and stuff the value of a character can be greater.

[Jacobus]

Okay , I'm a bit on edge right now, but no, I don't use any non-alphabet characters for my indexes, and the language file is saved in UTF-8 format with BOM, as Freak pointed out.
Example of language file contents:

Code: Select all

[Hyperlinks]
Accueil              = Home
MesApplications      = Home - My applications
Bureau               = My Desktop
Recents              = Recent files
Telechargements      = My downloads
MonPC                = My PC
OutilsWindows        = Windows Tools
Aide                 = Need help?
Apropos              = About...
MinimiserFen         = Minimize window
Minimiser            = Minimize
Reduire              = Minimize window to taskbar
Quitter              = Quit
WindowsExplorer      = Windows File Explorer
GestionTaches        = Taskmanager
AnalyseurPerf        = Performance Analyzer
MoniteurRessources   = Resource Monitor
PanneauConfig        = Control Panel
InfoVersion          = Version info
ObservateurEvent     = Event Observer
CentreMaintenance    = Maintenance Center
InfoSysteme          = System Information
ProprietesSysteme    = System Properties
GestionServices      = Windows Services Manager
ParamCompteUser      = User account settings
Programmes           = Programs, repair and uninstall
GestionDisquesDurs   = Hard Disk and Partition Manager
NettoyerDisquesDurs  = Clean disks of unnecessary items
CreerArchivesAutoEx  = Creating self-extracting archives
CommandeExecuter     = Run Command
ConsoleLigneComm     = Command line console
EditeurRegistre      = Registry Editor
PanneauControleGen   = General Windows Tools Control Panel

[HeX0R]

Does it crash with that example file also?
Because I used it now, altered the data section to fit that example and nothing crashed.
Guess your problem might be somewhere else.

[Jacobus]

Hi HeXOR, thanks for your tests.
It's indeed part of one of my language files. For my application, the file contains a little over 300 lines of the same type.
The worst part is that I'm almost certain it's just a bug that's causing the whole thing to crash. This only happens in 64-bit mode. I can't be far from the truth... but so far, all my attempts have failed.

[HeX0R]

I don't think there is much secret in this part of your code (those are nothing but language phrases...), why not paste the code from freak with the altered datasection and the whole language file here, so others can easily test it?
Or in case it is too large, upload it somewhere.
I also tested it with both x64 compilers.

[Jacobus]

I'll create a minimalist test code to see if the problem is related to the translation of the texts, and if so, I'll post it. Otherwise, it's related to an interaction with another part of my code, and it will be more difficult to detect without the full code.
My last debugging attempt revealed that the file loads correctly, and it's once loaded that the problem occurs when applying the translation.

[ChrisR]

I've also tested it with your language file contents extended to 300 entries. No worries either, it works fine here too, as expected.
I guess the related error is probably elsewhere, in interaction with your full code.

Otherwise, have you tried the GPI module and the double map Group()\Word() ?
It works just as well, I personally find it much simpler with the maps well adapted to the need

[Jacobus]

Hello ChrisR,
Yes, I did try GPI's method and Guimauve's, but it also crashes.
In fact, sometimes it works (perfectly) and sometimes it doesn't (unexpected termination). I get different errors depending on the program's mood, which is a bit too playful for my taste.
Obviously, it must be an interaction with some part of my code, and I can only see one solution: disassemble everything point by point to detect which part of the code is generating these errors. I suspect a memory problem; we'll see.
Thanks for your feedback; I'll come back and update you when I find the error.
And sorry for the clutter in this thread.

[Axolotl]

With the debugger you can try to set breakpoints around the suspicious codes.
Or you can do some LoggingToFile stuff.

[Jacobus]

With the debugger you can try to set breakpoints around the suspicious codes.
Or you can do some LoggingToFile stuff.

In fact, after various tests, it turns out that the problem occurs when the debugger is enabled. It systematically crashes unexpectedly, either at startup or at exit if the program was able to launch. I haven't been able to discover the reason. I used the OnError() lib instead of the debugger, but no error was returned. I tested on several versions of PB, on Windows 10 and 11, and the result is the same. Difficult to debug with debugger that crashes without saying why...

To remedy this, I modified Freak's method to create a multilingual program, and I succeeded by including each language in the DataSection. Thus, with automatic detection of the user's language, the program loads the correct translation at startup. This way, I no longer have any crashes in 64-bit.

Method used:

Code: Select all

;-===========================
;-LANGAGE FREAK ;Edited by Jacobus
;-===========================

Global NbLanguageGroups, NbLanguageStrings
Structure LanguageGroup
  Name$
  GroupStart.l
  GroupEnd.l
  IndexTable.l[256]
EndStructure

;Modified procedure:
;we no longer load an external file, but we directly indicate which language should be used.
Procedure LoadLanguage(FileLang$)

  ; do a quick count in the datasection first:
  NbLanguageGroups = 0
  NbLanguageStrings = 0
  ;selection of language data in the DataSection:
  Select FileLang$
    Case "French"  :  Restore French
    Case "English" :  Restore English
    Case "German"  :  Restore German
  EndSelect

  Repeat

    Read.s Name$
    Read.s String$

    Name$ = UCase(Name$)

    If Name$ = "_GROUP_"
      NbLanguageGroups + 1
    ElseIf Name$ = "_END_"
      Break
    Else
      NbLanguageStrings + 1
    EndIf
   
  ForEver

  Global Dim LanguageGroups.LanguageGroup(NbLanguageGroups)  ; all one based here
  Global Dim LanguageStrings.s(NbLanguageStrings)
  Global Dim LanguageNames.s(NbLanguageStrings)

  ; Now load the standard language In the user language data part:
  Group.i = 0
  StringIndex.i = 0  
  
  Select FileLang$
    Case "French"  :  Restore French
    Case "English" :  Restore English
    Case "German"  :  Restore German
  EndSelect

  Repeat

    Read.s Name$
    Read.s String$

    Name$ = UCase(Name$)

    If Name$ = "_GROUP_"
      LanguageGroups(Group)\GroupEnd   = StringIndex
      Group + 1

      LanguageGroups(Group)\Name$      = UCase(String$)
      LanguageGroups(Group)\GroupStart = StringIndex + 1
      For i = 0 To 255
        LanguageGroups(Group)\IndexTable[i] = 0
      Next i
      
    ElseIf Name$ = "_END_"
      Break

    Else
      StringIndex + 1
      LanguageNames(StringIndex)  = Name$ + Chr(1) + String$  ; keep name and string together for easier sorting 

    EndIf
   
  ForEver

  LanguageGroups(Group)\GroupEnd   = StringIndex ; set end for the last group! 
  
  ; Now do the sorting and the indexing for each group
  For Group = 1 To NbLanguageGroups
    If LanguageGroups(Group)\GroupStart <= LanguageGroups(Group)\GroupEnd  ; sanity check.. check for empty groups 
      
      SortArray(LanguageNames(), 0, LanguageGroups(Group)\GroupStart, LanguageGroups(Group)\GroupEnd)
 
      char = 0
      For StringIndex = LanguageGroups(Group)\GroupStart To LanguageGroups(Group)\GroupEnd
        LanguageStrings(StringIndex) = StringField(LanguageNames(StringIndex), 2, Chr(1)) ; splitt the value from the name
        LanguageNames(StringIndex)   = StringField(LanguageNames(StringIndex), 1, Chr(1))

        If Asc(Left(LanguageNames(StringIndex), 1)) <> char
          char = Asc(Left(LanguageNames(StringIndex), 1))
          LanguageGroups(Group)\IndexTable[char] = StringIndex
        EndIf
      Next StringIndex
      
    EndIf
  Next Group
  
  ProcedureReturn #True
EndProcedure

; No change in the language application procedure
Procedure.s Language(Group$, Name$)
  Static Group.i  ; for quicker access when using the same group more than once / pour un accès plus rapide lorsque vous utilisez le même groupe plusieurs fois
  Protected String$, StringIndex.i, Result.i

  Group$  = UCase(Group$)
  Name$   = UCase(Name$)
  String$ = "##### String not found! #####"  ; to help find bugs / pour aider à trouver des bugs

  If LanguageGroups(Group)\Name$ <> Group$  ; check if it is the same group as last time  / vérifier s'il s'agit du même groupe que la dernière fois
    For Group = 1 To NbLanguageGroups
      If Group$ = LanguageGroups(Group)\Name$
        Break
      EndIf
    Next Group

    If Group > NbLanguageGroups  ; check if group was found / vérifier si le groupe a été trouvé
      Group = 0
    EndIf
  EndIf
  
  If Group <> 0
    StringIndex = LanguageGroups(Group)\IndexTable[ Asc(Left(Name$, 1)) ]
    If StringIndex <> 0

      Repeat
        Result = CompareMemoryString(@Name$, @LanguageNames(StringIndex));,#PB_String_CaseSensitive,-1,#PB_UTF8)

        If Result = 0
          String$ = LanguageStrings(StringIndex)
          Break

        ElseIf Result = -1 ; string not found! / chaîne non trouvée !
          Break

        EndIf

        StringIndex + 1
      Until StringIndex > LanguageGroups(Group)\GroupEnd

    EndIf

  EndIf
  ;Debug "StringIndex = "+StringIndex + " : "+String$
  ProcedureReturn String$
EndProcedure

;We define what the user's language is:
Procedure.s DefineUserLanguage()
  
  Define Lang.s, UserIntLang, *Lang
  
  If OpenLibrary(0, "kernel32.dll")   
    *Lang = GetFunction(0, "GetUserDefaultUILanguage")   
    If *Lang
      UserIntLang = CallFunctionFast(*Lang)
    EndIf   
    CloseLibrary(0)
  EndIf
  
  Select UserIntLang
    Case 1031 : Lang = "German"
    Case 2057 : Lang = "English"
    Case 1036 : Lang = "French"
    Case 1049 : Lang = "Russian"
    Case 3082 : Lang = "Spanish"
    Default   : Lang = "French"
  EndSelect
  
  ProcedureReturn Lang 
  
EndProcedure

;-LOAD LANGUAGE
UserLang.s = DefineUserLanguage()
If UserLang = "French"
  LoadLanguage("French") ; default language
ElseIf UserLang = "English"
  LoadLanguage("English") 
ElseIf UserLang = "German"
  LoadLanguage("German")
EndIf

DataSection ;{
  French:
  ; ===================================================
  Data$ "_GROUP_",          "MenuTitle"
  ; =================================================== 
  Data$ "Fichier",            "Fichier"
  Data$ "Editer",             "Editer"
  
  ; ===================================================
  Data$ "_GROUP_",          "MenuItem"
  ; =================================================== 
  Data$ "Nouveau",            "Nouveau"
  Data$ "Ouvrir",             "Ouvrir..."
  Data$ "Sauvegarder",        "Sauvegarder"
  
  ; ===================================================
  Data$ "_END_",              ""
  ; ===================================================
  
  English:
  ; ===================================================
  Data$ "_GROUP_",          "MenuTitle"
  ; ===================================================  
  Data$ "Fichier",            "File"
  Data$ "Editer",             "Edit"
  
  ; ===================================================
  Data$ "_GROUP_",          "MenuItem"
  ; ===================================================  
  Data$ "Nouveau",            "New"
  Data$ "Ouvrir",             "Open..."
  Data$ "Sauvegarder",        "Save"
  
  ; ===================================================
  Data$ "_END_",              ""
  ; ===================================================
  
  German:
  ; ===================================================
  Data$ "_GROUP_",          "MenuTitle"
  ; =================================================== 
  Data$ "Fichier",            "Datei"
  Data$ "Editer",             "Bearbeiten"
  
  ; ===================================================
  Data$ "_GROUP_",          "MenuItem"
  ; ===================================================  
  Data$ "Nouveau",            "Neu"
  Data$ "Ouvrir",             "Offen..."
  Data$ "Sauvegarder",        "Zum Schutz"
  
  ; ===================================================
  Data$ "_END_",              ""
  ; =================================================== 
  ;}
EndDataSection

Debug Language("MenuTitle", "Fichier")
Debug Language("MenuItem", "Sauvegarder")
Debug Language("MenuItem", "Ouvrir")

I haven't had time to test other programs yet, which I will do to determine if this is due to a specific program or not. To be continued...