Page 1 of 1

Lang() fast flexible XML language support for your programs!

Posted: Tue Mar 30, 2010 6:33 am
by Rescator
Sure, there's been other language loaders posted, including the way the PureBasic IDE does it if I recall correctly.

But the following source is the easiest way there is, super fast using array instead of list, and simple to use constants as id's.

Your program will always have a default built in language! (I suggest using English for that)

It automatically enumerates the constants for you!

It automatically uses the default text if a translation is missing!

It automatically uses the default text if a translation is outdated or for the wrong version of your program!

It is not possible for the wrong text to be used for the wrong thing due to id confusion!

You can choose the constants naming your self (some consistency is advised though obviously)!

The XML layout is designed to be easy to edit by hand and understand!

It's very easy to make a simple language editor if your program needs a lot of translations or have a lot of text to translate!

The source is hereby placed in the Public Domain have fun PureBasic community, you no longer have any excuse not to support other languages. ;)

There are some comments in the source, however you really just need to read the text between the ***************** comments!

Code: Select all

	Structure _language
	 org$
	 trans$
	EndStructure
	
	Global Dim Lang.s(0)
	
	Enumeration 0
	EndEnumeration
	
	Macro _lang(_constant,_text)
	 Enumeration #PB_Compiler_EnumerationValue
	  _constant
	 EndEnumeration
	 AddElement(_language())
	 _language()\org$=_text
	EndMacro
	; lang(#PB_Compiler_EnumerationValue)\id=_id
	; lang(#PB_Compiler_EnumerationValue)\text$=_text
	
	Procedure LoadLanguage(filepath$="",programtitle$="")
	 Protected NewList _language._language(),i.i=0,l.i,xmlfile.i,xmlok.i=#False
	 Protected *MainNode,*ItemNode,text$,org$
	
	 ClearList(_language()) ;needed in case this is a language reloading or language change.
	
	 ;***************** The following part is the only part you need to worry about *****************
	
	 ;Define and add default language text entries here, at least 1 must be added.
	 _lang(#Lang_Test,"Test")
	 _lang(#Lang_Another_Test,"Another Test")
	
	 ;To use the language text in the program just use for example:
	 ;Debug Lang(#Lang_Test) ;Very simple right? And much faster than a list as well.
	 ;The XML file should be made to match, uncomment the next few lines to generate a template for it:
	
	 ;Debug "<?xml version="+#DQUOTE$+"1.0"+#DQUOTE$+" encoding="+#DQUOTE$+"UTF-8"+#DQUOTE$+"?>"
	 ;Debug "" ;version/translator/url is just for reference to make translation easier and credit translators.
	 ;Debug "<language version="+#DQUOTE$+"1.0"+#DQUOTE$+" translator="+#DQUOTE$+"Roger Hågensen"+#DQUOTE$+" url="+#DQUOTE$+"http://EmSai.net/"+#DQUOTE$+">"
	 ;ForEach _language()
	 ; Debug " <lang org="+#DQUOTE$+_language()\org$+#DQUOTE$+" trans="+#DQUOTE$+#DQUOTE$+"/>" ;Translators must leave org alone and enter text in trans only.
	 ;Next
	 ;Debug "</language>"
	
	 ;Note!
	 ;It might be best to provide a simple editing tool to ensure character escaping and UTF8 is properly done when editing the file instead.
	 ;If a future version of the program changes it's org text and the translation file is not up-to-date,
	 ;then the default text is used instead of the translation, this ensures that there never is an outdated translation text shown in the program.
	
	 ;***************** That's it, the rest is just the magic code doing all the work for you. *****************
	
	 ;Size the lang() array per the number of entries defined above.
	 l=ListSize(_language())
	 If ArraySize(Lang())<>l ;No point in ReDim'ing if we did so previously.
	  ReDim lang(l-1)
	 EndIf
	
	 ;Load language xml and copy text entries that are supported (defined previously above)
	 If filepath$
		 If FileSize(filepath$)>-1 ;if it's missing it's no error, we'll just use the default text later below instead.
			 xmlfile=LoadXML(#PB_Any,filepath$)
				If xmlfile And IsXML(xmlfile)
				 If XMLStatus(xmlfile)=#PB_XML_Success
					 *MainNode=MainXMLNode(xmlfile)
					 If *MainNode
					  If XMLNodeType(*MainNode)=#PB_XML_Normal
						  If "language"=GetXMLNodeName(*MainNode)
				     *ItemNode=ChildXMLNode(*MainNode)
				     While *ItemNode
					 		  If XMLNodeType(*ItemNode)=#PB_XML_Normal
									  text$=Trim(GetXMLNodeName(*ItemNode))
									  If text$="text"
									   org$=Trim(GetXMLAttribute(*ItemNode,"org"))
									   text$=Trim(GetXMLAttribute(*ItemNode,"trans"))
									   If org$ And text$
									    ForEach _language()
									     If _language()\org$=org$
									      _language()\trans$=text$
									     EndIf
									    Next
									   EndIf
								   EndIf
							   EndIf
			       *ItemNode=NextXMLNode(*ItemNode)
				     Wend
			     EndIf
			    EndIf
			   EndIf
			  Else
			   text$="There was a error in the XML language file:"+#CR$
			   text$+XMLError(xmlfile)+#CR$
			   text$+"On line "+Str(XMLErrorLine(xmlfile))+" at position "+Str(XMLErrorPosition(xmlfile))+"."
			   If programtitle$
			    MessageRequester(programtitle$+" Error!",text$)
			   Else
			    MessageRequester("XML Error!",text$)
			   EndIf
			  EndIf
				 FreeXML(xmlfile)
				Else
			  MessageRequester(programtitle$+" Error!","Unable to open language file.")
			 EndIf
		 EndIf
	 EndIf
	
	 ;Copy translated text or default text from language() list to lang() array.
	 ForEach _language()
	  If _language()\trans$
	   lang(i)=_language()\trans$
	  Else
	   lang(i)=_language()\org$
	  EndIf
	  i+1
	 Next
	EndProcedure

Code: Select all

;Example (default language only)

LoadLanguage()
Debug Lang(#Lang_Test)

Re: Lang() fast flexible XML language support for your progr

Posted: Tue Mar 30, 2010 7:15 pm
by peterb
a little bit improved code inspired by user Rescator

Code: Select all


EnableExplicit

Global NewMap Lang.s()
   
Macro RegisterLang( _constant, _original )
  _constant         = _original
  Lang( _original ) = _original
EndMacro

Procedure LoadLanguage ( filename.s )
  
  Protected xml, main_node, index, error.s, node

  If FileSize ( filename ) > 0
    xml = LoadXML ( #PB_Any, filename )
    If xml
      If XMLStatus ( xml ) = #PB_XML_Success
        main_node = MainXMLNode ( xml )
        If main_node
          index = 1
          Repeat
            node = XMLNodeFromPath ( main_node, "/language/lang[" + Str ( index ) + "]" )
            If node
              Lang ( Trim ( GetXMLAttribute ( node, "org" ) ) ) = Trim ( GetXMLAttribute ( node, "trans" ) )
            EndIf
            index + 1
          Until node = 0
        EndIf
      Else
        error = "XML Error:         " + XMLError ( xml ) + #CRLF$
        error + "XML Error on line: " + Str ( XMLErrorLine     ( xml ) ) + #CRLF$
        error + "XML Error on pos:  " + Str ( XMLErrorPosition ( xml ) )
        Debug error
      EndIf
      FreeXML ( xml )
    Else
      Debug "Unable to load xml file"
    EndIf
  EndIf

EndProcedure

; --------------------------------------

Define xml_data.s

If CreateFile ( 0, "c:\lang.xml" )

  xml_data.s = "<?xml version='1.0' encoding='UTF-8'?>" + #CRLF$
  xml_data   + "<language version='1.0' translator='Petr Vavrin' url='http://'>" + #CRLF$
  xml_data   + "  <lang org='Test' trans='Pokus'/>" + #CRLF$
  xml_data   + "  <lang org='Another Test' trans='Dalsi test'/>" + #CRLF$
  xml_data   + "</language>" + #CRLF$
  
  WriteString ( 0, xml_data )
  
  CloseFile ( 0 )

Else
  Debug "Unable to create language file."

EndIf

RegisterLang ( #Lang_Test, "Test" )
RegisterLang ( #Lang_Another_Test, "Another Test" )

Debug "DEFAULT"

Debug Lang ( #Lang_Test )
Debug Lang ( #Lang_Another_Test )

Debug ""
Debug "LOADED"

LoadLanguage ( "c:\lang.xml" )

Debug Lang ( #Lang_Test )
Debug Lang ( #Lang_Another_Test )


Re: Lang() fast flexible XML language support for your progr

Posted: Tue Mar 30, 2010 10:37 pm
by Rescator
Pretty nice, but Maps are slower than arrays, and they use more memory, also I use constants rather than strings for id's so less exe size due to that as well.
Mine:

Code: Select all

; text$=Lang(#Lang_Test)
  MOV    ebp,dword [a_Lang]
  MOV    edx,dword [ebp]
  PUSH   dword [_PB_StringBasePosition]
  CALL  _SYS_CopyString@0
  PUSH   dword v_text$
  CALL  _SYS_AllocateString4@8
Yours:

Code: Select all

; text$=Lang ( #Lang_Test )
  PUSH   dword _S15
  PUSH   dword [m_Lang]
  CALL  _PB_GetMapElement@8
  MOV    ebp,eax
  MOV    edx,dword [ebp]
  PUSH   dword [_PB_StringBasePosition]
  CALL  _SYS_CopyString@0
  PUSH   dword v_text$
  CALL  _SYS_AllocateString4@8
I'm not saying yours is worse or anything, just that mine is faster and uses less memory and gives a smaller exe if there is a lot of text.
I haven't checked if using nodefrompath is any faster or not than stepping through then normally, anybody here know or tested?
My code is more clunky during the loading/setup than yours, but I guess that is a natural tradeoff.

If anybody has alternatives by all means abuse this thread :)

PS! Ideally PureBasic would have some native support for something similar to my first post, that would reduce the need for "tricky" code to get this to work like above. Especially the Enumeration is very messy, if I find a way to improve this (or if anyone else does) please post.

Re: Lang() fast flexible XML language support for your progr

Posted: Wed Mar 31, 2010 3:38 am
by Rescator
@peterb
I kinda liked your map idea, so I looked at it further and turned my original code into a Map based system like yours... But I took it a even a step further than you did!
I'm taking advantage of the fact that PureBasic seems to only store text once, even if it's present more than once in the source. (nice work Fred :) )

There are also 5 examples that should cover all uses and explains things a little better.

Code: Select all

Macro ADDLANG(text)
	LANG(text)=text
EndMacro

Global NewMap LANG.s()

Procedure.i LoadLanguage(filepath$="",programtitle$="") ;returns false if there was an xml error, true if no errors.
	Protected result.i=#True,xmlfile.i,*MainNode,*ItemNode,text$,org$
	
	;LANG(): text is special and must be deleted, these will be filled by the language loader later if available.
	DeleteMapElement(LANG(),"LANG(): VERSION")
	DeleteMapElement(LANG(),"LANG(): RTL")
	DeleteMapElement(LANG(),"LANG(): TRANSLATOR")
	DeleteMapElement(LANG(),"LANG(): URL")
	
	;Restore defaults in case this is a language reloading or language change.
	ResetMap(LANG())
	While NextMapElement(LANG())
		LANG()=MapKey(LANG())
	Wend
	
	;Load language xml and copy text entries that are supported (defined previously above)
	If filepath$
		If FileSize(filepath$)>-1 ;if it's missing it's no error, we'll just use the default text later below instead.
			xmlfile=LoadXML(#PB_Any,filepath$)
			If xmlfile
				If XMLStatus(xmlfile)=#PB_XML_Success
					*MainNode=MainXMLNode(xmlfile)
					If *MainNode
						If XMLNodeType(*MainNode)=#PB_XML_Normal
							If "language"=GetXMLNodeName(*MainNode)
								text$=GetXMLAttribute(*MainNode,"version")
								If text$
									LANG("LANG(): VERSION")=text$
								EndIf
								text$=LCase(GetXMLAttribute(*MainNode,"rtl"))
								If text$
									If (text$="true") Or (text$="1") Or (text$="yes")
										LANG("LANG(): RTL")="1"
									Else
										LANG("LANG(): RTL")="0"
									EndIf
								EndIf
								text$=GetXMLAttribute(*MainNode,"translator")
								If text$
									LANG("LANG(): TRANSLATOR")=text$
								EndIf
								text$=GetXMLAttribute(*MainNode,"url")
								If text$
									LANG("LANG(): URL")=text$
								EndIf
								*ItemNode=ChildXMLNode(*MainNode)
								While *ItemNode
									If XMLNodeType(*ItemNode)=#PB_XML_Normal
										text$=Trim(GetXMLNodeName(*ItemNode))
										If text$="text"
											org$=Trim(GetXMLAttribute(*ItemNode,"org"))
											text$=Trim(GetXMLAttribute(*ItemNode,"trans"))
											If org$ And text$
												If FindMapElement(LANG(),org$)
													LANG()=text$
												EndIf
											EndIf
										EndIf
									EndIf
									*ItemNode=NextXMLNode(*ItemNode)
								Wend
							EndIf
						EndIf
					EndIf
				Else
					result=#False
					;The following text is not using LANG() for a simple reason, if loading failed there would be no translation to use anyway!
					text$="There was a error in the XML language file:"+#CR$
					text$+XMLError(xmlfile)+#CR$
					text$+"On line "+Str(XMLErrorLine(xmlfile))+" at position "+Str(XMLErrorPosition(xmlfile))+"."
					If programtitle$
						MessageRequester(programtitle$+" Error!",text$)
					Else
						MessageRequester("XML Error!",text$)
					EndIf
				EndIf
				FreeXML(xmlfile)
			Else
				MessageRequester(programtitle$+" Error!","Unable to open language file.")
			EndIf
		EndIf
	EndIf
	
	ProcedureReturn result
EndProcedure

;*********** Examples ***********

;Example 1, how to define the default language, make sure you do not use ADDLANG() before the procedure and macro and global above, etc.

ADDLANG("This is a test!")
ADDLANG("This is also a test!")


;Example 2, creating a language template using the specified language above.
Define xmltemp.i,mainnodetemp.i,itemtemp.i
xmltemp=CreateXML(#PB_Any)
If xmltemp
	mainnodetemp=CreateXMLNode(RootXMLNode(xmltemp))
	If mainnodetemp
		SetXMLNodeName(mainnodetemp,"language")
		SetXMLAttribute(mainnodetemp,"version","1.0")
		SetXMLAttribute(mainnodetemp,"translator","Roger Hågensen")
		SetXMLAttribute(mainnodetemp,"url","http://EmSai.net/")
		SetXMLAttribute(mainnodetemp,"rtl","false")
		ResetMap(LANG())
		While NextMapElement(LANG())
			itemtemp=CreateXMLNode(mainnodetemp)
			If itemtemp
				SetXMLNodeName(itemtemp,"text")
				SetXMLAttribute(itemtemp,"org",LANG())
				SetXMLAttribute(itemtemp,"trans",LANG())
			EndIf
		Wend
	EndIf
	FormatXML(xmltemp,#PB_XML_ReFormat)
	SaveXML(xmltemp,"language.xml")
	FreeXML(xmltemp)
EndIf

;Note!
;It might be best to provide a simple editing tool to ensure character escaping and UTF8 is properly done when editing the file instead.
;Unfotunately if if you type the key wrong with LANG() you only get "" which ideally should have been "***ERROR***" which is more visible in a program.
;Do not worry too much about bloat as PureBasic only stores a string once, even if it's repeated multiple times, it uses pointers to the string,
;this is not as fast as using numeric constants and an array, but the convenience and smaller code balances this out.


;Example 3, loading the default langage.
;LoadLanguage("") ;Kinda pointless really, as to use the default language you do not even need to use LoadLanguage() at all.

;Example 4, load language from file (that we created in Example 2), any missing translation will instead use the default text.
LoadLanguage("language.xml")

text$=LANG("This is a test!")
Debug text$
Debug ""

text$=LANG("This is also a test!")
Debug text$
Debug ""

text$=LANG("This is a intentional mistake test!")
Debug text$
Debug ""

;Example 5, special LANG(): info.
;These are special, they get created by the loader, and only if the info is actually available in the language file.
Debug LANG("LANG(): VERSION") ;Version of translation, hopefully same as the programs language version.
Debug LANG("LANG(): TRANSLATOR") ;Translator name
Debug LANG("LANG(): URL") ;Full url to the translators support page.
Debug LANG("LANG(): RTL") ;If "1" then the language isa Right-To-Left language, if "0" then it's a "normal" language.
Debug ""