PBOSL - A OpenSource Library-Collection for PureBasic

ts-soft · Post by **ts-soft** » Tue Mar 27, 2007 1:42 am

The fix for UTF-8 is in the recordset! I have tested with german "äöüß" and it
works for me. If there more bugs, please tell me (with small example)

Greetings
Thomas

rsts · Post by **rsts** » Tue Mar 27, 2007 2:49 am

ts-soft wrote:The fix for UTF-8 is in the recordset! I have tested with german "äöüß" and it
works for me. If there more bugs, please tell me (with small example)

Greetings
Thomas

Not exactly what you mean by 'in the recordset'. I'm strictly using the 'source' from the library which includes

If ValueAddress
*Recordset\sValue = PeekS(ValueAddress, #PB_Any, #PB_UTF8)
Else

this appears to be the source of the error I experience.

cheers

ts-soft · Post by **ts-soft** » Tue Mar 27, 2007 3:11 am

I have changed 2 functions to peeks with #PB_UTF8

SQLite3_GetRecordsetValueByName
SQLite3_GetRecordsetValueByIndex

Without this i can not read a string with äöü

According to SQLite dokumentation are stored all string in UTF-8, but I think,
there a few more are to be changed

rsts · Post by **rsts** » Tue Mar 27, 2007 3:25 am

Well, with the utf8 encoding a record containing X'A0' = ascii 160 can be stored in the database but is not properly retrieved, so from that aspect the utf8 is not good.

It may have solved one problem but it introduced another.

cheers

ts-soft · Post by **ts-soft** » Tue Mar 27, 2007 3:27 am

Sorry, i have edit my post

rsts · Post by **rsts** » Tue Mar 27, 2007 3:33 am

ts-soft wrote:Sorry, i have edit my post

I'm not sure which post you mean.

I'm certainly not aware of how library code is updated or controlled -

But we might want to let people know of possible problems with the source code in the library.

cheers

ts-soft · Post by **ts-soft** » Tue Mar 27, 2007 3:48 am

I have changed the source, please can you test it?
http://ts-soft.eu/dl/pbosl_sqlite3.pb

I'm not so good in sql and have no helper for this.

If no help, i restore to the old one, no problem.

Greetings
Thomas

rsts · Post by **rsts** » Tue Mar 27, 2007 4:23 am

Since that source includes the utf8 encoding at

line 658 *Recordset\sValue = PeekS(ValueAddress, #PB_Any, #PB_UTF8)

line 692 *Recordset\sValue = PeekS(ValueAddress, #PB_Any, #PB_UTF8)

It does not correctly return the datastream containing the ascii 160

So it does not work for me.

ts-soft · Post by **ts-soft** » Tue Mar 27, 2007 4:37 am

In UTF-8 only from 0 to 127 the same as in ASCII
SQLite3 uses UTF-8, not ASCII

Since version 3, sqlite support only UTF-8 and UTF-16
http://www.sqlite.org/version3.html

rsts · Post by **rsts** » Tue Mar 27, 2007 4:44 am

ts-soft wrote:In UTF-8 only from 0 to 127 the same as in ASCII
SQLite3 uses UTF-8, not ASCII

Maybe so, but if I change those lines to #PB_Ascii it works fine. I'm using the standard sqlite3.dll. The character I have problems is not 0-127 it's 160 or x'A0'

I'll change it maintain my own copy of the source. If it works all right for the rest of the world, fine. It CAN cause problems though. It IS NOT correct for all data. People should be aware of that.

cheers

as a test download http://htconsulting.com/pb_sqlerr.db

rename it to sqlerr.txt and open it in notepad.

Copy/paste it into a routine (I use and editor gadget) that will load it into a sqlite3 database. You will be able to 'see' the entire record in the database using a hex editor on the database. You can test the length of the loaded string as 400 characters.

Then read it from the database using those routines. It's all in the database but you'll only get 180 character back out.

Change those lines to #PB_ascii, or use kiffi's original code and you'll get all 400 characters out.

That seems like some sort of bug to me. And it's not in sqlite3. It's in the procedure used to get the data out of the database.

cheers

ts-soft · Post by **ts-soft** » Tue Mar 27, 2007 4:55 am

I have create some tables with datas with "SQLite Database Browser.exe"
Since i have changed the code to UTF-8 i have the same results in both.

So i hope my version is correct

rsts · Post by **rsts** » Tue Mar 27, 2007 5:02 am

It may be correct in some instances, but is incorrect in others.

In my opinion people should be made aware of the possible failure associated with the code.

cheers

By the way, the Name on the comment at the beginning of the source is
; ----------------------
; SQLite3
; by Peter Tuebben (aka Kiffi)
; ----------------------

Is it his source or your's? Is there some process for maintaining the library?

cheers

ts-soft · Post by **ts-soft** » Sun Apr 08, 2007 2:09 am

New Lib: WindowSubclass

four functions that make creating subclasses easier for all Windows versions

The lib is written in C by edel. The source based on the code by
Alexandre Julliard, from the wine-project. (LGPL)

This lib run in ansi-, unicode and threadsafe (not across threads) mode without problems, i hope

ts-soft · Post by **ts-soft** » Sun Apr 08, 2007 2:12 am

Small example:

Code: Select all

Macro HiWord(long)
  (long >> 16) & $FFFF
EndMacro

Macro LoWord(long)
  long & $FFFF
EndMacro

Structure Event
  event.l
  subevent1.l
  subevent2.l
  Proc.l
  dwData.l
EndStructure

Enumeration #WM_USER + 1000
  #OnClick
  #OnFokus
  #OnMouseMove
EndEnumeration

Procedure ButtonEventCallback(hwnd,uMsg,wParam,lParam,uId,*dwData.Event)
  
  If *dwData\event = #WM_COMMAND And lParam = uId
    If HiWord(wParam) = *dwData\subevent1
      CallFunctionFast(*dwData\Proc,hwnd,wParam,lParam)
    EndIf
  EndIf
  
  If *dwData\event = #WM_MOUSEMOVE And uMsg = #WM_MOUSEMOVE
    CallFunctionFast(*dwData\Proc,hwnd,wParam,lParam)
  EndIf
  
  ProcedureReturn DefSubclassProc(hwnd, uMsg , wParam, lParam)
EndProcedure

Procedure SetEvent(hwnd,ev,Proc)
  Protected Buffer.s
  Protected *evh.Event
  
  id = ev
  
  Select ev
    Case #OnClick
      Buffer = Space(128)
      GetClassName_(hwnd,Buffer,128)
      If LCase(Buffer) = "button"
        id        = hwnd
        hwnd      = GetParent_(hwnd)
        event     = #WM_COMMAND
        subevent1 = #BN_CLICKED
        evhp      = @ButtonEventCallback()
      EndIf
    Case #OnFokus
      Buffer = Space(128)
      GetClassName_(hwnd,Buffer,128)
      If LCase(Buffer) = "button"
        id        = hwnd
        style = GetWindowLong_(hwnd,#GWL_STYLE)
        SetWindowLong_(hwnd,#GWL_STYLE,style|$4000) ; #BS_NOTIFY
        hwnd      = GetParent_(hwnd)
        event     = #WM_COMMAND
        subevent1 = #BN_SETFOCUS
        evhp      = @ButtonEventCallback()
      EndIf
    Case #OnMouseMove
      event     = #WM_MOUSEMOVE
      evhp      = @ButtonEventCallback()
  EndSelect
  
  GetWindowSubclass(hwnd,evhp,id,@*evh)
  
  If Not *evh
    *evh = AllocateMemory(SizeOf(Event))
  EndIf
  
  *evh\event     = event
  *evh\subevent1 = subevent1
  *evh\Proc      = Proc
  
  ProcedureReturn SetWindowSubclass(hwnd,evhp,id,*evh)
EndProcedure

Procedure OnClick_button1(hwnd,wParam,lParam)
  Debug "OnClick_button1"
EndProcedure

Procedure OnClick_button2(hwnd,wParam,lParam)
  Debug "OnClick_button2"
EndProcedure

Procedure MouseMove_button2(hwnd,wParam,lParam)
  Debug "MouseMove_button2"
EndProcedure

hwnd = OpenWindow(0,0,0,200,200,"",#WS_OVERLAPPEDWINDOW|#PB_Window_ScreenCentered)

CreateGadgetList(hwnd)
button0 = ButtonGadget(0, 10, 10, 80, 30, "bla")
button1 = ButtonGadget(1, 10, 40, 80, 30, "bla")

SetEvent(button0,#OnClick,@OnClick_button1())
SetEvent(button1,#OnClick,@OnClick_button2())
SetEvent(button1,#OnMouseMove,@MouseMove_button2())

Repeat : Until WaitWindowEvent() = 16

Padde · Post by **Padde** » Wed Apr 11, 2007 3:30 am

I found a bug or lets say an unwanted limitation in the MSXML3 Lib.

It occurs while trying to load an XML document with the MSXML_LoadXML
procedure with large strings (somewhere between 16000 and 17000 bytes).
MSXML_LoadXML simply fail to load and return a 0.
Can't remember the exact problematic string size because i already fixed the
problem (doh!)

The problem is located in the Ansi2Uni procedure.. i replaced the old one
with the one i found in the Droopy lib and the problem was solved.

Code: Select all

Procedure.l    MSXML3_Ansi2Uni(string.s)
  *out = AllocateMemory(Len(string)*4) 
  MultiByteToWideChar_(#CP_ACP, 0, string, -1, *out, Len(string))  
  ProcedureReturn *out
EndProcedure

Don't know if there are any side-effects due to this little replacement.
As far as i can tell everything is still working.

Sorry if this issue was already known.. i was to lazy to read the hole
thread

Greetings
Padde