Hunspell

Perkin · **Posted:** Fri Sep 24, 2010 9:48 am

I've recently been adding an old spellchecking dll (from purearea) to a project.
I've discovered it has a few missing abilities for what I need.

I would like to ask whether anybody has got Hunspell working with PB
http://hunspell.sourceforge.net/

And if they have, could they show me how, or release some code/lib to be able to use it with PB
Thanks

rsts · **Posted:** Fri Sep 24, 2010 1:47 pm

I use hunspell for spell checking in PB.

I've basically modified freak's great aspell checker http://www.purebasic.fr/english/viewtopic.php?f=12&t=20008&hilit=hunspell to use hunspell, however my code is slightly less coherent than freak's and does not lend itself to publishing as a finished method, although I am looking into it.

If you have some specific questions I may be able to help. The only real problem in converting freak's code was exposing the hunspell functions. This should get you started, or at least provide an avenue for additional dialog.

The following code snippet will not stand alone and can probably be improved, but in combination with freak's routine, should provide a basis for understanding how to use hunspell. If you have additional questions, feel free to ask.

Code:

Global *hunspellInit,hunSpellDict, *hunspell_speller_check,hunspell_speller_suggest, delete_hunspell_speller, spellInstalled, spellExe$, hunspelllang.s
Procedure addWordForSession(word$)
    CallCFunctionFast(hunSpellAdd,hunSpellDict, @word$)
EndProcedure
Procedure CheckHSpell()
  Define lfh, hunspelldictPath.S,hunspelldictPatha.S

  hunspellDLL_Filename.S = (GetPathPart(ProgramFilename()))+"hunspelldll.dll" ; location of hunspelldll.dll
 
  hunspell_dict.S= GetPathPart(spellExe$) ; locate the appropriate dictionary
  
  hunspellDLL = OpenLibrary(#PB_Any, hunspellDLL_Filename)

  If hunspellDLL  ; we have a dll - check for a dict 
    SpellInstalled=#True

  Else
    debug "no hunspell dll"
    SpellInstalled = #False
    ProcedureReturn #False
  EndIf
  If hunspell_dict
   
  Else
    ProcedureReturn #False
  EndIf
    hunspellLang=parseFolder(hunspell_dict) ; parsefolder merely obtains the appropriate .dic file
   

    *hunspellInit= GetFunction(hunspellDLL, "hunspell_initialize") 
    hunspelldictPatha=hunspell_dict + hunspellLang+".aff"
    hunspelldictPath=hunspell_dict + hunspellLang+".dic"
    hunSpellDict= CallCFunctionFast(*hunspellInit,@hunspelldictPatha, @hunspelldictPath)
    
  If hunSpellDict
     *hunspell_speller_check= GetFunction(hunspellDLL, "hunspell_spell") 
     hunSpellAdd= GetFunction(hunspellDLL, "hunspell_put_word")
     hunspell_speller_suggest= GetFunction(hunspellDLL, "hunspell_suggest") 
     delete_hunspell_speller= GetFunction(hunspellDLL, "hunspell_suggest_free")
     lfh = ReadFile(#PB_Any, hunspell_dict +hunspellLang+".dic")
      
      If lfh
        SpellInstalled=#True
         
        spellExe$=hunspell_dict +hunspellLang+".dic"
      EndIf
      
      ;add words from personal dictionary
      lfh = ReadFile(#PB_Any, hunspell_dict +hunspellLang+".dat")
      If lfh
        While Eof(lfh) = 0           ; loop as long the 'end of file' isn't reached
          addWordForSession(ReadString(lfh))      ; add words from private dictionary
        Wend
        CloseFile(lfh)          
      EndIf
   Else ; no hunspell dictionary
      If SpellInstalled = #False
        MessageRequester(PG_Name$, "Unable to locate a proper spelling dictionary. Spell checking disabled.")
      EndIf
   EndIf

  
EndProcedure

cheers

Perkin · **Posted:** Fri Sep 24, 2010 4:46 pm

rsts, I've neary got a good working hunspell, using the older hunspelldll.dll, which your code seems to be using (v1.2.12). I'm also looking into using the newer NHunspeel (v0.9.6.0) which uses Hunspellx86.dll, and its associated bits.

Can you add you own words and have them stored?
(This is one of the major things I need, as well as storing hyphened words, which flag as being correct.)

At moment, if I add a word, it only stays while that session is in effect, quit out and added words are no longer there.

Edit:

How do you do the suggest words?

rsts · **Posted:** Fri Sep 24, 2010 6:01 pm

Perkin wrote:

Can you add you own words and have them stored?
(This is one of the major things I need, as well as storing hyphened words, which flag as being correct.)

At moment, if I add a word, it only stays while that session is in effect, quit out and added words are no longer there.

Edit:

How do you do the suggest words?

Yes, words you add are for session only, so I create an en_us.dat file (or whatever language/dictionary they're using) to which I add any words the users wants added permanently. These I add each hunspell session.

See the code from my previous post -

Code:

;add words from personal dictionary
      lfh = ReadFile(#PB_Any, hunspell_dict +hunspellLang+".dat")
      If lfh
        While Eof(lfh) = 0           ; loop as long the 'end of file' isn't reached
          addWordForSession(ReadString(lfh))      ; add words from private dictionary
        Wend
        CloseFile(lfh)         
      EndIf

For suggestions, I have the user rightclick the misspelled word and I popup a menu of suggestions which also includes options to allow the user to add the word for the session or to their personal dictionary.

Perkin · **Posted:** Fri Sep 24, 2010 6:36 pm

rsts, how do you actually get the list of suggested words.
(thats one of probs I'm having at moment).
I can get the number of suggested words, but can't see how to retrieve the actual words themselves.

Edit ---
If you added a hyphened word to you user dictionary, would that word be recognised if checked.

e.g. - add 'klik-klak' to user-dictionary, and then check that word.
(one of the other spell checkers, wouldn't see this as correct, as the hyphen caused problems)
(but if you added 'klik' and 'klak', it would recognise the full word, but also the seperate halves)

Edit - this bit solved - answer is yes it does recognise words with hypen in. YEAH!

rsts · **Posted:** Fri Sep 24, 2010 7:15 pm

For suggestions - see the code above for hunspell_speller_suggest

Code:

Suggestions = CallCFunctionFast(hunspell_speller_suggest,hunSpellDict, @word$ ,*elements) 
   
    If Suggestions 
      For i=0 To Suggestions-1;  Repeat 
        sa.S(i) = PeekS(PeekI(PeekI(*elements)+(i*SizeOf(*elements))))
      Next

word$ is the word in question.

Adding a word to the dictionary is via your own routine. I do not allow/accept hyphenated words so I'm unsure as to how/if they would be handled. You'd have to try it.

cheers

Perkin · **Posted:** Fri Sep 24, 2010 7:44 pm

I nearly had that, I had only one PeekI in the PeekS.

Thanks, just going to try that out now.

Edit: I'm getting 'Invalid memory access' on Suggestions = CallCFunctionFast line,
I think it's just *elements, is that pointing to anything?

So I suppose saving your own words to the user dictionary(for persistance), is just a simple, OpenFile, Fileseek (to end), and WriteStringN, to the user-dictionary.

I found that a hyphenated word is flagged as correctly spelled. (which I'm happy about)

rsts · **Posted:** Fri Sep 24, 2010 8:20 pm

Code:

*elements=AllocateMemory(1024 )

adding words

Code:

Procedure addWordForSession(word$)
    CallCFunctionFast(hunSpellAdd,hunSpellDict, @word$)
EndProcedure
Procedure addWordToDictionary(word$)
  ;write the personal dictionary entry
  Define lfh
    lfh = OpenFile(#PB_Any, hunspell_dict +hunspellLang+".dat") 
    If (lfh) 
      FileSeek(lfh,Lof(lfh)) ;append at end 
      WriteStringN(lfh,word$) 
      CloseFile(lfh) 
    EndIf 

  
  ;===================================
  addWordForSession(word$)
EndProcedure

cheers

Perkin · **Posted:** Fri Sep 24, 2010 8:37 pm

Thanks rsts.
I'd done the wordwrite routine, that was simple enough

DISREGARD - sorted (I'd messed up the call line and had @elements, not *elements :oops:

)
I'm now getting the 'Invalid memory access' on the PeekS(PeekI.....' line

If I remove one of the PeekI()'s then I get some results (though some is garbage)

Any suggestions?

THIS STILL APPLIES
Thanks for all your help.

rsts · **Posted:** Fri Sep 24, 2010 10:00 pm

No idea unless you're possibly not allocating enough space for the suggestions.

I just added these debug lines in my program and it shows just what it should.

Code:

Suggestions = CallCFunctionFast(hunspell_speller_suggest,hunSpellDict, @word$ ,*elements) 
    
    debug "suggestions " +Str(suggestions) 
    If Suggestions 
      
      For i=0 To Suggestions-1;  Repeat 
        sa.S(i) = PeekS(PeekI(PeekI(*elements)+(i*SizeOf(*elements))))
        debug "Suggestion : " + sa(i) 
      Next

Perkin · **Posted:** Fri Sep 24, 2010 10:14 pm

I solved the problem,

it was the organic bit in front of the keyboard. (Me) :oops:

Where I was trying to get it to work before I knew about the *elements=AllocateMemory(1024)
I'd changed one of the *elements to @elements, but hadn't changed it back again.

One thing I'm curious about, and trying to see if I can do, is to use an array elements(256) instead of the *elements, then recalling the contents should be corresponding to something like

Code:

sa.S(i) = PeekS(PeekI(elements(i)))

I said 'Something like', just testing.

Here's something to make you laugh.

This morning, I was just starting to sort out the hunsspelldll, and couldn't get any matches, couldn't for the life of me see why, using loads of debug statements it seemed to be working, library opening, lib functions were being assigned. It just wasn't matching any words as correctly spelt.

The answer - try copying the actual dictionary files into the directory. DOH' :oops:

Rook Zimbabwe · **Posted:** Fri Sep 24, 2010 10:58 pm

I cannot locate a win32 version so I downloaded the Delhi verssion... a DLL is a DLL after all...

This is what one of my DLL pokers came up with:

Code:

; German forum: http://www.purebasic.fr/german/archive/viewtopic.php?t=3216&highlight=
; Author: Mischa (updated for PB 4.00 by Andre)
; Date: 22. December 2003
; Modified for PB 4.3+ by Rook Zimbabwe
; Date 24 SEPTEMBER 2010
; OS: Windows
; Demo: Yes

; Automatically creates an include file for standard dll in the same path,
; which includes the definitions of their functions...

; Einleitung:
; Das kleine Ding erstellt fr Standard Dlls im gleichen Pfad eine
; fertige Include-Datei mit den Definitionen ihrer Funktionen, zwecks
; Nutzung mit CallFunctionFast().

;Include Generator
;Mischa Brandt

;Folgende Filter/Schnittmuster passend fr die fmod.dll
;(Include gibt's schon, wei ich. Ist nur zur Demonstration.)
leftcut     = 1
rightcut    = 0
separator.s = "@"
last_Path.s = GetCurrentDirectory()
;lib.s = "hunspelldll.dll" ;<-Pfad anpassen

libid = 0 ; temp number to hold place of file
lib.s=OpenFileRequester("DLL SCAN", last_path,"dll (*.dll)| *dll",0)

q.s=Chr(34):f1.s="DLLEntryPoint":f2.s="AttachProcess"
f3.s="DetachProcess":f4.s="AttachThread":f5.s="DetachThread"
If OpenLibrary(0,lib)
  If ExamineLibraryFunctions(0)
    If CreateFile(0,GetPathPart(lib)+"Include_"+ReplaceString(GetFilePart(lib),GetExtensionPart(lib),"pb"))
      WriteStringN(0,"OpenConsole()")
      WriteStringN(0,"  If OpenLibrary(libid,"+Chr(34)+GetFilePart(lib.s)+Chr(34)+")")
      While NextLibraryFunction()
        n.s  = LibraryFunctionName()
        nn.s = StringField(Mid(n,leftcut+1,Len(n)-leftcut-rightcut),1,separator)
        If n<>f1 And n<>f2 And n<>f3 And n<>f4 And n<>f5
          WriteStringN(0,"    Global *"+nn)
          WriteStringN(0,"    *"+nn+"=GetFunction(libid,"+q+n+q+")")
          WriteStringN(0,"Debug "+q+nn+"= "+q+"+"+q+q+"+Str(*"+nn+")")
        EndIf
      Wend
      WriteStringN(0,"  Else")
      WriteStringN(0,"    MessageRequester("+q+"error!"+q+","+q+"Can't open library!"+q+",0)")
      WriteStringN(0,"  Endif")
      WriteStringN(0,"Input()")
      WriteStringN(0,"End")
      CloseFile(0)
    Else
      MessageRequester("Error!","Can't create include file",0)
    EndIf
  Else
    CloseLibrary(0)
    MessageRequester("Error!","Can't examine library!",0)
  EndIf
  CloseLibrary(0)
Else
  MessageRequester("Error!","Can't open library!",0)
EndIf
End

I modded the program a bit sint GetFunction() is now the norm

[EDIT] CODE UPDATED NOW WITH CORRECT VERSION for PB4.3+
[[EDIT EDIT]] OK it is REALLY FIXED this time!!! :mrgreen:

What DLL are you using?

Perkin · **Posted:** Fri Sep 24, 2010 11:52 pm

Rook, the hunspelldll.dll I'm using is from v -1.2.12
located here, http://sourceforge.net/projects/hunspell/files/
view all files, hunspell 1.2.12 (rather than hyphen)

I'm looking into trying the NHunspell, but seems a lot more complicated.

I'll give that dll-extractor code a go tomorrow.

I used a delphi versions code/headers, to see what arguments each of the functions needed, to try and create a prototyped function list.
Got it to work, with rsts's help (see above)

I'll tidy it up when I can and post it, somebody else may be able to use and improve it.

Rook Zimbabwe · **Posted:** Sat Sep 25, 2010 8:45 pm

Give it a run, it creates a .pb program for you as well in the same directory you run it from.. I would simply copy it in to the folder you are using for hunspell and when run point it at the DLL (same one I am using I am pretty sure) and then open the PB code created and run to see what has given!

Perkin · **Posted:** Sat Sep 25, 2010 10:46 pm

Rook, it seems to miss the first letter off the functions, is that deliberate?

PureBasic Forum

Hunspell

Who is online