Page 2 of 2
Re: Convert all special chars in a text into regular letters
Posted: Wed Mar 07, 2018 10:46 am
by wilbert
Andre wrote:I noticed, that my Map of (e.g. Cities) >75,000 elements is increased by one element after the full-text search. And this one additional element contains only empty structure fields, finally causing the crash when calling the ConvertCharsPtr() function on an empty search string...
Maybe you are accidentally assigning something to a map element with an empty string for key ?
Code: Select all
NewMap Country.s()
Country("US") = "United States"
Debug MapSize(Country())
Country("") = "empty"
Debug MapSize(Country())
Andre wrote:About your question about the full-text search:
It's just my implementation of it, "collecting" the content of all available data fields for one element (city, country, etc.) and the comparing this (probably large) string with one or several searchterms, according to selected search-mode "contains all", "contains one", "contains none...", etc. As it's working well, I didn't thought about any change to it...
I asked it because combining strings with the + operator can be slow.
Re: Convert all special chars in a text into regular letters
Posted: Wed Mar 07, 2018 11:21 pm
by Andre
wilbert wrote:
Maybe you are accidentally assigning something to a map element with an empty string for key ?
Code: Select all
NewMap Country.s()
Country("US") = "United States"
Debug MapSize(Country())
Country("") = "empty"
Debug MapSize(Country())
Yes, this seems to be true. I already had this idea, when I went to bed last night.
But this evening I'm still debugging to find the causing piece of code (I already have an idea...)
When I see this problem and very hard to find bug (I would never have thought, that there could be a map element with an empty key), I'm thinking that a Compiler error (at least a warning) should be raised, if the programmer tries to use an empty key...
If not a bug in PB (at least unwanted feature), I would go for a feature request to forbid empty keys in PB maps. What do you think?
(I can't imagine, that someone really need this!?)
wilbert wrote:
Andre wrote:About your question about the full-text search:
It's just my implementation of it, "collecting" the content of all available data fields for one element (city, country, etc.) and the comparing this (probably large) string with one or several searchterms, according to selected search-mode "contains all", "contains one", "contains none...", etc. As it's working well, I didn't thought about any change to it...
I asked it because combining strings with the + operator can be slow.
Maybe I need to search for a faster variant, if the databases have grown. For the moment it's fast enough...
Re: Convert all special chars in a text into regular letters
Posted: Thu Mar 08, 2018 6:41 am
by wilbert
Andre wrote:When I see this problem and very hard to find bug (I would never have thought, that there could be a map element with an empty key), I'm thinking that a Compiler error (at least a warning) should be raised, if the programmer tries to use an empty key...
If not a bug in PB (at least unwanted feature), I would go for a feature request to forbid empty keys in PB maps. What do you think?
(I can't imagine, that someone really need this!?)
I probably wouldn't use an empty key but maybe someone else wants to.
An error seems to much but a warning might be convenient.
Since your problem with the character conversion procedure was related to an uninitialized string (null pointer), it might be possible this is also the problem in this case.
PureBasic not only allows for empty strings as key but even for uninitialized strings.
Code: Select all
NewMap Country.s()
A.s
Debug @A; null pointer
Country(A) = "My country"
Debug Country("")
Andre wrote:Maybe I need to search for a faster variant, if the databases have grown. For the moment it's fast enough...
If it's fast enough there's indeed no need to optimize
Re: Convert all special chars in a text into regular letters
Posted: Sun Mar 11, 2018 12:36 am
by Andre
Thank you, wilbert, for the further comments.
I've added a feature request here:
http://www.purebasic.fr/english/viewtop ... =3&t=70343
Re: Convert all special chars in a text into regular letters...
Posted: Wed Jun 09, 2021 8:25 pm
by Andre
I just started using PB6.00 alpha - and that's no surprise it don't work "out of the box" with my big project codes (105,000 and 40,000 lines). So I need to test and adapt step by step, to handle the needed changes...
One thing I came up with is the InlineASM code by 'wilbert' in this thread, which I heavily used for conversion of strings with umlauts/special chars into simple A-Z chars. This gives a "PureBasic - Assembler error" with the notes "error - unknown type name 'mov'" and "error: expected identifier or '(' before '[' token mov eax, [p.p_ConversionTable]".
Is this something which can be adapted? Or does the C-backend cause, that all InlineASM codes can't be used anymore?
Thanks for your help!
Re: Convert all special chars in a text into regular letters...
Posted: Wed Jun 09, 2021 8:35 pm
by Keya
Andre I don't think inline assembly is currently supported in the PB6 Alpha? (I think Fred's trying to get the C part working first!) It seems all "! code" lines are currently sent directly to the C compiler, as opposed to being interpreted as asm. Though you still can use gcc's inline assembly, that's pretty "quirky" though compared to what we're used to!