ADOmate - use OLE-DB datasources via ADO - (BLOBs added)

Developed or developing a new product in PureBasic? Tell the world about it.
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

ADOmate - use OLE-DB datasources via ADO - (BLOBs added)

Post by srod »

Update - 13th Aug 2010.
Version 2.0.2 of ADOmate has been updated to run faster... much faster in fact! :)

The general sluggishness of the previous versions of ADOmate was made apparent when I issued a query to retrieve 25000 records from a MS Access database! The damn thing just about ground to a halt!

Version 2.0.2 has been streamlined with the biggest change being the fact that ADOmate now uses, where appropriate, prepared COMatePLUS statements. This has made a huge difference. I should have done this earlier, but of course the original version of ADOmate was created before I added statements to COMate and, I kind of overlooked ADOmate! :)

You can also disable all error reporting from the record retrieval functions (for even more speed) by declaring the constant #ADOmate_NOERRORREPORTINGDURINGRECORDRETRIEVAL = 1 before including the ADOmate source etc.

ADOmate 2.0.2 now requires COMatePLUS and will not run with earlier versions of COMate.

===========================================


Update - 6th Aug 2010.
Version 2.0.1 of ADOmate alters one function and adds another.


ADOmate_ListDatabaseTablesFromConnectionString()
This function takes an ADO connection string and attempts to open the connection and list all of the tables in the underlying database/datasource. Suitable if you do not already hold a connection to the underlying database and just wish to query the tables.


ADOmate_ListDatabaseTables()
This function takes an existing connection and will attempt to list all of the tables in the underlying database/datasource.


The important thing to note is that the second of these functions, ADOmate_ListDatabaseTables(), will not open a separate connection to the database, it will use the existing one.

In terms of existing ADOmate applications, any application which previously called the original ADOmate_ListDatabaseTables() function, will now need to use ADOmate_ListDatabaseTablesFromConnectionString() instead because the original ADOmate_ListDatabaseTables() function has simply been renamed! :) I have done this for my own convenience.

Please see the updated user manual and the 'listing tables' demo for more details.

===========================================


Update - 31st Jan 2010.
Version 2.0.0 of ADOmate adds support for BLOBs and is to be considered as a test version at the moment.

Support for BLOBs in ADO is a little ‘mixed’ and perhaps the best way of storing such data within our databases involves the use of ADO stream objects alongside updateable recordset objects; something which, whilst relatively straightforward, is not really in keeping with the Purebasic database library and hence is not an approach which I have taken here with ADOmate. Developers are of course free to take this approach. Kiffi posted some code (using COMate) for doing just that in the following forum thread : http://www.purebasic.fr/english/viewtop ... 13&t=40857

Instead, and in keeping with Purebasic's database library, ADOmate attempts to allow the developer to bind BLOB data to ‘parameters’ for use with ADOmate_DatabaseUpdate(). The problem is, however, that ADO doesn’t really allow us to bind parameters in quite the same way as we can with ODBC or SQLite etc. Different providers will either recognise references to our parameters in our SQL statements or they will not. Those that do will invariably use their own syntax for referencing such parameters. Those that do not allow references to bound parameters within SQL statements will instead require us to work with blobs via the ADO stream objects mentioned above.

For example, the OLE-DB Jet provider (MS Access files) allows us to reference our parameters with the following kind of syntax in our SQL :

Code: Select all

SQL$ = "Insert Into Pictures (Pic1, Pic2) Values (?, ?);"
where the ? symbols are used as placeholders etc.

Other providers, if supporting the use of parameters in SQL statements at all, will invariably use a different kind of syntax for referencing the parameters.

Because of this I am classing this version of ADOmate as a test version. It works fine with the OLE-DB Jet provider with MS Access mdb database files, but even here we have a couple of things to note (as you will see in the ADOmate user manual). I will need to await reports of whether this mechanism can be found (or made) to work with other providers; e.g. SQL server and the like, but at least it works with Jet! :)

New functions added :

Code: Select all

ADOmate_DatabaseColumnSize()
ADOmate_GetDatabaseBlob()
ADOmate_GetDatabaseBlobByFieldName()
ADOmate_GetDatabaseVariant()
ADOmate_GetDatabaseVariantByFieldName()
ADOmate_SetDatabaseBlob
Functions altered :

Code: Select all

ADOmate_DatabaseUpdate
Please see the updated user manual and the heavily commented BLOB demo within the download.

===========================================


Update - 12th Nov 2008.
Added functions to retrieve data by field name to supplement those which retrieve data by column index etc.

Code: Select all

ADOmate_GetDatabaseDoubleByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseFloatByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseLongByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseQuadByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseStringByFieldName(*connection, fieldName$)
===========================================


Hi,

here is a simple source code wrapper for using OLE-DB datasources via Microsoft's ADO (ActiveX Data Objects) library.

ADOmate follows the Purebasic database library almost identically in the way that we define and execute queries and access the resulting recordsets etc. It is thus very easy to use. There are a couple of differences which have been forced upon me, but these are described in the accompanying user guide.

This ease of use does come at the cost of a little flexibility, however, in that some of the more advanced features of ADO are not supported directly by ADOmate. However, since ADOmate relies heavily upon my COMate library, it is consequently very very easy to use COMate to access some of ADO's more advanced features etc. Still, for most everyday uses this will not be required.

Basic feature set :
  • errors are reported on a thread-by-thread basis; meaning that multiple threads record their errors separately etc. (much like COMate does)
  • unicode support
  • create an empty database
  • many functions have counterparts within Purebasic's database library
  • full support for the different 'cursor types' (no knowledge of this is required however!)
ADOmate will only compile on PB 4.3 (and later! :wink: ) and as such requires the version of COMate for PB 4.3 (available separately at the nxSoftware site).

The ADOmate package includes all source files (other than COMate), some demo programs and a rather hastily thrown together user guide! There is also Kiffi's "adoconstants.pbi" file included within the download which is required by ADOmate (although only a small part of this file is used). My thanks to Kiffi for this.

See the nxSoftware site for the download link.

Enjoy.
Last edited by srod on Fri Aug 13, 2010 7:15 pm, edited 8 times in total.
I may look like a mule, but I'm not a complete ass.
User avatar
Kiffi
Addict
Addict
Posts: 1358
Joined: Tue Mar 02, 2004 1:20 pm
Location: Amphibios 9

Re: ADOmate - use OLE-DB datasources via ADO

Post by Kiffi »

srod wrote:here is a simple source code wrapper for using OLE-DB datasources via Microsoft's ADO (ActiveX Data Objects) library.
Great! Thanks a lot! :D

One little feature request: can you implement a function like this?

Code: Select all

ADOmate_GetDatabaseStringByFieldname(myConn, "myFieldname") ; or so...
So it is possible to get fieldvalues by name and not by an index.

Greetings ... Kiffi
Hygge
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Re: ADOmate - use OLE-DB datasources via ADO

Post by srod »

Kiffi wrote:
srod wrote:here is a simple source code wrapper for using OLE-DB datasources via Microsoft's ADO (ActiveX Data Objects) library.
Great! Thanks a lot! :D

One little feature request: can you implement a function like this?

Code: Select all

ADOmate_GetDatabaseStringByFieldname(myConn, "myFieldname") ; or so...
So it is possible to get fieldvalues by name and not by an index.

Greetings ... Kiffi
Aye, not a problem. Given the choice I would have had it this way anyhow, but decided against it just to keep things in line with Purebasic's database library etc. :)
I may look like a mule, but I'm not a complete ass.
User avatar
Kiffi
Addict
Addict
Posts: 1358
Joined: Tue Mar 02, 2004 1:20 pm
Location: Amphibios 9

Re: ADOmate - use OLE-DB datasources via ADO

Post by Kiffi »

srod wrote:Aye, not a problem.
sounds good! :D
srod wrote:but decided against it just to keep things in line with Purebasic's database library etc. :)
in fact: this is also a feature request for Fred but he hasn't implement it yet. ;-)

Thanks a lot & Greetings ... Kiffi
Hygge
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Post by srod »

Update - 12th Nov 2008.
Added functions to retrieve data by field name to supplement those which retrieve data by column index etc.

Code: Select all

ADOmate_GetDatabaseDoubleByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseFloatByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseLongByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseQuadByFieldName(*connection, fieldName$)
ADOmate_GetDatabaseStringByFieldName(*connection, fieldName$)
I may look like a mule, but I'm not a complete ass.
kinglestat
Enthusiast
Enthusiast
Posts: 732
Joined: Fri Jul 14, 2006 8:53 pm
Location: Malta
Contact:

Post by kinglestat »

ADOmate or sADOmate ?
I may not help with your coding
Just ask about mental issues!

http://www.lulu.com/spotlight/kingwolf
http://www.sen3.net
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Post by Little John »

Another cool and very useful library from you. Thanks a lot, srod!
Currently I'm especially fascinated by the possibility to handle Excel files without Excel. In order to list the tables in an Excel file, I appended your demo program Demo_ListingTables.pb to Demo_Excel.pb (slightly modified):

Code: Select all

IncludePath ".."
XIncludeFile "ADOmate.pbi"

EnableExplicit

Define Connection, Count, i
Define ExcelFile.s, ConnectionString.s

ExcelFile.s = "XlsWithoutExcel.xls" 

If FileSize(ExcelFile) > 0 
   DeleteFile(ExcelFile) 
EndIf 

ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + ExcelFile + ";Extended Properties=" + Chr(34) + "Excel 8.0;HDR=YES" + Chr(34)
Connection = ADOmate_OpenDatabase(ConnectionString)

If Connection = 0
   Debug ADOmate_GetLastErrorDescription()
   End
EndIf

;-- Create a table
ADOmate_DatabaseUpdate(Connection, "Create Table AddressBook (LastName String, FirstName String, Address String, City String)") 
ADOmate_DatabaseUpdate(Connection, "Insert Into AddressBook (LastName, FirstName, Address, City) Values ('Hansen', 'Ola', 'Timoteivn 10', 'Sandnes')")
ADOmate_DatabaseUpdate(Connection, "Insert Into AddressBook (LastName, FirstName, Address, City) Values ('Svendson', 'Tove', 'Borgvn 23', 'Sandnes')")

;-- List tables
Dim a.s(0)         ; string array which will be filled with the names of the tables

; First list all tables.
count = ADOmate_ListDatabaseTables(connectionString, a())
If count
   Debug "Listing all tables :"
   Debug "--------------------------"
   For i = 0 To count-1
      Debug "Table " + Str(i) + " --> " + a(i)
   Next
Else
   MessageRequester("ADOmate error!", ADOmate_GetLastErrorDescription())
   End
EndIf

; Now remove the system tables.
count = ADOmate_ListDatabaseTables(connectionString, a(), "table")
If count
   Debug ""
   Debug "Listing non system tables :"
   Debug "----------------------------------------"
   For i = 0 To count-1
      Debug "Table " + Str(i) + " --> " + a(i)
   Next
EndIf

ADOmate_CloseDatabase(Connection)
The output is:
Listing all tables :
--------------------------
Table 0 --> AddressBook
Table 1 --> AddressBook$

Listing non system tables :
----------------------------------------
Table 0 --> AddressBook
Table 1 --> AddressBook$
Does it make sense that (in both cases) the table name is listed twice, with and without a trailing $?

Regards, Little John
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Post by srod »

In kind of makes sense considering you are using ADOmate to create the table. With ADO (and ODBC) the Excel table names are usually referenced (within SQL statements) by [tableName$] etc. and can even be referenced by tableName$.

I don't think that there is anything to be concerned about.
I may look like a mule, but I'm not a complete ass.
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Post by Little John »

Well, I only created 1 table (and opening the file with Excel shows 1 table, too), but listing the tables with ADOmate reports 2 tables. This is confusing for me. :-)
Does this mean that I cannot rely on the count returned by ADOmate_ListDatabaseTables(), but always have to check the reported names whether or not they are identical exept of a trailing $ character?

Regards, Little John
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Post by srod »

Well whatever is happening is the fault of the Jet 4.0 driver. I tested with an Excel file created manually and the same code lists each table only once. By using the Jet 4.0 provider to create the Excel file it is apparently then deciding to list each table in this way. It is too much of a coincidence in my opinion that it is choosing to append a $ character.

At the end of the day, Excel is not a DBMS and so you must expect some peculiarities.

Writing some code to differentiate the table names should be simple enough.
I may look like a mule, but I'm not a complete ass.
Little John
Addict
Addict
Posts: 4527
Joined: Thu Jun 07, 2007 3:25 pm
Location: Berlin, Germany

Post by Little John »

I see, thanks for your explanation!

Regards, Little John
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Re: ADOmate - use OLE-DB datasources via ADO - (BLOBs added)

Post by srod »

Update - 31st Jan 2010.
Version 2.0.0 of ADOmate adds support for BLOBs and is to be considered as a test version at the moment.

Support for BLOBs in ADO is a little ‘mixed’ and perhaps the best way of storing such data within our databases involves the use of ADO stream objects alongside updateable recordset objects; something which, whilst relatively straightforward, is not really in keeping with the Purebasic database library and hence is not an approach which I have taken here with ADOmate. Developers are of course free to take this approach. Kiffi posted some code (using COMate) for doing just that in the following forum thread : http://www.purebasic.fr/english/viewtop ... 13&t=40857

Instead, and in keeping with Purebasic's database library, ADOmate attempts to allow the developer to bind BLOB data to ‘parameters’ for use with ADOmate_DatabaseUpdate(). The problem is, however, that ADO doesn’t really allow us to bind parameters in quite the same way as we can with ODBC or SQLite etc. Different providers will either recognise references to our parameters in our SQL statements or they will not. Those that do will invariably use their own syntax for referencing such parameters. Those that do not allow references to bound parameters within SQL statements will instead require us to work with blobs via the ADO stream objects mentioned above.

For example, the OLE-DB Jet provider (MS Access files) allows us to reference our parameters with the following kind of syntax in our SQL :

Code: Select all

SQL$ = "Insert Into Pictures (Pic1, Pic2) Values (?, ?);"
where the ? symbols are used as placeholders etc.

Other providers, if supporting the use of parameters in SQL statements at all, will invariably use a different kind of syntax for referencing the parameters.

Because of this I am classing this version of ADOmate as a test version. It works fine with the OLE-DB Jet provider with MS Access mdb database files, but even here we have a couple of things to note (as you will see in the ADOmate user manual). I will need to await reports of whether this mechanism can be found (or made) to work with other providers; e.g. SQL server and the like, but at least it works with Jet! :)

New functions added :

Code: Select all

ADOmate_DatabaseColumnSize()
ADOmate_GetDatabaseBlob()
ADOmate_GetDatabaseBlobByFieldName()
ADOmate_GetDatabaseVariant()
ADOmate_GetDatabaseVariantByFieldName()
ADOmate_SetDatabaseBlob
Functions altered :

Code: Select all

ADOmate_DatabaseUpdate
Please see the updated user manual and the heavily commented BLOB demo within the download.
I may look like a mule, but I'm not a complete ass.
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Re: ADOmate - use OLE-DB datasources via ADO - (BLOBs added)

Post by srod »

Update - 6th Aug 2010.
Version 2.0.1 of ADOmate alters one function and adds another.


ADOmate_ListDatabaseTablesFromConnectionString()
This function takes an ADO connection string and attempts to open the connection and list all of the tables in the underlying database/datasource. Suitable if you do not already hold a connection to the underlying database and just wish to query the tables.


ADOmate_ListDatabaseTables()
This function takes an existing connection and will attempt to list all of the tables in the underlying database/datasource.


The important thing to note is that the second of these functions, ADOmate_ListDatabaseTables(), will not open a separate connection to the database, it will use the existing one.

In terms of existing ADOmate applications, any application which previously called the original ADOmate_ListDatabaseTables() function, will now need to use ADOmate_ListDatabaseTablesFromConnectionString() instead because the original ADOmate_ListDatabaseTables() function has simply been renamed! :) I have done this for my own convenience.

Please see the updated user manual and the 'listing tables' demo for more details.
I may look like a mule, but I'm not a complete ass.
srod
PureBasic Expert
PureBasic Expert
Posts: 10589
Joined: Wed Oct 29, 2003 4:35 pm
Location: Beyond the pale...

Re: ADOmate - use OLE-DB datasources via ADO - (BLOBs added)

Post by srod »

Update - 13th Aug 2010.
Version 2.0.2 of ADOmate has been updated to run faster... much faster in fact! :)

The general sluggishness of the previous versions of ADOmate was made apparent when I issued a query to retrieve 25000 records from a MS Access database! The damn thing just about ground to a halt!

Version 2.0.2 has been streamlined with the biggest change being the fact that ADOmate now uses, where appropriate, prepared COMatePLUS statements. This has made a huge difference. I should have done this earlier, but of course the original version of ADOmate was created before I added statements to COMate and, I kind of overlooked ADOmate! :)

You can also disable all error reporting from the record retrieval functions (for even more speed) by declaring the constant #ADOmate_NOERRORREPORTINGDURINGRECORDRETRIEVAL = 1 before including the ADOmate source etc.

ADOmate 2.0.2 now requires COMatePLUS and will not run with earlier versions of COMate.
I may look like a mule, but I'm not a complete ass.
KIKI
Enthusiast
Enthusiast
Posts: 145
Joined: Thu Dec 28, 2006 11:49 am
Location: FRANCE

Re: ADOmate - use OLE-DB datasources via ADO - (BLOBs added)

Post by KIKI »

Since this new Version 2.0.2, i have a strnage problem with an access database
If i test the type of the fields it return me the right type (String) and when i want to access to the content of the fields it return me an error

Code: Select all

Type mismatch in the method  parmameters
, this problem wasn't present with my database and the older version
Post Reply