IMDB query fails to return content in xml file?

Just starting out? Need help? Post your questions and find answers here.
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4747
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

IMDB query fails to return content in xml file?

Post by Fangbeast »

I haven't looked at this code since 2013 and back then it was working fine. I can still get an xml file back but there seems to be no usable details in it for me to grab. (I seem to need this stuff again, wife wants to build her movie database again (groan!!))

Would some kind soul take a look at it for me please?

Code: Select all

; My constants

#FileNotFound = -1                                                                                                ; 

; My program constants

Structure ProgramData
  CurrentDirectory.s                                                                                              ; 
  TemporaryDirectory.s                                                                                            ; 
  
  HTTPHeader.s                                                                                                    ; 
  HTTPFile.s                                                                                                      ; 
  
  MovieFilename.s                                                                                                 ; 
EndStructure

Global Program.ProgramData                                                                                        ; 

Program\CurrentDirectory      = GetCurrentDirectory()                                                             ; 
Program\TemporaryDirectory    = "D:\Fresh_Goats\"                                                                 ; 
Program\IMDBDataFileName      = Program\TemporaryDirectory  + "ImdAPI.org.xml"                                    ; 

Program\HTTPHeader            = "http://imdbapi.org/?"                                                            ; 
Program\HTTPFile              = Program\CurrentDirectory    + "MovieBase.HTTPFile"                                ; 

; IMDB constants

Structure ImdbOptions
  Stitle.s                                                                                                        ; Name of the movie to search for
  Type.s                                                                                                          ; Type of data to return
  Year.s                                                                                                          ; The year to search for
  Plot.s                                                                                                          ; Full, short or none
  Limit.s                                                                                                         ; Limit number of results
  ImdbId.s                                                                                                        ; Search by Imdb ID field only
EndStructure

Global Imdb.ImdbOptions                                                                                           ; 

Imdb\Stitle       = "q="                                                                                          ; Name of the movie to search for
Imdb\Type         = "type="                                                                                       ; Type of data to return , XML, JSON, JSONP
Imdb\Year         = "year="                                                                                       ; The year to search for
Imdb\Plot         = "plot="                                                                                       ; none, simple, full
Imdb\Limit        = "limit="                                                                                      ; Limit number of results 1-10, 1 = default
Imdb\ImdbId       = "id="                                                                                         ; Search by IMDB id fields

; 

Structure MovieData
  MovieRating.s                                                                                                   ; 9.0
  MovieRatingCount.s                                                                                              ; 673844
  MovieYear.s                                                                                                     ; 1994
  MoviePlot.s                                                                                                     ; Jules Winnfieldents.
  MovieGenres.s                                                                                                   ; Crime
  MovieRated.s                                                                                                    ; R
  MovieTitle.s                                                                                                    ; Pulp Fiction
  MovieImdbUrl.s                                                                                                  ; http://www.imdb.com/title/tt0110912/
  MovieDirectors.s                                                                                                ; Quentin Tarantino
  MovieWriters.s                                                                                                  ; Quentin Tarantino
  MovieActors.s                                                                                                   ; John Travolta
  MoviePlotSimple.s                                                                                               ; The lives of two n.
  MovieType.s                                                                                                     ; M (Movie etc)
  MoviePoster.s                                                                                                   ; http://ia.media-imdb.com/images/M/MV5BMjE0ODk2NjczOV5BMl5BanBnXkFtZTYwNDQ0NDg4._V1._SY317_CR4,0,214,317_.jpg
  MovieImdbId.s                                                                                                   ; tt0110912
  MovieAlsoKnownAs.s                                                                                              ; Tiempos violentos (A.K.A)
  MovieLanguage.s                                                                                                 ; English, Spanish
  MovieCountry.s                                                                                                  ; USA
  MovieReleaseDate.s                                                                                              ; 19941014
  MovieFilmingLocations.s                                                                                         ; 1525 N. Van Ness Avenue, Los Angeles, California, USA
  MovieRuntime.s                                                                                                  ; 154 min</item><item>USA: 168 min (special edition)
  MoviePicture.s                                                                                                  ; Pulp Fiction.jpg (Going to be a BLOB field from now on)
  MovieFavourite.s                                                                                                ; Is this one of your favourite movies?
  MovieLastEdited.s                                                                                               ; laste dited
  MovieDeleted.s                                                                                                  ; 1 for yes, 0 for no
  MovieRecord.s                                                                                                   ; Autogenerated
EndStructure

Global Movie.MovieData                                                                                            ; All movie data information

; 

SearchTitle.s = "Stargate: Continuum"
SearchYear.s  = "2008"
SearchImdb.s  = "t0929629"

; 

Declare   ReadImdbXmlDataFile(*CurrentNode, CurrentSublevel.i)
Declare.s GetChildItems(*node)

; Go ahead with the search if the info file doesn't exist (Windows 8 security problem, can't 
; seem to overwrite an existing file. have to delete it first

If FileSize(Program\IMDBDataFileName.s) <> #FileNotFound
  DeleteFile(Program\IMDBDataFileName.s)
EndIf

 ;Program\HTTPHeader   +       Imdb\STitle       + SearchTitle.s   ; Search for a title      "q=star trek"
  
 ;Program\HTTPHeader   +       Imdb\STitle       + SearchTitle.s   ; Search for a title      "q=star trek"
 ;Program\HTTPHeader   + "&" + Imdb\Year         + SearchYear.s    ; Add the year search     "&year=1943"
  
  Program\HTTPHeader   +       Imdb\ImdbId       + SearchImdb.s    ; Add the IMDB id field   "id=tt0344854"
  
; If the title field was not empty, there are two arguments to pass. Title and year

Program\HTTPHeader     + "&" + Imdb\Type         + "XML"           ; Return XML data format  "&type=xml"
Program\HTTPHeader     + "&" + Imdb\Plot         + "full"          ; Return the full plot    "&Plot=full"

; 

Debug Program\HTTPHeader

; Read and process the IMDB XML data file

Procedure ReadImdbXmlDataFile(*CurrentNode, CurrentSublevel.i)
  Protected NodeName.s
  If XMLNodeType(*CurrentNode) = #PB_XML_Normal
    NodeName.s = GetXMLNodeName(*CurrentNode)
    Select NodeName.s
      Case "rating"             : Movie\MovieRating             = GetXMLNodeText(*CurrentNode)  ; 
      Case "rating_count"       : Movie\MovieRatingCount        = GetXMLNodeText(*CurrentNode)  ;
      Case "year"               : Movie\MovieYear               = GetXMLNodeText(*CurrentNode)  ;
      Case "plot"               : Movie\MoviePlot               = GetXMLNodeText(*CurrentNode)  ;
      Case "genres"             : Movie\MovieGenres             =  GetChildItems(*CurrentNode)  ; Has sub-items
      Case "rated"              : Movie\MovieRated              = GetXMLNodeText(*CurrentNode)  ;
      Case "title"              : Movie\MovieTitle              = GetXMLNodeText(*CurrentNode)  ;
      Case "imdb_url"           : Movie\MovieImdbUrl            = GetXMLNodeText(*CurrentNode)  ;
      Case "directors"          : Movie\MovieDirectors          =  GetChildItems(*CurrentNode)  ; Has sub-items
      Case "writers"            : Movie\MovieWriters            =  GetChildItems(*CurrentNode)  ; Has sub-items
      Case "actors"             : Movie\MovieActors             =  GetChildItems(*CurrentNode)  ; Has sub-items
      Case "plot_simple"        : Movie\MoviePlotSimple         = GetXMLNodeText(*CurrentNode)  ;
      Case "type"               : Movie\MovieType               = GetXMLNodeText(*CurrentNode)  ;
      Case "poster"             : Movie\MoviePoster             = GetXMLNodeText(*CurrentNode)  ;
      Case "imdb_id"            : Movie\MovieImdbId             = GetXMLNodeText(*CurrentNode)  ;
      Case "also_known_as"      : Movie\MovieAlsoKnownAs        = GetXMLNodeText(*CurrentNode)  ;
      Case "language"           : Movie\MovieLanguage           = GetXMLNodeText(*CurrentNode)  ;
      Case "country"            : Movie\MovieCountry            = GetXMLNodeText(*CurrentNode)  ;
      Case "release_date"       : Movie\MovieReleaseDate        = GetXMLNodeText(*CurrentNode)  ;
      Case "filming_locations"  : Movie\MovieFilmingLocations   = GetXMLNodeText(*CurrentNode)  ;
      Case "runtime"            : Movie\MovieRuntime            =  GetChildItems(*CurrentNode)  ; Has sub-items
    EndSelect
    *ChildNode = ChildXMLNode(*CurrentNode)
    While *ChildNode <> 0
      ReadImdbXmlDataFile(*ChildNode, CurrentSublevel + 1)
      *ChildNode = NextXMLNode(*ChildNode)
    Wend
  EndIf
EndProcedure

; Get the child items of a selected XML node and return them

Procedure.s GetChildItems(*node)
  IndexField.i = #True
  *ChildNode = ChildXMLNode(*node, IndexField.i)
  ResourceString.s = #Empty$
  DelimiterString.s = #Empty$
  While *ChildNode
    ResourceString.s + DelimiterString.s + GetXMLNodeText(*ChildNode)
    DelimiterString.s = ", "
    IndexField.i + 1
    *ChildNode = ChildXMLNode(*node, IndexField.i)
  Wend
  ProcedureReturn ResourceString
EndProcedure

; Properly encode the request URL and search text to prevent errors

Program\HTTPHeader     = URLEncoder(Program\HTTPHeader)

; Try to receive the requested information to a file on disk

If URLDownloadToFile_(0, Program\HTTPHeader, Program\IMDBDataFileName.s, 0, 0) = #S_OK
  ; Let the user know that we received some data
  Debug "We successfully retrieved the movie data XML file to disk"
  ; Proceed if there was a received movie information file
  If FileSize(Program\IMDBDataFileName.s) <> #FileNotFound
    Debug "We managed to find the XML movie file that we just downloaded"
    ImdbXMLFile.i = ReadFile(#PB_Any, Program\IMDBDataFileName.s)
    If ImdbXMLFile.i  <>  #False
      Debug "We managed to read from the XML movie file that we just downloaded"
      ImdbXMLFileSize.q = Lof(ImdbXMLFile.i)
      If ImdbXMLFileSize.q
        Debug "There seems to be some data in the XML movie file that we just downloaded"
        *Buffer = AllocateMemory(ImdbXMLFileSize.q)
        If *Buffer
          Debug "We managed to allocate a data buffer to load the XML movie file into that we just downloaded"
          If ReadData(ImdbXMLFile, *Buffer, ImdbXMLFileSize.q) = ImdbXMLFileSize.q
            Debug "We managed read the XML movie file into the data buffer allocated"
            XMLDataHandle.i = CatchXML(#PB_Any, *Buffer, ImdbXMLFileSize.q)
            If XMLDataHandle.i
              Debug "We managed to read/parse the XML data file into memory from the buffer"
              *Node = MainXMLNode(XMLDataHandle.i)
              If *Node
                ReadImdbXmlDataFile(*Node, 0)
              Else
                Debug "No main XML data node int he file so there is nothing to do"
              EndIf
              FreeXML(XMLDataHandle.i)
              Debug "Found title      ::: " + Movie\MovieTitle
              Debug "Found year       ::: " + Movie\MovieYear
              Debug "Found genre      ::: " + Movie\MovieGenres
              Debug "Found directors  ::: " + Movie\MovieDirectors
              Debug "Found actors     ::: " + Movie\MovieActors
              Debug "Found plot       ::: " + Movie\MoviePlot
              Debug "Found IMDB id    ::: " + Movie\MovieImdbId
            EndIf
          Else
            Debug "We didn't manage read the XML movie file into the data buffer allocated"
          EndIf
          FreeMemory(*Buffer)
        Else
          Debug "We didn't to allocate a data buffer to load the XML movie file into that we just downloaded"
        EndIf
        CloseFile(ImdbXMLFile.i)
      Else
        Debug "There seems to be no data in the XML movie file that we just downloaded"
      EndIf
    Else
      Debug "We didn't manage to read from the XML movie file that we just downloaded"
    EndIf
    
    ; Process the next bit code if there is a picture reference to fetch
    If Movie\MoviePoster <> #Empty$
      ; Make sure the picture filename to retrieve is properly encoded
      Movie\MoviePoster = URLEncoder(Movie\MoviePoster)
      ; Take the new picture filename from the movie title
      NewPictureFilename.s   = Program\TemporaryDirectory + "MovieBaseImportPicture." + GetExtensionPart(Movie\MoviePoster)
      ; Delete any previously cached download data. API
      If FileSize(NewPictureFilename.s) <> #FileNotFound
        DeleteFile(NewPictureFilename.s, #PB_FileSystem_Force)
      EndIf
      ; Delete any previously cached download data. API
      DeleteUrlCacheEntry_(@StringData.s)
      ; Now try to retrieve the picture from the imdb site. API. PB command didn't work with this
      If URLDownloadToFile_(0, Movie\MoviePoster, NewPictureFilename.s, 0, 0) = #S_OK
        ; Check if the selected picture already lives in the temporary directory
        If FileSize(NewPictureFilename.s) <> #FileNotFound
          ; Now try to load the image from the local picture storage area
          Debug NewPictureFilename.s
        Else
          Debug "A picture was downloaded successfully but has gone missing?"
        EndIf
      Else
        Debug "The picture was not downloaded successfully to the temporary directory"
      EndIf
    Else
      Debug "There is no picture to download for this movie"
    EndIf
    ; 
  Else
    Debug "Imdb movie data XML data was downloaded but I can't find it?"
  EndIf
  ; 
Else
  Debug "Failed to retrieve the movie data XML file to disk"
EndIf

End
Amateur Radio, D-STAR/VK3HAF
User avatar
Bisonte
Addict
Addict
Posts: 1226
Joined: Tue Oct 09, 2007 2:15 am

Re: IMDB query fails to return content in xml file?

Post by Bisonte »

The API's of IMDB have always been "unofficial" and are no longer in use since 2016.

A comparable database is TMDb. On this page you have to login and get an API-Key to use the API (free for non commercial) and the key will be sent quite fast via mail..
The Api documentation is very detailed, but it doesn't use xml but json, which in my opinion makes data processing much easier.
PureBasic 6.04 LTS (Windows x86/x64) | Windows10 Pro x64 | Asus TUF X570 Gaming Plus | R9 5900X | 64GB RAM | GeForce RTX 3080 TI iChill X4 | HAF XF Evo | build by vannicom​​
English is not my native language... (I often use DeepL to translate my texts.)
User avatar
Fangbeast
PureBasic Protozoa
PureBasic Protozoa
Posts: 4747
Joined: Fri Apr 25, 2003 3:08 pm
Location: Not Sydney!!! (Bad water, no goats)

Re: IMDB query fails to return content in xml file?

Post by Fangbeast »

Bisonte wrote:The API's of IMDB have always been "unofficial" and are no longer in use since 2016.

A comparable database is TMDb. On this page you have to login and get an API-Key to use the API (free for non commercial) and the key will be sent quite fast via mail..
The Api documentation is very detailed, but it doesn't use xml but json, which in my opinion makes data processing much easier.
Thanks, I'll have a look at TMDB. As for 2016, I told everyone how long it was since I used this code!!

As for json, I know bugger all about it and find it hard to follow. I'll get a key next week and see what I can't do, then yell for help:):)

**UPDATE** TMDB says they accept IMDB id's for movies so I tested the one in my code and it worked fine for "Pulp fiction, returned data.

Then I tried the StarGate Continuum IMDB id and it returned nothing. When I searched for that movie on IMDB, the id in the web address was correct.

Then I searched for that movie on TMDB using the title and it came up. So it appears that their results are very inconsistent with what id's they have for movies.

Do you have an example for dealing with json data for them that I could play around with?
Amateur Radio, D-STAR/VK3HAF
Post Reply