Searching session history

Got an idea for enhancing PureBasic? New command(s) you'd like to see?
Dude
Addict
Addict
Posts: 1907
Joined: Mon Feb 16, 2015 2:49 pm

Searching session history

Post by Dude »

I'd like to be able to search session history for text. Currently I need to find some code I wrote 4 months ago, but I have no easy way of finding it. (My session history is 12 months). I'd love to just enter a constant name to search for, that would show the session file that it was in. Thank you.
Last edited by Dude on Sat Mar 03, 2018 12:24 am, edited 1 time in total.
IdeasVacuum
Always Here
Always Here
Posts: 6425
Joined: Fri Oct 23, 2009 2:33 am
Location: Wales, UK
Contact:

Re: Searching session history

Post by IdeasVacuum »

+1
IdeasVacuum
If it sounds simple, you have not grasped the complexity.
davido
Addict
Addict
Posts: 1890
Joined: Fri Nov 09, 2012 11:04 pm
Location: Uttoxeter, UK

Re: Searching session history

Post by davido »

@Dude,
What a great idea. So... +1
DE AA EB
User avatar
kenmo
Addict
Addict
Posts: 1967
Joined: Tue Dec 23, 2003 3:54 am

Re: Searching session history

Post by kenmo »

+1 Would be a nice feature.

Here's a quick and dirty tool to search your history. Fun 2-day project :) Had to do a little reverse-engineering on the IDE's History.db file.

EDIT: Minor update March 5-6

Code: Select all

; +----------------+
; | SessionHistory |
; +----------------+
; | 2018-03-03 . Creation
; | 2018-03-05 . Cleanup, ASCII/UTF-8 detection, EnableExplicit
; | 2018-03-06 . Improvements based on new Database/Diff format info

CompilerIf (Not Defined(_SessionHistory_Included, #PB_Constant))
#_SessionHistory_Included = #True

CompilerIf (#PB_Compiler_IsMainFile)
  EnableExplicit
CompilerEndIf


;-
;- Constants

Enumeration 1
  #DATA_Empty ; data is null. the file is empty
  #DATA_Same  ; data is null. the file is the same as on the previous event
  #DATA_Diff  ; data is a diff since the previous event
  #DATA_Full  ; data is the full file content (new files in a session that are not empty, also used if diff is too large or if the chain of diff'ed events becomes too long)
  #DATA_DiffZ ; data is a diff (zlib compressed)
  #DATA_FullZ ; data is full file (zlib compressed)
EndEnumeration


;-
;- Structures

Structure SessionHistoryFileStruct
  Filename.s
  Event_ID.s
  Session_ID.s
  Text.s
  Time.i
EndStructure


;-
;- Globals

Global NewList SessionHistoryFile.SessionHistoryFileStruct()


;-
;- Procedures

Procedure.i UnpackHistoryBlob(*Blob)
  Protected *Result = #Null
  If (*Blob)
    Protected UnpackedSize.i = PeekL(*Blob)
    If (UnpackedSize > 0)
      *Result = AllocateMemory(UnpackedSize, #PB_Memory_NoClear)
      If (*Result)
        UseZipPacker()
        If (UncompressMemory(*Blob + 4, MemorySize(*Blob) - 4, *Result, UnpackedSize))
          ; OK
        Else
          FreeMemory(*Result)
          *Result = #Null
        EndIf
      EndIf
    EndIf
  EndIf
  ProcedureReturn (*Result)
EndProcedure

Procedure.i GetDatabaseBlobBuffer(Database.i, Column.i)
  Protected *Result = #Null
  Protected BlobSize.i = DatabaseColumnSize(Database, Column)
  If (BlobSize > 0)
    *Result = AllocateMemory(BlobSize, #PB_Memory_NoClear)
    If (*Result)
      If (GetDatabaseBlob(Database, Column, *Result, BlobSize))
        ; OK
      Else
        FreeMemory(*Result)
        *Result = #Null
      EndIf
    EndIf
  EndIf
  ProcedureReturn (*Result)
EndProcedure

Procedure.s ApplyHistoryChanges(Text.s, *Mem, MemSize.i, Format.i)
  If (*Mem And (MemSize > 0))
    Protected PartLen.i
    Protected TargetSize.i = PeekL(*Mem)
    If (TargetSize > 0)
      
      Protected *In
      If (Format & #PB_UTF8)
        *In = UTF8(Text)
      Else
        *In = Ascii(Text)
      EndIf
      Protected *InPtr = *In
      Protected *Out = AllocateMemory(TargetSize + 1)
      Protected *OutPtr = *Out
      
      Protected *DiffPtr.BYTE = *Mem + 4
      While (*DiffPtr < *Mem + MemSize)
       
        If (*DiffPtr\b = 'C') ; Copy (from source)
          *DiffPtr + 1
          PartLen = PeekL(*DiffPtr)
          *DiffPtr + 4
          CopyMemory(*InPtr, *OutPtr, PartLen)
          *InPtr  + PartLen
          *OutPtr + PartLen
       
        ElseIf (*DiffPtr\b = 'A') ; Add (from diff)
          *DiffPtr + 1
          PartLen = PeekL(*DiffPtr)
          *DiffPtr + 4
          CopyMemory(*DiffPtr, *OutPtr, PartLen)
          *DiffPtr + PartLen
          *OutPtr  + PartLen
       
        ElseIf (*DiffPtr\b = 'D') ; Delete (skip from source)
          *DiffPtr + 1
          PartLen = PeekL(*DiffPtr)
          *DiffPtr + 4
          *InPtr + PartLen
       
        Else
          ;Debug "Unknown: " + Hex(*DiffPtr\b)
          Break
        EndIf
      Wend
      ; (*OutPtr - *Out) always matches TargetSize (good!)
      Text = PeekS(*Out, TargetSize, Format)
      ; but StringByteLength(Text, ...) doesn't always match TargetSize ... ???
      ; seems to be because of $00 bytes in the output, which shortens the string
      If (*In)
        FreeMemory(*In)
      EndIf
      If (*Out)
        FreeMemory(*Out)
      EndIf
    EndIf
  EndIf
  ProcedureReturn (Text)
EndProcedure

Procedure.i ExamineSessionHistory(File.s = "")
  Protected Result.i = #False
  ClearList(SessionHistoryFile())
 
  ; Search for History file
  If (File = "")
   
    ; Try PB Prefs location
    File = GetEnvironmentVariable("PB_TOOL_Preferences")
    If (File)
      File = GetPathPart(File) + "History.db"
      If (FileSize(File) < 0)
        File = ""
      EndIf
    EndIf
   
    ; Try AppData
    If (File = "")
      File = GetUserDirectory(#PB_Directory_ProgramData)
      If (File)
        File = File + "PureBasic" + Right(File, 1) + "History.db"
        If (FileSize(File) < 0)
          File = ""
        EndIf
      EndIf
    EndIf
  EndIf
 
  ; Open Database
  If (File)
    UseSQLiteDatabase()
    Protected DB.i = OpenDatabase(#PB_Any, File, "", "")
    If (DB)
     
      ; Find all
      If (DatabaseQuery(DB, "SELECT * FROM event WHERE previous_event='0'"))
        Result = #True
        While (NextDatabaseRow(DB))
          AddElement(SessionHistoryFile())
          SessionHistoryFile()\Event_ID   = GetDatabaseString(DB, 0)
          SessionHistoryFile()\Session_ID = GetDatabaseString(DB, 1)
          SessionHistoryFile()\Filename   = GetDatabaseString(DB, 2)
          SessionHistoryFile()\Time       = GetDatabaseLong(DB, 4)
         
          Protected Format.i
          If (GetDatabaseLong(DB, 7))
            Format = #PB_UTF8 | #PB_ByteLength
          Else
            Format = #PB_Ascii
          EndIf
         
          If (FindString(SessionHistoryFile()\Filename, "::unsaved"))
            SessionHistoryFile()\Filename = "Unsaved File"
          EndIf
         
          Protected *Blob = GetDatabaseBlobBuffer(DB, 8)
          If (*Blob)
            Select (GetDatabaseLong(DB, 5))
              Case #DATA_FullZ
                Protected *Unpacked = UnpackHistoryBlob(*Blob)
                If (*Unpacked)
                  SessionHistoryFile()\Text = PeekS(*Unpacked, MemorySize(*Unpacked), Format)
                  FreeMemory(*Unpacked)
                EndIf
              Case #DATA_Full
                SessionHistoryFile()\Text = PeekS(*Blob, MemorySize(*Unpacked), Format)
            EndSelect
            FreeMemory(*Blob)
          EndIf
        Wend
        FinishDatabaseQuery(DB)
       
        ; Parse incremental code changes
        ForEach (SessionHistoryFile())
          Protected ID.s = SessionHistoryFile()\Event_ID
          While (ID)
            If (DatabaseQuery(DB, "SELECT * FROM event WHERE previous_event='" + ID + "'"))
              If (NextDatabaseRow(DB))
                SessionHistoryFile()\Time = GetDatabaseLong(DB, 4)
                ID = GetDatabaseString(DB, 0)
                *Blob = GetDatabaseBlobBuffer(DB, 8)
                If (*Blob)
                  If (GetDatabaseLong(DB, 7))
                    Format = #PB_UTF8 | #PB_ByteLength
                  Else
                    Format = #PB_Ascii
                  EndIf
                  Select (GetDatabaseLong(DB, 5))
                  
                    Case #DATA_FullZ ; Full text, zipped
                      *Unpacked = UnpackHistoryBlob(*Blob)
                      If (*Unpacked)
                        SessionHistoryFile()\Text = PeekS(*Unpacked, MemorySize(*Unpacked), Format)
                        FreeMemory(*Unpacked)
                      EndIf
                    Case #DATA_Full ; Full text
                      SessionHistoryFile()\Text = PeekS(*Blob, MemorySize(*Blob), Format)
                      
                    Case #DATA_DiffZ ; Diff, zipped
                      *Unpacked = UnpackHistoryBlob(*Blob)
                      If (*Unpacked)
                        SessionHistoryFile()\Text = ApplyHistoryChanges(SessionHistoryFile()\Text,
                            *Unpacked, MemorySize(*Unpacked), Format)
                        FreeMemory(*Unpacked)
                      EndIf
                    Case #DATA_Diff ; Diff
                      SessionHistoryFile()\Text = ApplyHistoryChanges(SessionHistoryFile()\Text,
                        *Blob, MemorySize(*Blob), Format)
                        
                  EndSelect
                  FreeMemory(*Blob)
                EndIf
              Else
                ID.s = ""
              EndIf
              FinishDatabaseQuery(DB)
            Else
              Break
            EndIf
          Wend
        Next
       
        ; Remove blank files
        ForEach SessionHistoryFile()
          If (Trim(SessionHistoryFile()\Text) = "")
            DeleteElement(SessionHistoryFile())
          EndIf
        Next
       
        ; Sort reverse-chronological
        SortStructuredList(SessionHistoryFile(), #PB_Sort_Descending,
            OffsetOf(SessionHistoryFileStruct\Time), #PB_Integer)
      EndIf
     
      CloseDatabase(DB)
    EndIf
  EndIf
  ProcedureReturn (Result)
EndProcedure

;-
;-
;- Demo Program (Search)

CompilerIf (#PB_Compiler_IsMainFile)
DisableExplicit

If (Not ExamineSessionHistory())
  MessageRequester("SessionSearch", "Could not examine session history!", #PB_MessageRequester_Error)
  End
EndIf

Procedure Search()
  Query.s = Trim(GetGadgetText(0))
  ClearGadgetItems(1)
  ForEach SessionHistoryFile()
    If ((Query = "") Or (FindString(SessionHistoryFile()\Text, Query, 1, #PB_String_NoCase)))
      AddGadgetItem(1, n, GetFilePart(SessionHistoryFile()\Filename) + #LF$ +
          FormatDate("%yyyy-%mm-%dd %hh:%ii", SessionHistoryFile()\Time))
      SetGadgetItemData(1, n, @SessionHistoryFile())
      n + 1
    EndIf
  Next
EndProcedure

Procedure ResizeCB()
  ResizeGadget(0, #PB_Ignore, #PB_Ignore, WindowWidth(0) - 10, #PB_Ignore)
  ResizeGadget(3, #PB_Ignore, #PB_Ignore, WindowWidth(0), WindowHeight(0) - GadgetHeight(0) - StatusBarHeight(0) - 15)
EndProcedure

Procedure PreviewCode()
  i = GetGadgetState(1)
  If (i >= 0)
    ChangeCurrentElement(SessionHistoryFile(), GetGadgetItemData(1, i))
    SetGadgetText(2, "")
    SetGadgetText(2, SessionHistoryFile()\Text)
  EndIf
EndProcedure

Procedure LaunchCode()
  i = GetGadgetState(1)
  If (i >= 0)
    ChangeCurrentElement(SessionHistoryFile(), GetGadgetItemData(1, i))
    j = 1
    Repeat
      File.s = GetTemporaryDirectory() +
          GetFilePart(SessionHistoryFile()\Filename, #PB_FileSystem_NoExtension) +
          "-" + Str(j) + ".pb"
      j + 1
    Until (FileSize(File) = -1)
    If CreateFile(1, File)
      WriteString(1, SessionHistoryFile()\Text)
      CloseFile(1)
      CompilerIf (#PB_Compiler_OS = #PB_OS_MacOS)
        RunProgram("open", #DQUOTE$ + File + #DQUOTE$, GetCurrentDirectory())
      CompilerElse
        RunProgram(File)
      CompilerEndIf
    EndIf
  EndIf
EndProcedure

OpenWindow(0, 0, 0, 640, 480, "SessionSearch", #PB_Window_ScreenCentered |
    #PB_Window_MinimizeGadget | #PB_Window_MaximizeGadget | #PB_Window_SizeGadget)
CreateStatusBar(0, WindowID(0))
  AddStatusBarField(#PB_Ignore)
  StatusBarText(0, 0, "Press Enter to search, Click to view code, Double-Click to launch code in PB")
StringGadget(0, 5, 5, 640, 20, "")
  ResizeGadget(0, #PB_Ignore, #PB_Ignore, 640, GadgetHeight(0, #PB_Gadget_RequiredSize))
ListIconGadget(1, 0, 20, 320, 460, "File", 145, #PB_ListIcon_FullRowSelect)
  AddGadgetColumn(1, 1, "Date/Time", 145)
EditorGadget(2, 320, 20, 320, 460, #PB_Editor_ReadOnly)
  SetGadgetFont(2, LoadFont(0, "Courier New", 10))
SplitterGadget(3, 0, GadgetHeight(0) + 10, WindowWidth(0),
    WindowHeight(0)-GadgetHeight(0), 1, 2, #PB_Splitter_Vertical | #PB_Splitter_FirstFixed)
BindEvent(#PB_Event_SizeWindow, @ResizeCB())

Search()
SetActiveGadget(0)

AddKeyboardShortcut(0, #PB_Shortcut_Return, 0)
AddKeyboardShortcut(0, #PB_Shortcut_Escape, 1)
ResizeCB()

Repeat
  Event = WaitWindowEvent()
  If (Event = #PB_Event_CloseWindow)
    Done = #True
  ElseIf (Event = #PB_Event_Menu)
    Select EventMenu()
      Case 0
        Select GetActiveGadget()
          Case 0
            Search()
          Case 1
            LaunchCode()
        EndSelect
      Case 1
        If GetGadgetText(0)
          SetGadgetText(0, "")
          Search()
        EndIf
        SetActiveGadget(0)
    EndSelect
  ElseIf (Event = #PB_Event_Gadget)
    If (EventGadget() = 1) And ((EventType() = #PB_EventType_LeftClick) Or (EventType() = #PB_EventType_Change))
      PreviewCode()
    ElseIf (EventGadget() = 1) And (EventType() = #PB_EventType_LeftDoubleClick)
      LaunchCode()
    EndIf
  EndIf
Until (Done)

CompilerEndIf
CompilerEndIf
;-
Last edited by kenmo on Wed Mar 07, 2018 3:18 pm, edited 2 times in total.
User avatar
Zebuddi123
Enthusiast
Enthusiast
Posts: 794
Joined: Wed Feb 01, 2012 3:30 pm
Location: Nottinghamshire UK
Contact:

Re: Searching session history

Post by Zebuddi123 »

@Kenmo Very nice this was one of my next projects and I was looking through the History.db 2 days ago. As I save all my History.db backups there`s all ways something in there that`s useful but you can only remember a bit of the code. It`s great as is so Thanks. :)

Now what should I do with that 3 months it would have taken me :shock: :oops: :lol: :lol: :lol:

Zebuddi.
malleo, caput, bang. Ego, comprehendunt in tempore
davido
Addict
Addict
Posts: 1890
Joined: Fri Nov 09, 2012 11:04 pm
Location: Uttoxeter, UK

Re: Searching session history

Post by davido »

I don't seem able to get this to work. :oops:
Whether I compile and run the code or make a tool, nothing happens; no window appears.

Clearly, I'm missing something here! Can anyone help?
DE AA EB
User avatar
kenmo
Addict
Addict
Posts: 1967
Joined: Tue Dec 23, 2003 3:54 am

Re: Searching session history

Post by kenmo »

Maybe find the path of your "History.db" file and pass it into ExamineSessionHistory() ... right now it just guesses a couple locations.

- Are you sure your Session History is enabled and not empty??

- Running on Windows / Mac / Linux? I only tested on Windows.


Other disclaimers:

- It assumes everything is UTF-8 encoded... it might screw up characters if you're saving files in ASCII [EDIT March 5: updated code above, I think it detects ASCII now]

- History.db uses ZIP compressed data in a SQLite database. It stores source code as one fulltext blob, then all the updates as incremental changes (insert new text, delete text, ...).
I had to figure out the format, it might not be 100% correct... Maybe Freak can give details.
freak
PureBasic Team
PureBasic Team
Posts: 5929
Joined: Fri Apr 25, 2003 5:21 pm
Location: Germany

Re: Searching session history

Post by freak »

The format is not very complicated. I think you got it mostly right. Here is the info:

Code: Select all

;
;- Editing history events
;
Enumeration
  #HISTORY_Create
  #HISTORY_Open
  #HISTORY_Close
  #HISTORY_Save
  #HISTORY_SaveAs  
  #HISTORY_Reload
  #HISTORY_Edit
EndEnumeration

;- Data types in history
;
Enumeration 1
  #DATA_Empty ; data is null. the file is empty
  #DATA_Same  ; data is null. the file is the same as on the previous event
  #DATA_Diff  ; data is a diff since the previous event 
  #DATA_Full  ; data is the full file content (new files in a session that are not empty, also used if diff is too large or if the chain of diff'ed events becomes too long)
  #DATA_DiffZ ; data is a diff (zlib compressed)
  #DATA_FullZ ; data is full file (zlib compressed)
EndEnumeration

; Session table:
;   session_id : primary key of table
;   os_id      : ID created by Session_Start() to detect dead instances. set to null for properly ended sessions
;   version    : Compiler vesion (default compiler of the IDE)
;   user       : OS user name
;   start_time : session start time
;   end_time   : session end time (0 for still running)
;   warned     : 1 if a warning was displayed for a detected dead session
;

; Event table:
;   event_id       : primary key
;   session_id     : session id
;   filename       : file name (for new files: "::unsaved::" + unique id + creation time stamp)
;   event          : one of the #HISTORY_XXX values
;   time           : event time stamp
;   type           : one of the #DATA_XXX values
;   previous_event : id of previous event for the file (if any)
;   encoding       : encoding of source file (0 for ascii, 1 for utf-8)
;   data           : file data or null, depending on type
;  

; Diff format:
;
; Beginning: <long> full size of target
;
; C<long> : copy from source
; D<long> : skip from source
; A<long><content> : add content from diff
To reconstruct the file for a specific event you have to go backward the events chain until you reach one with #DATA_Empty, #DATA_Full or #DATA_FullZ. The non-compressed data types #DATA_Diff and #DATA_Full are used mostly in debug builds, but they can appear in the real world too if compression fails on an entry.
quidquid Latine dictum sit altum videtur
User avatar
kenmo
Addict
Addict
Posts: 1967
Joined: Tue Dec 23, 2003 3:54 am

Re: Searching session history

Post by kenmo »

Great! Thank you for the official details :)
davido
Addict
Addict
Posts: 1890
Joined: Fri Nov 09, 2012 11:04 pm
Location: Uttoxeter, UK

Re: Searching session history

Post by davido »

@kenmo,
Thank you very much for your comments. :D
I later discovered that the problem is with me.
I, apparently, have a bloated History.db file and it takes some time to process, 'ere the window opening.

Apologies for wasting your time. :oops:
DE AA EB
User avatar
kenmo
Addict
Addict
Posts: 1967
Joined: Tue Dec 23, 2003 3:54 am

Re: Searching session history

Post by kenmo »

I should have warned: that code is not optimized at all!!! Just tried to get the bare minimum working :)
mestnyi
Addict
Addict
Posts: 995
Joined: Mon Nov 25, 2013 6:41 am

Re: Searching session history

Post by mestnyi »

+1
BarryG
Addict
Addict
Posts: 3292
Joined: Thu Apr 18, 2019 8:17 am

Re: Searching session history

Post by BarryG »

How do I get this Session History search tool to work? It just sits there when I run it, for over 2 minutes (then I killed it).
Post Reply