Searching session history
Searching session history
I'd like to be able to search session history for text. Currently I need to find some code I wrote 4 months ago, but I have no easy way of finding it. (My session history is 12 months). I'd love to just enter a constant name to search for, that would show the session file that it was in. Thank you.
Last edited by Dude on Sat Mar 03, 2018 12:24 am, edited 1 time in total.
-
- Always Here
- Posts: 6425
- Joined: Fri Oct 23, 2009 2:33 am
- Location: Wales, UK
- Contact:
Re: Searching session history
+1
IdeasVacuum
If it sounds simple, you have not grasped the complexity.
If it sounds simple, you have not grasped the complexity.
Re: Searching session history
+1 Would be a nice feature.
Here's a quick and dirty tool to search your history. Fun 2-day project Had to do a little reverse-engineering on the IDE's History.db file.
EDIT: Minor update March 5-6
Here's a quick and dirty tool to search your history. Fun 2-day project Had to do a little reverse-engineering on the IDE's History.db file.
EDIT: Minor update March 5-6
Code: Select all
; +----------------+
; | SessionHistory |
; +----------------+
; | 2018-03-03 . Creation
; | 2018-03-05 . Cleanup, ASCII/UTF-8 detection, EnableExplicit
; | 2018-03-06 . Improvements based on new Database/Diff format info
CompilerIf (Not Defined(_SessionHistory_Included, #PB_Constant))
#_SessionHistory_Included = #True
CompilerIf (#PB_Compiler_IsMainFile)
EnableExplicit
CompilerEndIf
;-
;- Constants
Enumeration 1
#DATA_Empty ; data is null. the file is empty
#DATA_Same ; data is null. the file is the same as on the previous event
#DATA_Diff ; data is a diff since the previous event
#DATA_Full ; data is the full file content (new files in a session that are not empty, also used if diff is too large or if the chain of diff'ed events becomes too long)
#DATA_DiffZ ; data is a diff (zlib compressed)
#DATA_FullZ ; data is full file (zlib compressed)
EndEnumeration
;-
;- Structures
Structure SessionHistoryFileStruct
Filename.s
Event_ID.s
Session_ID.s
Text.s
Time.i
EndStructure
;-
;- Globals
Global NewList SessionHistoryFile.SessionHistoryFileStruct()
;-
;- Procedures
Procedure.i UnpackHistoryBlob(*Blob)
Protected *Result = #Null
If (*Blob)
Protected UnpackedSize.i = PeekL(*Blob)
If (UnpackedSize > 0)
*Result = AllocateMemory(UnpackedSize, #PB_Memory_NoClear)
If (*Result)
UseZipPacker()
If (UncompressMemory(*Blob + 4, MemorySize(*Blob) - 4, *Result, UnpackedSize))
; OK
Else
FreeMemory(*Result)
*Result = #Null
EndIf
EndIf
EndIf
EndIf
ProcedureReturn (*Result)
EndProcedure
Procedure.i GetDatabaseBlobBuffer(Database.i, Column.i)
Protected *Result = #Null
Protected BlobSize.i = DatabaseColumnSize(Database, Column)
If (BlobSize > 0)
*Result = AllocateMemory(BlobSize, #PB_Memory_NoClear)
If (*Result)
If (GetDatabaseBlob(Database, Column, *Result, BlobSize))
; OK
Else
FreeMemory(*Result)
*Result = #Null
EndIf
EndIf
EndIf
ProcedureReturn (*Result)
EndProcedure
Procedure.s ApplyHistoryChanges(Text.s, *Mem, MemSize.i, Format.i)
If (*Mem And (MemSize > 0))
Protected PartLen.i
Protected TargetSize.i = PeekL(*Mem)
If (TargetSize > 0)
Protected *In
If (Format & #PB_UTF8)
*In = UTF8(Text)
Else
*In = Ascii(Text)
EndIf
Protected *InPtr = *In
Protected *Out = AllocateMemory(TargetSize + 1)
Protected *OutPtr = *Out
Protected *DiffPtr.BYTE = *Mem + 4
While (*DiffPtr < *Mem + MemSize)
If (*DiffPtr\b = 'C') ; Copy (from source)
*DiffPtr + 1
PartLen = PeekL(*DiffPtr)
*DiffPtr + 4
CopyMemory(*InPtr, *OutPtr, PartLen)
*InPtr + PartLen
*OutPtr + PartLen
ElseIf (*DiffPtr\b = 'A') ; Add (from diff)
*DiffPtr + 1
PartLen = PeekL(*DiffPtr)
*DiffPtr + 4
CopyMemory(*DiffPtr, *OutPtr, PartLen)
*DiffPtr + PartLen
*OutPtr + PartLen
ElseIf (*DiffPtr\b = 'D') ; Delete (skip from source)
*DiffPtr + 1
PartLen = PeekL(*DiffPtr)
*DiffPtr + 4
*InPtr + PartLen
Else
;Debug "Unknown: " + Hex(*DiffPtr\b)
Break
EndIf
Wend
; (*OutPtr - *Out) always matches TargetSize (good!)
Text = PeekS(*Out, TargetSize, Format)
; but StringByteLength(Text, ...) doesn't always match TargetSize ... ???
; seems to be because of $00 bytes in the output, which shortens the string
If (*In)
FreeMemory(*In)
EndIf
If (*Out)
FreeMemory(*Out)
EndIf
EndIf
EndIf
ProcedureReturn (Text)
EndProcedure
Procedure.i ExamineSessionHistory(File.s = "")
Protected Result.i = #False
ClearList(SessionHistoryFile())
; Search for History file
If (File = "")
; Try PB Prefs location
File = GetEnvironmentVariable("PB_TOOL_Preferences")
If (File)
File = GetPathPart(File) + "History.db"
If (FileSize(File) < 0)
File = ""
EndIf
EndIf
; Try AppData
If (File = "")
File = GetUserDirectory(#PB_Directory_ProgramData)
If (File)
File = File + "PureBasic" + Right(File, 1) + "History.db"
If (FileSize(File) < 0)
File = ""
EndIf
EndIf
EndIf
EndIf
; Open Database
If (File)
UseSQLiteDatabase()
Protected DB.i = OpenDatabase(#PB_Any, File, "", "")
If (DB)
; Find all
If (DatabaseQuery(DB, "SELECT * FROM event WHERE previous_event='0'"))
Result = #True
While (NextDatabaseRow(DB))
AddElement(SessionHistoryFile())
SessionHistoryFile()\Event_ID = GetDatabaseString(DB, 0)
SessionHistoryFile()\Session_ID = GetDatabaseString(DB, 1)
SessionHistoryFile()\Filename = GetDatabaseString(DB, 2)
SessionHistoryFile()\Time = GetDatabaseLong(DB, 4)
Protected Format.i
If (GetDatabaseLong(DB, 7))
Format = #PB_UTF8 | #PB_ByteLength
Else
Format = #PB_Ascii
EndIf
If (FindString(SessionHistoryFile()\Filename, "::unsaved"))
SessionHistoryFile()\Filename = "Unsaved File"
EndIf
Protected *Blob = GetDatabaseBlobBuffer(DB, 8)
If (*Blob)
Select (GetDatabaseLong(DB, 5))
Case #DATA_FullZ
Protected *Unpacked = UnpackHistoryBlob(*Blob)
If (*Unpacked)
SessionHistoryFile()\Text = PeekS(*Unpacked, MemorySize(*Unpacked), Format)
FreeMemory(*Unpacked)
EndIf
Case #DATA_Full
SessionHistoryFile()\Text = PeekS(*Blob, MemorySize(*Unpacked), Format)
EndSelect
FreeMemory(*Blob)
EndIf
Wend
FinishDatabaseQuery(DB)
; Parse incremental code changes
ForEach (SessionHistoryFile())
Protected ID.s = SessionHistoryFile()\Event_ID
While (ID)
If (DatabaseQuery(DB, "SELECT * FROM event WHERE previous_event='" + ID + "'"))
If (NextDatabaseRow(DB))
SessionHistoryFile()\Time = GetDatabaseLong(DB, 4)
ID = GetDatabaseString(DB, 0)
*Blob = GetDatabaseBlobBuffer(DB, 8)
If (*Blob)
If (GetDatabaseLong(DB, 7))
Format = #PB_UTF8 | #PB_ByteLength
Else
Format = #PB_Ascii
EndIf
Select (GetDatabaseLong(DB, 5))
Case #DATA_FullZ ; Full text, zipped
*Unpacked = UnpackHistoryBlob(*Blob)
If (*Unpacked)
SessionHistoryFile()\Text = PeekS(*Unpacked, MemorySize(*Unpacked), Format)
FreeMemory(*Unpacked)
EndIf
Case #DATA_Full ; Full text
SessionHistoryFile()\Text = PeekS(*Blob, MemorySize(*Blob), Format)
Case #DATA_DiffZ ; Diff, zipped
*Unpacked = UnpackHistoryBlob(*Blob)
If (*Unpacked)
SessionHistoryFile()\Text = ApplyHistoryChanges(SessionHistoryFile()\Text,
*Unpacked, MemorySize(*Unpacked), Format)
FreeMemory(*Unpacked)
EndIf
Case #DATA_Diff ; Diff
SessionHistoryFile()\Text = ApplyHistoryChanges(SessionHistoryFile()\Text,
*Blob, MemorySize(*Blob), Format)
EndSelect
FreeMemory(*Blob)
EndIf
Else
ID.s = ""
EndIf
FinishDatabaseQuery(DB)
Else
Break
EndIf
Wend
Next
; Remove blank files
ForEach SessionHistoryFile()
If (Trim(SessionHistoryFile()\Text) = "")
DeleteElement(SessionHistoryFile())
EndIf
Next
; Sort reverse-chronological
SortStructuredList(SessionHistoryFile(), #PB_Sort_Descending,
OffsetOf(SessionHistoryFileStruct\Time), #PB_Integer)
EndIf
CloseDatabase(DB)
EndIf
EndIf
ProcedureReturn (Result)
EndProcedure
;-
;-
;- Demo Program (Search)
CompilerIf (#PB_Compiler_IsMainFile)
DisableExplicit
If (Not ExamineSessionHistory())
MessageRequester("SessionSearch", "Could not examine session history!", #PB_MessageRequester_Error)
End
EndIf
Procedure Search()
Query.s = Trim(GetGadgetText(0))
ClearGadgetItems(1)
ForEach SessionHistoryFile()
If ((Query = "") Or (FindString(SessionHistoryFile()\Text, Query, 1, #PB_String_NoCase)))
AddGadgetItem(1, n, GetFilePart(SessionHistoryFile()\Filename) + #LF$ +
FormatDate("%yyyy-%mm-%dd %hh:%ii", SessionHistoryFile()\Time))
SetGadgetItemData(1, n, @SessionHistoryFile())
n + 1
EndIf
Next
EndProcedure
Procedure ResizeCB()
ResizeGadget(0, #PB_Ignore, #PB_Ignore, WindowWidth(0) - 10, #PB_Ignore)
ResizeGadget(3, #PB_Ignore, #PB_Ignore, WindowWidth(0), WindowHeight(0) - GadgetHeight(0) - StatusBarHeight(0) - 15)
EndProcedure
Procedure PreviewCode()
i = GetGadgetState(1)
If (i >= 0)
ChangeCurrentElement(SessionHistoryFile(), GetGadgetItemData(1, i))
SetGadgetText(2, "")
SetGadgetText(2, SessionHistoryFile()\Text)
EndIf
EndProcedure
Procedure LaunchCode()
i = GetGadgetState(1)
If (i >= 0)
ChangeCurrentElement(SessionHistoryFile(), GetGadgetItemData(1, i))
j = 1
Repeat
File.s = GetTemporaryDirectory() +
GetFilePart(SessionHistoryFile()\Filename, #PB_FileSystem_NoExtension) +
"-" + Str(j) + ".pb"
j + 1
Until (FileSize(File) = -1)
If CreateFile(1, File)
WriteString(1, SessionHistoryFile()\Text)
CloseFile(1)
CompilerIf (#PB_Compiler_OS = #PB_OS_MacOS)
RunProgram("open", #DQUOTE$ + File + #DQUOTE$, GetCurrentDirectory())
CompilerElse
RunProgram(File)
CompilerEndIf
EndIf
EndIf
EndProcedure
OpenWindow(0, 0, 0, 640, 480, "SessionSearch", #PB_Window_ScreenCentered |
#PB_Window_MinimizeGadget | #PB_Window_MaximizeGadget | #PB_Window_SizeGadget)
CreateStatusBar(0, WindowID(0))
AddStatusBarField(#PB_Ignore)
StatusBarText(0, 0, "Press Enter to search, Click to view code, Double-Click to launch code in PB")
StringGadget(0, 5, 5, 640, 20, "")
ResizeGadget(0, #PB_Ignore, #PB_Ignore, 640, GadgetHeight(0, #PB_Gadget_RequiredSize))
ListIconGadget(1, 0, 20, 320, 460, "File", 145, #PB_ListIcon_FullRowSelect)
AddGadgetColumn(1, 1, "Date/Time", 145)
EditorGadget(2, 320, 20, 320, 460, #PB_Editor_ReadOnly)
SetGadgetFont(2, LoadFont(0, "Courier New", 10))
SplitterGadget(3, 0, GadgetHeight(0) + 10, WindowWidth(0),
WindowHeight(0)-GadgetHeight(0), 1, 2, #PB_Splitter_Vertical | #PB_Splitter_FirstFixed)
BindEvent(#PB_Event_SizeWindow, @ResizeCB())
Search()
SetActiveGadget(0)
AddKeyboardShortcut(0, #PB_Shortcut_Return, 0)
AddKeyboardShortcut(0, #PB_Shortcut_Escape, 1)
ResizeCB()
Repeat
Event = WaitWindowEvent()
If (Event = #PB_Event_CloseWindow)
Done = #True
ElseIf (Event = #PB_Event_Menu)
Select EventMenu()
Case 0
Select GetActiveGadget()
Case 0
Search()
Case 1
LaunchCode()
EndSelect
Case 1
If GetGadgetText(0)
SetGadgetText(0, "")
Search()
EndIf
SetActiveGadget(0)
EndSelect
ElseIf (Event = #PB_Event_Gadget)
If (EventGadget() = 1) And ((EventType() = #PB_EventType_LeftClick) Or (EventType() = #PB_EventType_Change))
PreviewCode()
ElseIf (EventGadget() = 1) And (EventType() = #PB_EventType_LeftDoubleClick)
LaunchCode()
EndIf
EndIf
Until (Done)
CompilerEndIf
CompilerEndIf
;-
Last edited by kenmo on Wed Mar 07, 2018 3:18 pm, edited 2 times in total.
- Zebuddi123
- Enthusiast
- Posts: 794
- Joined: Wed Feb 01, 2012 3:30 pm
- Location: Nottinghamshire UK
- Contact:
Re: Searching session history
@Kenmo Very nice this was one of my next projects and I was looking through the History.db 2 days ago. As I save all my History.db backups there`s all ways something in there that`s useful but you can only remember a bit of the code. It`s great as is so Thanks.
Now what should I do with that 3 months it would have taken me
Zebuddi.
Now what should I do with that 3 months it would have taken me
Zebuddi.
malleo, caput, bang. Ego, comprehendunt in tempore
Re: Searching session history
I don't seem able to get this to work.
Whether I compile and run the code or make a tool, nothing happens; no window appears.
Clearly, I'm missing something here! Can anyone help?
Whether I compile and run the code or make a tool, nothing happens; no window appears.
Clearly, I'm missing something here! Can anyone help?
DE AA EB
Re: Searching session history
Maybe find the path of your "History.db" file and pass it into ExamineSessionHistory() ... right now it just guesses a couple locations.
- Are you sure your Session History is enabled and not empty??
- Running on Windows / Mac / Linux? I only tested on Windows.
Other disclaimers:
- It assumes everything is UTF-8 encoded... it might screw up characters if you're saving files in ASCII [EDIT March 5: updated code above, I think it detects ASCII now]
- History.db uses ZIP compressed data in a SQLite database. It stores source code as one fulltext blob, then all the updates as incremental changes (insert new text, delete text, ...).
I had to figure out the format, it might not be 100% correct... Maybe Freak can give details.
- Are you sure your Session History is enabled and not empty??
- Running on Windows / Mac / Linux? I only tested on Windows.
Other disclaimers:
- It assumes everything is UTF-8 encoded... it might screw up characters if you're saving files in ASCII [EDIT March 5: updated code above, I think it detects ASCII now]
- History.db uses ZIP compressed data in a SQLite database. It stores source code as one fulltext blob, then all the updates as incremental changes (insert new text, delete text, ...).
I had to figure out the format, it might not be 100% correct... Maybe Freak can give details.
Re: Searching session history
The format is not very complicated. I think you got it mostly right. Here is the info:
To reconstruct the file for a specific event you have to go backward the events chain until you reach one with #DATA_Empty, #DATA_Full or #DATA_FullZ. The non-compressed data types #DATA_Diff and #DATA_Full are used mostly in debug builds, but they can appear in the real world too if compression fails on an entry.
Code: Select all
;
;- Editing history events
;
Enumeration
#HISTORY_Create
#HISTORY_Open
#HISTORY_Close
#HISTORY_Save
#HISTORY_SaveAs
#HISTORY_Reload
#HISTORY_Edit
EndEnumeration
;- Data types in history
;
Enumeration 1
#DATA_Empty ; data is null. the file is empty
#DATA_Same ; data is null. the file is the same as on the previous event
#DATA_Diff ; data is a diff since the previous event
#DATA_Full ; data is the full file content (new files in a session that are not empty, also used if diff is too large or if the chain of diff'ed events becomes too long)
#DATA_DiffZ ; data is a diff (zlib compressed)
#DATA_FullZ ; data is full file (zlib compressed)
EndEnumeration
; Session table:
; session_id : primary key of table
; os_id : ID created by Session_Start() to detect dead instances. set to null for properly ended sessions
; version : Compiler vesion (default compiler of the IDE)
; user : OS user name
; start_time : session start time
; end_time : session end time (0 for still running)
; warned : 1 if a warning was displayed for a detected dead session
;
; Event table:
; event_id : primary key
; session_id : session id
; filename : file name (for new files: "::unsaved::" + unique id + creation time stamp)
; event : one of the #HISTORY_XXX values
; time : event time stamp
; type : one of the #DATA_XXX values
; previous_event : id of previous event for the file (if any)
; encoding : encoding of source file (0 for ascii, 1 for utf-8)
; data : file data or null, depending on type
;
; Diff format:
;
; Beginning: <long> full size of target
;
; C<long> : copy from source
; D<long> : skip from source
; A<long><content> : add content from diff
quidquid Latine dictum sit altum videtur
Re: Searching session history
Great! Thank you for the official details
Re: Searching session history
@kenmo,
Thank you very much for your comments.
I later discovered that the problem is with me.
I, apparently, have a bloated History.db file and it takes some time to process, 'ere the window opening.
Apologies for wasting your time.
Thank you very much for your comments.
I later discovered that the problem is with me.
I, apparently, have a bloated History.db file and it takes some time to process, 'ere the window opening.
Apologies for wasting your time.
DE AA EB
Re: Searching session history
I should have warned: that code is not optimized at all!!! Just tried to get the bare minimum working
Re: Searching session history
How do I get this Session History search tool to work? It just sits there when I run it, for over 2 minutes (then I killed it).