Not Responding problem - Reading the SHA1 of an 6GB file

[hoangdiemtinh]

I am new to PB.
My primary language is not US/UK. I am using Google Translate.

I'm trying to read the SHA1 of a 6GB ISO file and it's causing the Not Responding issue.
Please help.

This is code.

Code: Select all

EnableExplicit

;- Enumerations
Enumeration Window
  #mainWindow
EndEnumeration

Enumeration Gadgets
  #Btn_get_SHA1
  #String_SHA1
  #Txt_1
EndEnumeration

;- Declare
Declare Resize_mainWindow()
Declare Open_mainWindow(X = 0, Y = 0, Width = 490, Height = 70)

Procedure Resize_mainWindow()
  Protected mainWindow_WidthIni = 490, mainWindow_HeightIni = 70
  Protected ScaleX.f, ScaleY.f

  ScaleX = WindowWidth(#mainWindow) / mainWindow_WidthIni : ScaleY = WindowHeight(#mainWindow) / mainWindow_HeightIni
  ResizeGadget(#Txt_1, ScaleX * 10, ScaleY * 10, ScaleX * 90, ScaleY * 20)
  ResizeGadget(#String_SHA1, ScaleX * 110, ScaleY * 10, ScaleX * 370, ScaleY * 20)
  ResizeGadget(#Btn_get_SHA1, ScaleX * 370, ScaleY * 40, ScaleX * 110, ScaleY * 24)
EndProcedure

Procedure Open_mainWindow(X = 0, Y = 0, Width = 490, Height = 70)
  If OpenWindow(#mainWindow, X, Y, Width, Height, "get SHA1 without Not Responding Problem", #PB_Window_SystemMenu | #PB_Window_MinimizeGadget | #PB_Window_MaximizeGadget | #PB_Window_SizeGadget | #PB_Window_ScreenCentered)
    TextGadget(#Txt_1, 10, 10, 90, 20, "SHA1 output:")
    StringGadget(#String_SHA1, 110, 10, 370, 20, "......", #PB_String_ReadOnly | #ES_CENTER)
    ButtonGadget(#Btn_get_SHA1, 370, 40, 110, 24, "get SHA1")

    BindEvent(#PB_Event_SizeWindow, @Resize_mainWindow(), #mainWindow)
    PostEvent(#PB_Event_SizeWindow, #mainWindow, 0)
    ProcedureReturn #True
  EndIf
EndProcedure

Global.s vSHA1

Procedure get_SHA1(*File_Name)
  ;#BufferSize = 16384
  ;#BufferSize = 65536
  Protected *Buffer = AllocateMemory(16384)
  Protected.w readByte = 0
  Protected outSHA1.s{40}
  Protected.d t1 = ElapsedMilliseconds()
  Protected.S fn = PeekS(*File_Name)
  If (ReadFile(0, GetCurrentDirectory() + fn))
    If (*Buffer) And (StartFingerprint(0, #PB_Cipher_SHA1))
      readByte = ReadData(0, *Buffer, 16384)
      While readByte > 0
        AddFingerprintBuffer(0, *Buffer, readByte)
        readByte = ReadData(0, *Buffer, 16384) 
      Wend
      CloseFile(0)
      outSHA1 = FinishFingerprint(0)
      FreeMemory(*Buffer)
    EndIf
  Else
    Debug "File: "+ fn +" not found"
  EndIf
  vSHA1 = outSHA1
  Protected.d t2 = ElapsedMilliseconds()
  Debug "Elapsed: "+ StrF((t2-t1)/1000) + " seconds"
  Debug vSHA1
EndProcedure

Procedure.s get_SHA1_Thread(*File_Name)
  CreateThread(@get_SHA1(), *File_Name)
  ProcedureReturn
EndProcedure

;- Main Program
  Repeat
  
    Select EventWindow()
      Case #mainWindow
        Case #PB_Event_CloseWindow
          Break
        Case #PB_Event_Gadget
          Select EventGadget()
            Case #Btn_get_SHA1
              get_SHA1_Thread(@"Win11.iso")
          EndSelect
    EndSelect
    
  Until WaitWindowEvent() = #PB_Event_CloseWindow

[jassing]

the window will hang since it isn't receiving /processing messages...
since you don't actually need that, just rid of everything....
I have an old win8 laptop, did it fairly quickly.

Code: Select all

UseSHA1Fingerprint()
Global.s vSHA1
DisableDebugger

Procedure get_SHA1(*File_Name)
  ;#BufferSize = 16384
  ;#BufferSize = 65536
  Protected *Buffer = AllocateMemory(16384)
  Protected.w readByte = 0
  Protected outSHA1.s{40}
  Protected.d t1 = ElapsedMilliseconds()
  Protected.S fn = PeekS(*File_Name)
  If ReadFile(0, fn)
    If (*Buffer) And (StartFingerprint(0, #PB_Cipher_SHA1))
      readByte = ReadData(0, *Buffer, 16384)
      While readByte > 0
        AddFingerprintBuffer(0, *Buffer, readByte)
        readByte = ReadData(0, *Buffer, 16384) 
      Wend
      CloseFile(0)
      outSHA1 = FinishFingerprint(0)
      FreeMemory(*Buffer)
    EndIf
  Else
    Debug "File: "+ fn +" not found"
  EndIf
  vSHA1 = outSHA1
  Protected.d t2 = ElapsedMilliseconds()
  MessageRequester("Done","Elapsed: "+ StrF((t2-t1)/1000) + " seconds")
  SetClipboardText(vSHA1)
EndProcedure

Beep_(3000,30)
file.s=GetCurrentDirectory()+"Win11.iso"

get_SHA1(@file)
Beep_(3000,30)

You probably shouldn't be using sha1, but others will be slower.

[AZJIO]

Code: Select all

  Until WaitWindowEvent() = #PB_Event_CloseWindow

classic example of an event loop

[hoangdiemtinh]

the window will hang since it isn't receiving /processing messages...
since you don't actually need that, just rid of everything....
I have an old win8 laptop, did it fairly quickly.
...........
You probably shouldn't be using sha1, but others will be slower.

I tried copying 3 6GB files to HDD and SSD hard drives, main-window still show "Not Responding".
I find a way to fix it.

[AZJIO]

https://www.purebasic.fr/english/viewtopic.php?t=81399

[hoangdiemtinh]

https://www.purebasic.fr/english/viewtopic.php?t=81399

the window will hang since it isn't receiving /processing messages...
since you don't actually need that, just rid of everything....
I have an old win8 laptop, did it fairly quickly.
....................
You probably shouldn't be using sha1, but others will be slower.

I have revised the code like @jassing, @AZJIO.
The window is still frozen.

I need to check the SHA1 of 3 files, consecutively, each file has a capacity of 5-6GB.
I tried Thread, no success.

Is there any solution to make #Btn_get_SHA1 work independently ?

[AZJIO]

To prevent the window from freezing, you need to call the WindowEvent() function every 5-30 seconds. To calculate the checksum, you read the file in 50 MB blocks and after each 50 MB block read, call the WindowEvent() function to handle window events. Or you calculate it in a separate thread so as not to block the main loop.

[jassing]

I only suggested a straight forward approach since your example wasn't doing anything.

You need to put in a waitwindowevent() in your loop.
Avoid threads until you get the code working...
How long have you left it running?

[hoangdiemtinh]

This is the code I revised, removing Thread. Where I live, still using HDD hard drives, when I run code, the "window freezing" phenomenon still persists.
When I ran the code on the SSD, the situation improved a little, but the window freeze still appeared.

Is there any solution to completely handle this problem?
Thank you.
.

Code: Select all

EnableExplicit

Global.s vSHA1
Global.i ev_WINDOW
#SHA1_WIN8 = "SHA1 OF WIN8"
#SHA1_WIN10 = "SHA1 OF WIN10"
#SHA1_WIN11 = "SHA1 OF WIN11"

;- Enumerations
Enumeration Window
  #mainWindow
EndEnumeration

Enumeration Gadgets
  #Btn_get_SHA1
  #String_SHA1
  #Txt_1
EndEnumeration

;- Declare
Declare Resize_mainWindow()
Declare Open_mainWindow(X = 0, Y = 0, Width = 490, Height = 70)

Procedure Resize_mainWindow()
  Protected mainWindow_WidthIni = 490, mainWindow_HeightIni = 70
  Protected ScaleX.f, ScaleY.f
  
  ScaleX = WindowWidth(#mainWindow) / mainWindow_WidthIni : ScaleY = WindowHeight(#mainWindow) / mainWindow_HeightIni
  ResizeGadget(#Txt_1, ScaleX * 10, ScaleY * 10, ScaleX * 90, ScaleY * 20)
  ResizeGadget(#String_SHA1, ScaleX * 110, ScaleY * 10, ScaleX * 370, ScaleY * 20)
  ResizeGadget(#Btn_get_SHA1, ScaleX * 370, ScaleY * 40, ScaleX * 110, ScaleY * 24)
EndProcedure

Procedure Open_mainWindow(X = 0, Y = 0, Width = 490, Height = 70)
  If OpenWindow(#mainWindow, X, Y, Width, Height, "get SHA1 without Not Responding Problem", #PB_Window_SystemMenu | #PB_Window_MinimizeGadget | #PB_Window_MaximizeGadget | #PB_Window_SizeGadget | #PB_Window_ScreenCentered)
    TextGadget(#Txt_1, 10, 10, 90, 20, "SHA1 output:")
    StringGadget(#String_SHA1, 110, 10, 370, 20, "......", #PB_String_ReadOnly | #ES_CENTER)
    ButtonGadget(#Btn_get_SHA1, 370, 40, 110, 24, "get SHA1")
    
    BindEvent(#PB_Event_SizeWindow, @Resize_mainWindow(), #mainWindow)
    PostEvent(#PB_Event_SizeWindow, #mainWindow, 0)
    ProcedureReturn #True
  EndIf
EndProcedure

Procedure.s get_SHA1(fn.s)
  ;#BufferSize = 16384
  ;#BufferSize = 65536
  Protected *Buffer = AllocateMemory(16384)
  Protected.w readByte = 0
  Protected outSHA1.s{40}
  Protected.d t1 = ElapsedMilliseconds()
  If (ReadFile(0, GetCurrentDirectory() + fn))
    If (*Buffer) And (StartFingerprint(0, #PB_Cipher_SHA1))
      readByte = ReadData(0, *Buffer, 16384)
      While readByte > 0
        AddFingerprintBuffer(0, *Buffer, readByte)
        readByte = ReadData(0, *Buffer, 16384) 
      Wend
      CloseFile(0)
      outSHA1 = FinishFingerprint(0)
      FreeMemory(*Buffer)
    EndIf
  Else
    Debug "File: "+ fn +" not found"
  EndIf
  vSHA1 = outSHA1
  Protected.d t2 = ElapsedMilliseconds()
  Debug "Elapsed: "+ StrF((t2-t1)/1000) + " seconds"
  Debug vSHA1
  ProcedureReturn vSHA1
EndProcedure

Procedure.i run_get_SHA1()
  Protected.s yy
  get_SHA1("win8.iso")
  yy = #SHA1_WIN8
  If CompareMemoryString(@vSHA1, @yy, #PB_String_NoCaseAscii) ! #PB_String_Equal
    ProcedureReturn #False
  EndIf
  ;
  get_SHA1("win10.iso")
  yy = #SHA1_WIN10
  If CompareMemoryString(@vSHA1, @yy, #PB_String_NoCaseAscii) ! #PB_String_Equal
    ProcedureReturn #False
  EndIf
  ;
  get_SHA1("win11.iso")
  yy = #SHA1_WIN11
  If CompareMemoryString(@vSHA1, @yy, #PB_String_NoCaseAscii) ! #PB_String_Equal
    ProcedureReturn #False
  EndIf
  ProcedureReturn #True
EndProcedure

;- Main Program
Open_mainWindow()

Repeat
  ev_WINDOW = WaitWindowEvent()
  Select EventWindow()
    Case #mainWindow
      Select ev_WINDOW
        Case #PB_Event_CloseWindow
          Break
        Case #PB_Event_Gadget
          Select EventGadget()
            Case #Btn_get_SHA1
              If run_get_SHA1() = #False
                Continue
              EndIf
              SetGadgetText(#String_SHA1, vSHA1)
              MessageRequester("ok", "all sha1 is ok.", #PB_MessageRequester_Ok)
          EndSelect
      EndSelect
  EndSelect
ForEver

[HeX0R]

You should read and try to understand, what people are trying to tell you.
Here is your thread example in a useful (RUNNABLE!) way:

Code: Select all

CompilerIf #PB_Compiler_Thread = 0
	CompilerWarning "Enable thread safe option!"
CompilerEndIf

UseSHA1Fingerprint()
EnableExplicit

;- Enumerations
Enumeration Window
	#mainWindow
EndEnumeration

Enumeration Gadgets
	#Btn_get_SHA1
	#String_SHA1
	#Txt_1
EndEnumeration

Structure _THREAD_
	ID.i
	File$
	StopIt.i
EndStructure

Procedure Resize_mainWindow()
	Protected mainWindow_WidthIni = 490, mainWindow_HeightIni = 70
	Protected ScaleX.f, ScaleY.f
	
	ScaleX = WindowWidth(#mainWindow) / mainWindow_WidthIni : ScaleY = WindowHeight(#mainWindow) / mainWindow_HeightIni
	ResizeGadget(#Txt_1, ScaleX * 10, ScaleY * 10, ScaleX * 90, ScaleY * 20)
	ResizeGadget(#String_SHA1, ScaleX * 110, ScaleY * 10, ScaleX * 370, ScaleY * 20)
	ResizeGadget(#Btn_get_SHA1, ScaleX * 370, ScaleY * 40, ScaleX * 110, ScaleY * 24)
EndProcedure

Procedure Open_mainWindow(X = 0, Y = 0, Width = 490, Height = 70)
	If OpenWindow(#mainWindow, X, Y, Width, Height, "get SHA1 without Not Responding Problem", #PB_Window_SystemMenu | #PB_Window_MinimizeGadget | #PB_Window_MaximizeGadget | #PB_Window_SizeGadget | #PB_Window_ScreenCentered)
		TextGadget(#Txt_1, 10, 10, 90, 20, "SHA1 output:")
		StringGadget(#String_SHA1, 110, 10, 370, 20, "......", #PB_String_ReadOnly | #ES_CENTER)
		ButtonGadget(#Btn_get_SHA1, 370, 40, 110, 24, "get SHA1")
		
		BindEvent(#PB_Event_SizeWindow, @Resize_mainWindow(), #mainWindow)
		PostEvent(#PB_Event_SizeWindow, #mainWindow, 0)
		ProcedureReturn #True
	EndIf
EndProcedure

Global vSHA1$, MyThread._THREAD_

Procedure get_SHA1(*T._THREAD_)
	Protected *Buffer = AllocateMemory($100000)
	Protected readByte, t2, t1 = ElapsedMilliseconds()
	Protected outSHA1$
	
	If (ReadFile(0, *T\File$))
		If (*Buffer) And (StartFingerprint(0, #PB_Cipher_SHA1))
			While Eof(0) = 0
				readByte = ReadData(0, *Buffer, MemorySize(*Buffer))
				AddFingerprintBuffer(0, *Buffer, readByte)
				If *T\StopIt
					CloseFile(0)
					ProcedureReturn 
				EndIf
			Wend
			CloseFile(0)
			outSHA1$ = FinishFingerprint(0)
			FreeMemory(*Buffer)
		EndIf
	Else
		Debug "File: " + *T\File$ + " not found"
	EndIf
	vSHA1$ = outSHA1$
	t2     = ElapsedMilliseconds()
	*T\ID  = #Null
	Debug "Elapsed: " + StrF((t2 - t1) / 1000) + " seconds"
	Debug vSHA1$
EndProcedure

Open_mainWindow()

;- Main Program
Repeat
	Select WaitWindowEvent()
		Case #PB_Event_CloseWindow
			If MyThread\ID And IsThread(MyThread\ID)
				MyThread\StopIt = #True
				WaitThread(MyThread\ID)
			EndIf
			Break
		Case #PB_Event_Gadget
			Select EventGadget()
				Case #Btn_get_SHA1
					MyThread\File$ = OpenFileRequester("Select huge file", "", "*.*", 0)
					If MyThread\File$
						MyThread\StopIt = #Null
						MyThread\ID = CreateThread(@get_SHA1(), @MyThread)
					EndIf
			EndSelect
	EndSelect
	
ForEver

[hoangdiemtinh]

Thanks @Hex0r. I understood more about Thread.

I added 1 Clock (with Thread for it) and 1 Editor.
When running check Sha1, for 3 files, I saw the clock stopped. Why ?.
And when each file is checked sha1, immediately the results are available on editor. As a result, editor has not changed.
And the problem occurs: Not Responding.

Code: Select all

CompilerIf #PB_Compiler_Thread = 0
  CompilerWarning "Enable thread safe option!"
CompilerEndIf

UseSHA1Fingerprint()
EnableExplicit

;- Enumerations
Enumeration Window
  #mainWindow
EndEnumeration

Enumeration Gadgets
  #Btn_get_SHA1
  #String_SHA1
  #Txt_Clock
  #Txt_1
  #Edit_1
EndEnumeration

Structure _THREAD_
  ID.i
  File$
  StopIt.i
EndStructure

Declare.i update_clock(ii)

Procedure.i update_clock(ii)
  Repeat
    SetGadgetText(#Txt_Clock, FormatDate("%hh:%ii:%ss", Date()))
    Delay(500)
  ForEver
EndProcedure

Procedure Resize_mainWindow()
  Protected Window_0_WidthIni = 500, Window_0_HeightIni = 150
  Protected ScaleX.f, ScaleY.f
  
  ScaleX = WindowWidth(#mainWindow) / Window_0_WidthIni : ScaleY = WindowHeight(#mainWindow) / Window_0_HeightIni
  ResizeGadget(#Txt_1, ScaleX * 10, ScaleY * 10, ScaleX * 90, ScaleY * 25)
  ResizeGadget(#String_SHA1, ScaleX * 110, ScaleY * 10, ScaleX * 290, ScaleY * 25)
  ResizeGadget(#Btn_get_SHA1, ScaleX * 410, ScaleY * 10, ScaleX * 80, ScaleY * 25)
  ResizeGadget(#Txt_CLOCK, ScaleX * 210, ScaleY * 50, ScaleX * 60, ScaleY * 25)
  ResizeGadget(#Edit_1, ScaleX * 10, ScaleY * 80, ScaleX * 480, ScaleY * 60)
EndProcedure

Procedure Open_mainWindow(X = 0, Y = 0, Width = 500, Height = 150)
  OpenWindow(#mainWindow, X, Y, Width, Height, "get SHA1 without NOT RESPONDING problem", #PB_Window_SystemMenu | #PB_Window_MinimizeGadget | #PB_Window_MaximizeGadget | #PB_Window_SizeGadget | #PB_Window_ScreenCentered)
  TextGadget(#Txt_1, 10, 10, 90, 25, "SHA1 output:", #SS_CENTERIMAGE)
  StringGadget(#String_SHA1, 110, 10, 290, 25, "sha1...", #PB_String_ReadOnly)
  ButtonGadget(#Btn_get_SHA1, 410, 10, 80, 25, "get SHA1")
  TextGadget(#Txt_CLOCK, 210, 50, 60, 25, "8:18:18", #PB_Text_Center | #SS_CENTERIMAGE)
  EditorGadget(#Edit_1, 10, 80, 480, 60)
  AddGadgetItem(#Edit_1, -1, "Act log...")
  CreateThread(@update_clock(),0)
  BindEvent(#PB_Event_SizeWindow, @Resize_mainWindow(), #mainWindow)
  PostEvent(#PB_Event_SizeWindow, #mainWindow, 0)
EndProcedure

Global vSHA1$, MyThread._THREAD_

Procedure get_SHA1(*T._THREAD_)
  Protected *Buffer = AllocateMemory($100000)
  Protected readByte, t2, t1 = ElapsedMilliseconds()
  Protected outSHA1$
  
  If (ReadFile(0, *T\File$))
    If (*Buffer) And (StartFingerprint(0, #PB_Cipher_SHA1))
      While Eof(0) = 0
        readByte = ReadData(0, *Buffer, MemorySize(*Buffer))
        AddFingerprintBuffer(0, *Buffer, readByte)
        If *T\StopIt
          CloseFile(0)
          ProcedureReturn 
        EndIf
      Wend
      CloseFile(0)
      outSHA1$ = FinishFingerprint(0)
      FreeMemory(*Buffer)
    EndIf
  Else
    Debug "File: " + *T\File$ + " not found"
  EndIf
  vSHA1$ = outSHA1$
  t2     = ElapsedMilliseconds()
  *T\ID  = #Null
  Debug "Elapsed: " + StrF((t2 - t1) / 1000) + " seconds"
  Debug vSHA1$
EndProcedure

Open_mainWindow()

;- Main Program
Repeat
  Select WaitWindowEvent()
    Case #PB_Event_CloseWindow
      If MyThread\ID And IsThread(MyThread\ID)
        MyThread\StopIt = #True
        WaitThread(MyThread\ID)
      EndIf
      Break
    Case #PB_Event_Gadget
      Select EventGadget()
        Case #Btn_get_SHA1
          ;MyThread\File$ = OpenFileRequester("Select huge file", "", "*.*", 0)
          MyThread\File$ = GetCurrentDirectory() + "win8.iso"
          If MyThread\File$
            MyThread\StopIt = #Null
            MyThread\ID = CreateThread(@get_SHA1(), @MyThread)
          EndIf
          If IsThread(MyThread\ID)
            WaitThread(MyThread\ID)
          EndIf
          SetGadgetText(#Edit_1, "win8 sha1: " + vSHA1$)
          ;
          MyThread\File$ = GetCurrentDirectory() + "win10.iso"
          If MyThread\File$
            MyThread\StopIt = #Null
            MyThread\ID = CreateThread(@get_SHA1(), @MyThread)
          EndIf
          If IsThread(MyThread\ID)
            WaitThread(MyThread\ID)
          EndIf
          SetGadgetText(#Edit_1, "win10 sha1: " + vSHA1$)
          ;
          MyThread\File$ = GetCurrentDirectory() + "win11.iso"
          If MyThread\File$
            MyThread\StopIt = #Null
            MyThread\ID = CreateThread(@get_SHA1(), @MyThread)
          EndIf
          If IsThread(MyThread\ID)
            WaitThread(MyThread\ID)
          EndIf
          SetGadgetText(#Edit_1, "win11 sha1: " + vSHA1$)
      EndSelect
  EndSelect
  
ForEver

[infratec]

Sorry to say this, but ... you have nothing understood.

First:

You need to understand how programming in a multitasking environment is working.

In general: it is event driven, to reduce the time a program uses from the main system.

Your code should have one event loop which should always possible to receive events.
You should not block it in any way. This results in a responsive GUI.

Second:

You need to know how a thread is working and what it is.

To avoid blocking the loop you can use threads.
A thread is running on its own. But if you use WaitThread() in the main loop, you destroy this behaviour.
You simply wait and block again the main loop. The thread is meaningless in such a case.

You can send events to the main loop from your threads to decide what to do next.
But you should never use WaitThread() inside the main event loop (or you have terminated the thread 'by hand' and it needs only a few ms to
completely terminate).

I hope it is a bit more clear now. And you can see the 'bug' in your code.

[hoangdiemtinh]

Thank for all.
Special thanks to @HeX0R and @mk-soft.

I have fixed the problem.

I add Enumeration:

Code: Select all

Enumeration #PB_Event_FirstCustomValue
  #MyEvent_Sha1Start
  #MyEvent_Sha1Doing
  #MyEvent_Sha1Finished
  #MyEvent_Sha1Validate
EndEnumeration

then add to mainThread:

Code: Select all

...
PostEvent(#MyEvent_Sha1Start, #mainWindow, 0, 0, AllocateString(sdata$))
...
...
PostEvent(#MyEvent_Sha1Doing, #mainWindow, 0, 0, AllocateString(sdata$))
...

Part of the main loop processing, I keep adding Cases.

Everything worked beautifully