Basic artificial neural network evolution: cars on a track

Advanced game related topics
User avatar
Fig
Enthusiast
Enthusiast
Posts: 351
Joined: Thu Apr 30, 2009 5:23 pm
Location: Côtes d'Azur, France

Basic artificial neural network evolution: cars on a track

Post by Fig »

These cars learn to drive on a track. They have a number of laser telemeter (kind of) and a neural network. (2-3 layers depend how you count)
Genetic algorithm is so-so. Fitness function is not very good (time on track) and i am not sure averaging weights when they reproduct is relevant.

Anyway, it should work after a while, a car will do all the track by itself.
You can change number of laser (it change the number of neurons as well), distance they can detect and mutationrate by constants.

Next step is backpropagation in the library.
Actually there is a small prog at the end of the lib allowing to display the neural network.

Save as Dll "NeuroNetLib"

Code: Select all

;create a new neural net with nblayer deep (inputs include as 1 layer)
ProcedureDLL CreateNeuroNet(nbLayer.i)
   *net=0
   size=SizeOf(*net)
   *net=AllocateMemory(size+size+size*(nblayer+1)) 
   PokeI(*net,nblayer)
   PokeI(*net+size,1)
   ;*net => nblayer
   ;*net+size => next layer to fill 
   ;*net+size*2 => pointer to the  first layer *net+size*3 => pointer to the second layer etc...
   ProcedureReturn *net
EndProcedure

;specify composition of each layer
ProcedureDLL AddNeuroInputs(*net,NbInputs.i)
   ;currentlayer start at 1 (first)
   size.i=SizeOf(*net)
   sizedouble.i=8
   nblayer.i=PeekI(*net)
   *layer=AllocateMemory(size*3+sizedouble*NbInputs)
   *startOutput=*layer+size*3
   PokeI(*layer,NbInputs)
   PokeI(*layer+size,1)
   PokeI(*layer+size*2,*startOutput)                                     
   PokeI(*net+2*size,*layer)
   ProcedureReturn
EndProcedure


;specify composition of each layer
ProcedureDLL AddNeuroLayer(*net,NbNeuron.i)
   size.i=SizeOf(*net)
   sizedouble.i=8
   nblayer.i=PeekI(*net)
   currentlayer.i=PeekI(*net+size):If currentlayer>nblayer:ProcedureReturn 0:EndIf
   *prevLayer=PeekI(*net+size+size*currentlayer)
   NbNeuronPrevOutput.i=PeekI(*prevLayer)
   
   output=sizedouble*NbNeuron
   weight=sizedouble*NbNeuron*NbNeuronPrevOutput
   bias=sizedouble*NbNeuron
   
   *layer=AllocateMemory(size*3+output+weight+bias)
   *startOutput=*layer+size*3+weight+bias
   PokeI(*layer,NbNeuron)
   PokeI(*layer+size,NbNeuronPrevOutput)
   PokeI(*layer+size*2,*startOutput)
   
   
   PokeI(*net+size,currentlayer+1)
   PokeI(*net+2*size+size*currentlayer,*layer)
   ProcedureReturn currentlayer
EndProcedure

;initiation weights With randoms values
ProcedureDLL InitRandomWeightNeuroNet(*net)
   size=SizeOf(*net)
   sizedouble.i=8
   nblayer.i=PeekI(*net)
   For i=1 To nblayer
      *layer=PeekI(*net+2*size+size*i)
      nbneuron=PeekI(*layer)
      nbprevoutput=PeekI(*layer+size)
      For t=0 To nbneuron*(nbprevoutput+1)
         PokeD(*layer+size*3+sizedouble*t,(Random(200)-100)/100) ;write random weight between -1;+1
      Next t  
   Next i   
EndProcedure

ProcedureDLL.d ReadNeuronOutput(*net,layer.i,neuron.i)
   size=SizeOf(*net)
   sizedouble.i=8
   *layer=PeekI(*net+2*size+size*layer)
   *startoutput=PeekI(*layer+size*2)
   value.d=PeekD(*startoutput+(neuron-1)*sizedouble)
   ProcedureReturn value
EndProcedure

ProcedureDLL DeleteNeuroNet(*net)
   size=SizeOf(*net)
   nblayer=PeekI(*net)
   If nblayer=0:ProcedureReturn 1:EndIf
   For i=1 To nblayer
      *layer=PeekI(*net+size*i)
      FreeMemory(*layer)
   Next i
   FreeMemory(*net)
   ProcedureReturn 1
EndProcedure

;list activation functions https://en.wikipedia.org/wiki/Activation_function
;here, sigmoïd
ProcedureDLL.d activation(output.d)
   output.d=1/(1+Exp(-output))
   ProcedureReturn output
EndProcedure   

;calculate outputs
ProcedureDLL Propagate(*net)
   size=SizeOf(*net)
   sizedouble.i=8
   nblayer.i=PeekI(*net)
   For i=1 To nblayer
      *layer=PeekI(*net+2*size+size*i)
      nbneuron=PeekI(*layer)
      nbprevoutput=PeekI(*layer+size)
      *startoutput=PeekI(*layer+size*2)
      *currentweight=*layer+size*3
      *prevLayer=PeekI(*net+2*size+size*(i-1))
      *prevstartoutput=PeekI(*prevLayer+size*2)
      For j=0 To nbneuron-1
         output.d=0
         For k=0 To nbprevoutput
            input.d=PeekD(*prevstartoutput+k*sizedouble)
            ;bias
            If k=nbprevoutput:input=1:EndIf
            weight.d=PeekD(*currentweight+k*sizedouble+j*(nbprevoutput+1)*sizedouble)
            output.d=output+weight*input
         Next k
         PokeD(*startoutput+j*sizedouble,activation(output))
      Next j
   Next i   
EndProcedure

;define inputs
ProcedureDLL SetInput(*net,Input.i,value.d)
   size=SizeOf(*net)
   sizedouble.i=8
   *layer=PeekI(*net+size+size)
   *startoutput=PeekI(*layer+size*2)
   PokeD(*startoutput+(Input-1)*sizedouble,value)
   ProcedureReturn 1
EndProcedure


; ;program To visualize the neural net
; Procedure.d distance(x1,y1,x2,y2,x0,y0)
;    dist.d=Abs((y2-y1)*x0-(x2-x1)*y0+x2*y1-y2*x1)/Sqr((y2-y1)*(y2-y1)+(x2-x1)*(x2-x1))
;    ProcedureReturn dist
; EndProcedure
; 
; If InitSprite() = 0 Or InitKeyboard() = 0 Or InitMouse() = 0 Or OpenWindow(0, 0, 0, 800, 600, "Neural Network Test", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)=0 Or OpenWindowedScreen(WindowID(0),0,0,800,600,0,0,0,#PB_Screen_NoSynchronization )=0
;    MessageRequester("Error", "Can't open the sprite system", 0)
;    End
; EndIf
; #mouse=0
; CreateSprite(#mouse,16,16)
; StartDrawing(SpriteOutput(#mouse))
; Box(0,0,16,16,#Red)
; StopDrawing()
; 
; *net=CreateNeuroNet(1)
; AddNeuroInputs(*net,2)
; AddNeuroLayer(*net,2)
; InitRandomWeightNeuroNet(*net)
; SetInput(*net,1,0.1)
; 
; Propagate(*net)
; Repeat
;    Repeat:Until WindowEvent()=0
;    FlipBuffers()
;    ClearScreen(#Black)
;    ExamineKeyboard()
;    ExamineMouse()
;    xm=MouseX()
;    ym=MouseY()
;    
;    size=SizeOf(*net)
;    StartDrawing(ScreenOutput())
;    ;display inputs
;    *input=PeekI(*net+size*2)
;    nbinput.i=PeekI(*input) 
;    y.f=300-(nbinput/2)*100
;    For i=1 To nbinput
;       Circle(100,i*80+y,30)  
;       Circle(100,i*80+y,28,#Black)  
;       DrawText(84,i*80+y-7,StrD(readneuronoutput(*net,0,i),3)) 
;    Next i
;    nblayer=PeekI(*net)
;    min.d=10000
;    a1=0:a2=0:a3=0
;    ;display layers
;    For i=1 To nblayer
;       *layer=PeekI(*net+2*size+size*i)
;       nbneuron=PeekI(*layer)
;       nbprevneuron=PeekI(*layer+size)
;       y.f=300-(nbneuron/2)*100
;       yy.f=300-(nbprevneuron/2)*100
;       For j=1 To nbneuron
;          For t=1 To nbprevneuron
;             LineXY(100+i*100,j*80+y-7,100+(i-1)*100,t*80+yy-7)
;             d.d=distance(100+i*100,j*80+y-7,100+(i-1)*100,t*80+yy-7,xm,ym)
;             If d<min:min=d:a1=i:a2=j:a3=t:a4=y:a5=yy:EndIf
;          Next t   
;          Circle(100+i*100,j*80+y,30)  
;          Circle(100+i*100,j*80+y,28,#Black)  
;          ;DrawText(84+i*100,j*80+y-7,StrD(readneuroBias(*net,i,j),3))
;          DrawText(84+i*100,j*80+y-7,StrD(readneuronoutput(*net,i,j),3))
;       Next j
;    Next i
;    LineXY(100+a1*100,a2*80+a4-7,100+(a1-1)*100,a3*80+a5-7,#Red)
;    ;DrawText(xm,ym,StrD(readneuroWeight(*net,i,j),3))
; 
;    StopDrawing()
;       DisplaySprite(#mouse,xm,ym)
; 
; Until KeyboardPushed(#PB_Key_Escape)

Code: Select all

If InitSprite() = 0 Or InitKeyboard() = 0 Or InitMouse() = 0 Or OpenWindow(0, 0, 0, 800, 600, "Smart Rockets", #PB_Window_SystemMenu | #PB_Window_ScreenCentered)=0 Or OpenWindowedScreen(WindowID(0),0,0,800,600,0,0,0,#PB_Screen_NoSynchronization )=0
   MessageRequester("Error", "Can't open the sprite system", 0)
   End
EndIf
UsePNGImageDecoder()
Import "NeuroNetLib.lib"
   CreateNeuroNet(nbLayer.i)
   AddNeuroInputs(*net,Nbinputs.i)
   AddNeuroLayer(*net,NbNeuron.i)
   InitRandomWeightNeuroNet(*net)
   ReadNeuronOutput.d(*net,layer.i,neuron.i)
   DeleteNeuroNet(*net)
   Propagate(*net)
   SetInput(*net,RowInput.i,value.d)
   activation.d(output.d)
EndImport

#car=0:#track=1      ;sprites
#PopulationSize=25   ;number of cars
#laserNumber= 5      ;number of lasertelemeter per car     
#rayonmax=60       ;depth of lasertelemeters
#MutationRate=10     ;mutation rate
#startx=428:#starty=275 ;starts coord
CreateSprite(#car,32,32,#PB_Sprite_PixelCollision)
LoadSprite(#track,"circuit.png",#PB_Sprite_PixelCollision)
StartDrawing(SpriteOutput(#car))
LineXY(0,5,31,10,#Red)
LineXY(31,10,31,20,#Red)
LineXY(31,20,0,25,#Red)
LineXY(0,25,0,5,#Red)
FillArea(15,15,#Red)
StopDrawing()
Structure vector
   x.f
   y.f
EndStructure
Structure captor
   angle.i ;angle du laser en degré
   x.i     ;point detecté
   y.i     ;point detecté
   dist.f  ;distance mesurée
EndStructure
Structure physic
   pos.vector
   vel.vector
   acc.vector
   rotation.f
   fitness.i
   dead.b
   couleur.i
   *net
   laser.captor[#lasernumber]
EndStructure
Global Dim car.physic(#PopulationSize)
Global PopulationAlive.i
Global Maxfitness.i
Global Fitness.i
Global Gen.i

Procedure Offspring(*net1,*net2,*net3)
   size=SizeOf(*net1)
   sizefl.i=8
   nblayer.i=PeekI(*net1)
   For i=1 To nblayer
      *layer1=PeekI(*net1+size+size*i)
      *layer2=PeekI(*net2+size+size*i)
      *layer3=PeekI(*net3+size+size*i)
      nbneuron=PeekI(*layer1)
      nbprevoutput=PeekI(*layer1+size)
      For t=0 To nbneuron*nbprevoutput-1
         weight1.d=PeekD(*layer1+size*3+sizefl*t)
         weight2.d=PeekD(*layer2+size*3+sizefl*t)
         rand=Random(2)
         If rand=0
            weight3.d=weight1
         ElseIf rand=1
            weight3=weight2
         Else
            weight3=(weight1+weight2)/2
         EndIf
         ;mutation
         If Random(100)<#MutationRate:weight3=(Random(200)-100)/100:EndIf
         PokeD(*layer3+size*3+sizefl*t,weight3)
      Next t  
   Next i   
EndProcedure 

Procedure UpdatePhysic()
   Fitness+1
   For i=0 To #PopulationSize-1
      If car(i)\dead:Continue:EndIf
      right.d=ReadNeuronOutput(car(i)\net,2,1)
      left.d=ReadNeuronOutput(car(i)\net,2,2)
      center.d=ReadNeuronOutput(car(i)\net,2,3)
      If left>right And left>center
         car(i)\rotation+2
      ElseIf right>left And right>center
         car(i)\rotation-2
      EndIf
      
      car(i)\pos\x+Cos(Radian(car(i)\rotation))
      car(i)\pos\y+Sin(Radian(car(i)\rotation))
      
      x=SpriteWidth(#car)/2+car(i)\pos\x
      y=SpriteHeight(#car)/2+car(i)\pos\y
      ;test telemeterlasers
      StartDrawing(SpriteOutput(#track))
      For t=0 To #laserNumber-1
         rayon=0:angle.d=Radian(car(i)\laser[t]\angle+car(i)\rotation)
         angle1.d=Cos(angle)
         angle2.d=Sin(angle)
         Repeat
         xx.i=(x+rayon*angle1)
         yy.i=(y+rayon*angle2)
         rayon+1
         Until Point(xx,yy)=RGB(32,0,150) Or rayon>#rayonmax Or xx<0 Or yy<0 Or xx>=800 Or yy>=600
         car(i)\laser[t]\dist=rayon
         ;update neural network inputs
         SetInput(car(i)\net,t+1,rayon/#rayonmax)
      Next t
      StopDrawing()

      If SpritePixelCollision(#car,car(i)\pos\x,car(i)\pos\y,#track,0,0)
         car(i)\dead=1
         PopulationAlive-1
         car(i)\fitness=Fitness
         ;Debug "collision "+Str(i)
      EndIf
      If fitness=150 And Abs(car(i)\pos\x-#startx)<10 And Abs(car(i)\pos\y-#starty)<10
         car(i)\dead=1
         PopulationAlive-1
         car(i)\fitness=Fitness
         ;Debug "tourne en rond "+Str(i)
      EndIf
      If fitness>5000
         car(i)\dead=1
         PopulationAlive-1
         car(i)\fitness=Fitness
      EndIf      
      If PopulationAlive=0:Maxfitness=fitness:EndIf
      Propagate(car(i)\net)
   Next i
EndProcedure

;(r)initialise la population
Procedure ResetPopulation()
   Gen+1
   PopulationAlive=#PopulationSize
      If car(0)\net<>0
         SortStructuredArray(car(),#PB_Sort_Descending,OffsetOf(physic\fitness),TypeOf(physic\fitness))
         ;fill the pool
         NewList pool.i()
         Dim carcopy.physic(#PopulationSize)
         CopyArray(car(),carcopy())
         For i=0 To #PopulationSize-1
            rat.d=(car(i)\fitness/Maxfitness)*100
            rate.i=rat
            For t=1 To rate
               AddElement(pool())
               pool()=i
            Next t
         Next i
         total.i=ListSize(pool())
         ;keep 2 best
         For son=2 To #PopulationSize-2
            SelectElement(pool(),Random(total-1))
            father=pool()
            SelectElement(pool(),Random(total-1))
            mother=pool()
            Offspring(carcopy(father)\net,carcopy(mother)\net,car(son)\net)
         Next son
         ;one totaly random
         InitRandomWeightNeuroNet(car(#PopulationSize-1)\net)
      EndIf
   For i=0 To #PopulationSize-1
      If car(i)\net=0
         car(i)\net=CreateNeuroNet(2)
         AddNeuroInputs(car(i)\net,#laserNumber) ;nb laser inputs
         AddNeuroLayer(car(i)\net,#laserNumber) ;hidden layer nb laser neurons
         AddNeuroLayer(car(i)\net,3)            ;5 outputs: left, right , center
         a=180/(#laserNumber+1)
         b=a-90
         For t=0 To #laserNumber-1
            car(i)\laser[t]\angle=b
            b+a
         Next t
         InitRandomWeightNeuroNet(car(i)\net)
      EndIf
      car(i)\rotation=0
      If Random(1)=0
         car(i)\rotation=180
      EndIf
      car(i)\pos\x=#startx
      car(i)\pos\y=#starty
      car(i)\dead=0
      car(i)\fitness=0
      car(i)\couleur=RGB(Random(155)+100,Random(155)+100,Random(155)+100)   
   Next i
   Fitness=0
   Maxfitness=0
EndProcedure

Repeat
   Repeat:Until WindowEvent()=0
   Delay(3)
   FlipBuffers()
   ExamineKeyboard()
   DisplaySprite(#track,0,0)
   If PopulationAlive=0
      ResetPopulation()
   EndIf
   If PopulationAlive:UpdatePhysic():EndIf
   For i=0 To #PopulationSize-1
      If car(i)\dead=1:Continue:EndIf
      StartDrawing(ScreenOutput())
      For t=0 To #laserNumber-1
         LineXY(SpriteWidth(#car)/2+car(i)\pos\x,SpriteHeight(#car)/2+car(i)\pos\y,SpriteWidth(#car)/2+car(i)\pos\x+car(i)\laser[t]\dist*Cos(Radian(car(i)\laser[t]\angle+car(i)\rotation)),SpriteHeight(#car)/2+car(i)\pos\y+car(i)\laser[t]\dist*Sin(Radian(car(i)\laser[t]\angle+car(i)\rotation)),RGB(120,120,120))
         DrawText(0,0,"Generation : "+Str(gen))
         DrawText(0,20,"Time on track (fitness) : "+Str(fitness))
         DrawText(0,40,"Population alived : "+Str(PopulationAlive))
         DrawText(0,60,Str(#laserNumber+3)+" neurons, "+Str(#laserNumber)+" inputs")
      Next t
      StopDrawing()
      
      RotateSprite(#car,car(i)\rotation,#PB_Absolute)
      DisplayTransparentSprite(#car,car(i)\pos\x,car(i)\pos\y,255,car(i)\couleur)
   Next i
Until KeyboardPushed(#PB_Key_Escape)

Picture you'll need "circuit.png"
Image

https://www.youtube.com/watch?v=Jvrfbu0Glyo&t=

Edit: modify genetic algo
Edit2: modify Dll: add bias.
Last edited by Fig on Tue Oct 30, 2018 9:18 pm, edited 10 times in total.
There are 2 methods to program bugless.
But only the third works fine.

Win10, Pb x64 5.71 LTS
User avatar
Bisonte
Addict
Addict
Posts: 1226
Joined: Tue Oct 09, 2007 2:15 am

Re: Basic artificial neural network evolution: cars on a tra

Post by Bisonte »

Very nice work !

On my first test, it was generation 45 that will complete the track....
PureBasic 6.04 LTS (Windows x86/x64) | Windows10 Pro x64 | Asus TUF X570 Gaming Plus | R9 5900X | 64GB RAM | GeForce RTX 3080 TI iChill X4 | HAF XF Evo | build by vannicom​​
English is not my native language... (I often use DeepL to translate my texts.)
coco2
Enthusiast
Enthusiast
Posts: 368
Joined: Mon Nov 25, 2013 5:38 am
Location: Australia

Re: Basic artificial neural network evolution: cars on a tra

Post by coco2 »

Very nice works well
User avatar
pdwyer
Addict
Addict
Posts: 2813
Joined: Tue May 08, 2007 1:27 pm
Location: Chiba, Japan

Re: Basic artificial neural network evolution: cars on a tra

Post by pdwyer »

Fig,

I have some ANN code that has backprop, I think an earlier version is posted somewhere here but I can dig up a more recent one that handles 2 hidden layers

I've recently been getting back into the machine learning thing and am thinking of trying to implement a CNN https://en.wikipedia.org/wiki/Convoluti ... al_network since I have some new uses but these things are getting more complex and time consuming. Also, I might want to bite off something easier first like improving what I have.

I've often wondered about using a GA like this so I will spend some time to understand this, Thanks!!

I don't suppose you would be willing to collaborate on something in this space? PB has no libs in this area

(Offer is open to everyone of course)
Paul Dwyer

“In nature, it’s not the strongest nor the most intelligent who survives. It’s the most adaptable to change” - Charles Darwin
“If you can't explain it to a six-year old you really don't understand it yourself.” - Albert Einstein
Post Reply