CT/MR scans > 3D viewer

[marc_256]

A result of my post in the "off topic" section.
http://www.purebasic.fr/english/viewtop ... 17&t=59197

My project:

1) Converter to enlarge the gray-scale images size and contrast.
2) Convert this 2D images to a 3D mesh
3) Build a 2D CT/MT images to 3D Mesh viewer to see/detect early stage diseases.

next step is to read and convert directly from DICOM format to bmp gray-scale images.

some first results
First results - http://www.marc-systems.be/scan_view_001.png
Last results - http://www.marc-systems.be/new_2010/Vie ... amples.htm

marc,

Every help is welcome...

[Mythros]

If this works for any 2d image COMPLETELY, im talking textures as well, then count me in!

[mpz]

Hi Marc_256,

is it possible to load the "150 gray-scale images of (518x518) = 40 k Points" to a server (http://uploaded.net)? I will test to convert these pictures to a 40k points mesh

like the following mesh (= 100k Pointmesh). I thing this ist not difficult to realize

http://www.purebasic.fr/english/viewtop ... 00#p433109

nest step ist to makes moves with the camera and export a 3d film for 3d TVs or change grey color to color in realtime ...

Greetings Michael

[marc_256]

Hi Mythros and mpz,

thanks for the help,
yes, I gone a need a lot of stuff ...

@mpz
I go try to make some convertions ... (10 images to start with)

The difficulty I have now is,
that the scans are made every 3.1mm distance,
and if I zoom in and out, I get some moire effect.



* For now, the over-night version of my 3D viewer (test is the skeleton selection algorithm)
so from the 150 - 2D CT scan images, I can now select the skeleton.
Here you can see my mom almost naked (80+).



marc,

[marc_256]

@mpz,

I think you made a calculation error here,

518 px * 518 px * 150 images = 40 248 600 pixels/mesh
this is over the 40 million pixels

marc

[Mythros]

Can you release the source code, marc? I would like to mess with it myself. Is there a way you can create triangles for the point cloud, based on the color of each pixel? For instance, you have:

Code: Select all

 .
. .

=
   |
  / \
 /   \
/_____\

Each color uses a heightmap-like algorithm in order to grab the correct height for each of the points in the pointcloud and thus allows for a TRUE 3D mesh.

[marc_256]

@Mythros,

I'm looking for it,
and found http://en.wikipedia.org/wiki/MeshLab

But it looks very difficult to do ??

The source code is very messy for now...need to clean up first.

[Mythros]

We should be able to just turn the image into a point cloud, then use the colors of each pixel to establish a "height" for the 3D mesh. Then, for the depth, use the light or color (not sure which) of either the 3d mesh or the color of each point in order to connect each point to each other point, forming triangles.

We basically just need to give each 2d image to 3d model its own depth in a correct sense. Basically, make it pop out with not just the points, but curves too. Instead of creating a "spikey" 3d object, it would give the correct depth for each 3d model created.

[marc_256]

We should be able to just turn the image into a point cloud, then use the colors of each pixel to establish a "height" for the 3D mesh. Then, for the depth, use the light or color (not sure which) of either the 3d mesh or the color of each point in order to connect each point to each other point, forming triangles.

We basically just need to give each 2d image to 3d model its own depth in a correct sense. Basically, make it pop out with not just the points, but curves too. Instead of creating a "spikey" 3d object, it would give the correct depth for each 3d model created.

Thats what I did already, Step 1 => I convert the 2D images to a point cloud.
But if you will connect all the same colored points, there will be >100000 of triangles...within the same color.

[IdeasVacuum]

I think having a high density of points might be enough, without meshing?

[marc_256]

I think having a high density of points might be enough, without meshing?

Yes, you are right, but finding high resolution CT scans is the problem here.
The ones I have are 518x518 px for +-500mm => resolution => +-1mm/px
and are 3.1 mm distance between each scan.

[Mythros]

It doesn't matter the amount of triangles, just as long as all the triangles are interconnected, then we can have an actual 3D mesh. And we don't connect the same colored triangles. We connect all triangles to each other.

[IdeasVacuum]

Mythos, the number of mesh faces does matter, it matters a lot, because viewing performance can be severely hampered. This is a known issue in CAD/CAM apps for example.

Marc, 1mm/px is not too bad? The main issue is the 3.1mm distance between slices. I think it is possible to vary this on the machine itself, but 3.1mm is probably already the minimum distance. The more slices, the longer it takes and that has two issues - firstly, being scanned is stressful for the patient and secondly there are not enough machines to meet demand so the fastest possible scan session is always a requirement.

[marc_256]

It doesn't matter the amount of triangles, just as long as all the triangles are interconnected, then we can have an actual 3D mesh. And we don't connect the same colored triangles. We connect all triangles to each other.

Like I wrote here,

518 px * 518 px * 150 images = 40 248 600 pixels/mesh
this is over the 40 million pixels

If you want to connect all the points, this gone a give a lot of triangles

Mythos, the number of mesh faces does matter, it matters a lot, because viewing performance can be severely hampered. This is a known issue in CAD/CAM apps for example.

Marc, 1mm/px is not too bad? The main issue is the 3.1mm distance between slices.
I think it is possible to vary this on the machine itself, but 3.1mm is probably already the minimum distance. The more slices, the longer it takes and that has two issues - firstly, being scanned is stressful for the patient and secondly there are not enough machines to meet demand so the fastest possible scan session is always a requirement.

It's all money / time, so they slices the scan every 3.1 mm
What I did now, I pushed all the scans together of a 1 pixel distance,
this gives a deformed body, (3 times shorter) but for testing it works.

[Mythros]

Well ok, yes. In THAT way, it DOES matter. HOWEVER. I think if we mess with the below example, we should be able to just remove SOME of the non-needed points, and connect the points as triangles, that DO matter.

For instance, I have messed with a demo in order to **TRY** and create depth, and I actually succeeded 75%.

I would possibly like to see if there is a way to create the effect of taking a cloth, and putting it on an ordinary, every day object such as an apple. The cloth would then assume the shape of the apple, therefore, creating a 3d model out of the "2d" image. Let me know when you guys come up with something.

Without further adieu, here's the source:

Code: Select all

Declare DrawMatrix()

#Mode = 7 ; #Mode = 1 or #Mode = 7

    Structure PointVertex
      x.f
      y.f
      z.f
      CompilerIf #Mode = 7
        Size.f ;--- Only if Mode = 7
      CompilerEndIf
      Color.l;d3dcolor
    EndStructure

ExamineDesktops()
MP_Graphics3D (DesktopWidth(0),DesktopHeight(0),0,3) ; Erstelle ein WindowsFenster #Window = 0
MP_Viewport(0,0,DesktopWidth(0),DesktopHeight(0)-5)

SetWindowTitle(0, "Pic_to_Mesh .. press Up, Down keys to move mesh forward/back | Left, Right keys to turn mesh left/right")

camera=MP_CreateCamera()

light=MP_CreateLight(2)
MP_LightSetColor (light, RGB(255,255,255))

texture2 = MP_CreateTextureColor(256, 256, MP_ARGB(0,255,255,255))
texture = MP_CreateTextureColor(256, 256, MP_ARGB(0,255,0,0))

plane = MP_CreatePlane(10, 10)
MP_EntitySetTexture(plane, texture2)
MP_RotateEntity(plane, 90, 0, 90)
MP_PositionEntity(plane, 0, -5, 0)

;MP_PositionCamera(camera, 0, 5, 20)
;MP_CameraLookAt(camera,0,0,0)

MP_PositionEntity(light, 0, 10, 20)
MP_EntityLookAt(light,0,0,0)
;==============================================================

UseJPEGImageDecoder()

; load image
Global image_id.i

image_id = LoadImage(#PB_Any, GetCurrentDirectory()+"woman.bmp")

;Dim image_pixel_data.b( ImageWidth(image_id), ImageHeight(image_id) )
Global size = Int(ImageWidth(image_id)*ImageHeight(image_id))
Global Entity= MP_CreatePrimitives (size, #Mode)

Global pointcloud_point_size.f = 1.50

DrawMatrix()
;==============================================================

MP_PositionEntity(light, 0 , 10, 10)
MP_EntityLookAt(light,0,0,0)

z.f=10
While Not MP_KeyDown(#PB_Key_Escape) And Not WindowEvent() = #PB_Event_CloseWindow
 
  If MP_KeyDown(#PB_Key_Up)
    z.f - 0.1
  ElseIf MP_KeyDown(#PB_Key_Down)
    z.f + 0.1
  EndIf
  
  If MP_KeyDown(#PB_Key_Left)
    MP_TurnEntity(Entity,0, 1, 0)
  ElseIf MP_KeyDown(#PB_Key_Right)
    MP_TurnEntity(Entity,0, -1, 0)
  EndIf
 
    MP_PositionEntity(Entity, 0, 10, z)
   
  MP_RenderWorld()
   
  MP_Flip ()

Wend


 Procedure DrawMatrix()
 i=-1
*Memory = MP_GetMemPrimitives(Entity)

StartDrawing( ImageOutput( image_id ) )

For y = 0 To ImageHeight(image_id) - 1
  For x = 0 To ImageWidth(image_id) - 1
    r = Red(Point( x, y ))
    g = Green(Point( x, y ))
    b = Blue(Point( x, y ))
   
    ;r = MP_Red(Point( x, y ))
    ;g = MP_Green(Point( x, y ))
    ;b = MP_Blue(Point( x, y ))
   
    i+1
   
    *myvertex.PointVertex = *Memory + i * SizeOf(PointVertex)
   
     *myvertex\x = x - ImageWidth(image_id)/2
     *myvertex\y = y
     
     If Y > ImageHeight(image_id)/2
        *myvertex\z = (MP_Red(Point( x, y )-8))
      EndIf
      
     *myvertex\Size = pointcloud_point_size
     *myvertex\color = MP_ARGB(0,r,g,b)
  Next x
  Next y
  StopDrawing()
  MP_ScaleEntity(Entity, 0.03, 0.03, 0.03)
  ;MP_MeshSetAlpha(Entity,2)
  MP_PositionEntity(Entity, 0, 0, 0)
  MP_RotateEntity(Entity,180,0,0)
 
  MP_CloseMemPrimitives(Entity)
  
  EndProcedure

I have also included the image I used:




Rename the above image to "woman.bmp" and make SURE that the file type is INDEED bitmap & not JPG.

Also, can someone tweak this code so that it gives depth to more than just half the image?

Thanks again, and I hope to continue this research with you guys!

Sincerely,

Mythros